From: Randy MacLeod <randy.macleod@windriver.com>
To: engineering@neighbourhood.ie,
openembedded-core@lists.openembedded.org,
Ninette Adhikari <ninette@thehoodiefirm.com>
Cc: richard.purdie@linuxfoundation.org
Subject: Re: [PATCH v2 0/5] Improvements for performance test report view
Date: Fri, 3 May 2024 13:22:23 -0400 [thread overview]
Message-ID: <350638b5-9151-4e7a-993a-cdaf042f50f7@windriver.com> (raw)
In-Reply-To: <20240503144340.27385-1-ninette@thehoodiefirm.com>
[-- Attachment #1: Type: text/plain, Size: 8150 bytes --]
On 2024-05-03 10:43 a.m., Ninette Adhikari wrote:
> This work is done according to "Milestone 9: Build performance test report view" as stated in the Scope of Work with Sovereign Tech Fund (STF) (https://www.sovereigntechfund.de/).
> The current report can be accessed here:
> Performance test report HTML (https://autobuilder.yocto.io/pub/non-release/20240117-15/testresults/buildperf-alma8/perf-alma8_master_20240117090048_663f180574.html)
> The report is created using the `oe-build-perf-report` script in the poky repository. This script generates a performance test report in HTML format using the data from the yocto-buildstats (https://git.yoctoproject.org/yocto-buildstats/) repository.
> The report displays the performance test results in line chart format. The chart x-axis represents the commit numbers, and the y-axis represents the test duration in minutes.
> The report also includes a table that displays the measurement statistics data for each test. The report is interactive and allows users to zoom in on specific sections of the line chart.
>
> The current report format required some updates to make it more interactive and user-friendly. And this patch addresses such improvements:
>
> - Add [Apache echart](https://echarts.apache.org/en/index.html) library to create oe build performance report charts and make them interactive.
> - Restructure data to time and value array format to be used by echarts. It also converts test duration to minutes and adds zoom to the charts.
> - Update measurement statistics data to include `start_time` so that time can be displayed instead of commit numbers on the chart. It also updates default commit history length to 300.
> - Add styling updates including page margin, labels for x and y axis, tooltip, and section descriptions.
The tooltips are very nice and useful so
> - The charts are created as step charts instead of plain line charts.
> - Add start time, size, and commit number to tooltip.
> - Add dark mode view
>
> Updated report screenshots:
> https://github.com/neighbourhoodie/poky/assets/13760198/65a1890c-fd2a-40d4-ac90-f13055735e53
> https://github.com/neighbourhoodie/poky/assets/13760198/b40c326b-17d2-42e2-8e41-72e52ed2c003
> https://github.com/neighbourhoodie/poky/assets/13760198/cc7ec996-9dab-435a-8fdc-82a2a4193c0a
> https://github.com/neighbourhoodie/poky/assets/13760198/6e0fe09d-50e5-4b0b-b70b-6943f71b5208
Very nice! Thanks for the work and the v2 improvements.
I applied the patches by saving them from email and there was a minor
whitespace warning which
is really nothing to worry about but if you want to avoid such noise,
you could send yourself
patches by email and then apply them in a different branch if you want
to check for such problems.
❯ git am /tmp/ninette-v2/*
Applying: oe-build-perf-report: Add apache echarts to make report
interactive
Applying: oe-build-perf-report: Display more than 300 commits and date
instead of commit number
Applying: oe-build-perf-report: Improve report styling and add descriptions
.git/rebase-apply/patch:240: trailing whitespace.
start_time = time
warning: 1 line adds whitespace errors.
Applying: oe-build-perf-report: Update chart tooltip and chart type
Applying: oe-build-perf-report: Add dark mode
poky.git on ninette-v2 [$?]
❯ ls /tmp/ninette-v2/*
'/tmp/ninette-v2/[OE-core] [PATCH v2 1_5] oe-build-perf-report: Add
apache echarts to make report interactive - "Ninette Adhikari via
lists.openembedded.org"
<ninette=thehoodiefirm.com@lists.openembedded.org> - 2024-05-03 1043.eml'
'/tmp/ninette-v2/[PATCH v2 2_5] oe-build-perf-report: Display more than
300 commits and date instead of commit number - Ninette Adhikari
<ninette@thehoodiefirm.com> - 2024-05-03 1043.eml'
'/tmp/ninette-v2/[PATCH v2 3_5] oe-build-perf-report: Improve report
styling and add descriptions - Ninette Adhikari
<ninette@thehoodiefirm.com> - 2024-05-03 1043.eml'
'/tmp/ninette-v2/[PATCH v2 4_5] oe-build-perf-report: Update chart
tooltip and chart type - Ninette Adhikari <ninette@thehoodiefirm.com> -
2024-05-03 1043.eml'
'/tmp/ninette-v2/[PATCH v2 5_5] oe-build-perf-report: Add dark mode -
Ninette Adhikari <ninette@thehoodiefirm.com> - 2024-05-03 1043.eml'
>
> For local setup, you can do the following:
>
> 1. Clone the yocto-buildstats (https://git.yoctoproject.org/yocto-buildstats/) and the poky repository (https://git.yoctoproject.org/poky/)
>
> 2. In the poky repository run the following to build the report HTML:
> ```bash
> ./scripts/oe-build-perf-report -r "LOCAL_PATH_TO_YOCTO_BUILDSTATS" --branch "master" --commit "663f1805742ff6fb6955719d0ab7846a425debcf" --branch2 "master" --html > test.html
These are the same setup step as last time I think and again, they
aren't quite right but
the script tells you do do:
❯ git fetch origin
refs/notes/buildstats/perf-debian11/master/qemux86:refs/notes/buildstats/perf-debian11/master/qemux86
and that works.
> ```
> Note:
> - Add your local path to the yocto-buildstats repo
> - The above command builds the report in a file called `test.html`. You can access it in the root directory in poky.
> - This exmaple report uses the commit `663f1805742ff6fb6955719d0ab7846a425debcf` from `master` branch.
Hmmm, ah, that's a poky commit id. I (stupidly!) expected it to be a
commit ID in the yocto-buildstats repo.
One thing that you can think about fixing or just ignore is that the
--commit arg needs to be the
full commit hash not the shortened version. i.e. this doesn't work:
❯ ./scripts/oe-build-perf-report -r
"/media/rmacleod/gitter/rmacleod/src/distro/yocto/yocto-buildstats.git"
--branch "master" --commit "663f180574" --branch2 "master" --html >
/tmp/ninette-v2-test-663f180574.html
Traceback (most recent call last):
File
"/media/rmacleod/gitter/rmacleod/src/distro/yocto/poky.git/./scripts/oe-build-perf-report",
line 617, in <module>
sys.exit(main())
File
"/media/rmacleod/gitter/rmacleod/src/distro/yocto/poky.git/./scripts/oe-build-perf-report",
line 532, in main
index1 = gitarchive.rev_find(revs, 'commit', args.commit)
File
"/media/rmacleod/gitter/rmacleod/src/distro/yocto/poky.git/meta/lib/oeqa/utils/gitarchive.py",
line 282, in rev_find
raise ValueError("Unable to find '{}' value '{}'".format(attr, val))
ValueError: Unable to find 'commit' value '663f180574'
Since the script is going to be used in other scripts run on the
autobuilder, this is a low priority enhancement.
I was happy to see that a newer poky commit, worked fine:
❯ ./scripts/oe-build-perf-report -r
"/media/rmacleod/gitter/rmacleod/src/distro/yocto/yocto-buildstats.git"
--branch "master" --commit "632e3170595bb32717c0471a55a619b4b33fe787"
--branch2 "master" --html >
/tmp/ninette-v2-test-632e3170595bb32717c0471a55a619b4b33fe787.html
INFO: Parsing buildstats from
'refs/notes/buildstats/perf-debian11/master/qemux86'
but I did notice: Stdev: nan
whereas in your linked charts, and in my first run with commit:
663f1805742ff6fb6955719d0ab7846a425debcf
the correct numerical standard deviation is displayed.
I can debug this is you can't reproduce it yourself.
Again. these reports and beautiful interactive charts are great to see.
Thanks!
../Randy
>
>
> Ninette Adhikari (5):
> oe-build-perf-report: Add apache echarts to make report interactive
> oe-build-perf-report: Display more than 300 commits and date instead
> of commit number
> oe-build-perf-report: Improve report styling and add descriptions
> oe-build-perf-report: Update chart tooltip and chart type
> oe-build-perf-report: Add dark mode
>
> .../build_perf/html/measurement_chart.html | 140 ++++++++++++------
> scripts/lib/build_perf/html/report.html | 124 +++++++++++-----
> scripts/lib/build_perf/report.py | 5 +-
> scripts/oe-build-perf-report | 6 +-
> 4 files changed, 193 insertions(+), 82 deletions(-)
>
--
# Randy MacLeod
# Wind River Linux
[-- Attachment #2.1: Type: text/html, Size: 12105 bytes --]
[-- Attachment #2.2: ipZPodcPGxdg4E4H.png --]
[-- Type: image/png, Size: 65076 bytes --]
next prev parent reply other threads:[~2024-05-03 17:22 UTC|newest]
Thread overview: 29+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-04-15 14:41 [PATCH 0/3] Improvements for performance test report view Ninette Adhikari
2024-04-15 14:41 ` [PATCH 1/3] oe-build-perf-report: Add apache echarts to make report interactive Ninette Adhikari
2024-04-16 16:39 ` [OE-core] " Ross Burton
2024-04-18 14:23 ` [PATCH v2] oe-build-perf-report: Update chart tooltip format Ninette Adhikari
2024-04-15 14:41 ` [PATCH 2/3] oe-build-perf-report: Display more than 300 commits and date instead of commit number Ninette Adhikari
2024-04-15 14:41 ` [PATCH 3/3] oe-build-perf-report: Improve report styling and add descriptions Ninette Adhikari
2024-04-15 14:52 ` Patchtest results for " patchtest
2024-04-16 14:49 ` [OE-core] [PATCH 0/3] Improvements for performance test report view Richard Purdie
2024-05-03 14:43 ` [PATCH v2 0/5] " Ninette Adhikari
2024-05-03 14:43 ` [PATCH v2 1/5] oe-build-perf-report: Add apache echarts to make report interactive Ninette Adhikari
2024-05-15 15:56 ` Richard Purdie
2024-05-15 16:00 ` [OE-core] " Ross Burton
2024-05-22 14:52 ` [PATCH 0/1] oe-build-perf-report: Update tooltip to include commit link Ninette Adhikari
2024-05-22 14:52 ` [PATCH 1/1] oe-build-perf-report: Add commit hash link to chart tooltip" Ninette Adhikari
2024-05-22 15:03 ` Patchtest results for " patchtest
2024-05-24 16:01 ` Richard Purdie
2024-05-22 15:16 ` [PATCH v2 1/5] oe-build-perf-report: Add apache echarts to make report interactive Ninette Adhikari
2024-05-24 16:02 ` Richard Purdie
2024-05-03 14:43 ` [PATCH v2 2/5] oe-build-perf-report: Display more than 300 commits and date instead of commit number Ninette Adhikari
2024-05-03 14:43 ` [PATCH v2 3/5] oe-build-perf-report: Improve report styling and add descriptions Ninette Adhikari
2024-05-03 14:43 ` [PATCH v2 4/5] oe-build-perf-report: Update chart tooltip and chart type Ninette Adhikari
2024-05-03 14:43 ` [PATCH v2 5/5] oe-build-perf-report: Add dark mode Ninette Adhikari
2024-05-03 17:22 ` Randy MacLeod [this message]
[not found] ` <17CC0A5CB7913FF6.8557@lists.openembedded.org>
2024-05-03 18:10 ` [OE-core] [PATCH v2 0/5] Improvements for performance test report view Randy MacLeod
2024-05-07 14:17 ` Ninette Adhikari
2024-05-08 15:53 ` Randy MacLeod
2024-05-13 14:28 ` Ninette Adhikari
2024-05-15 12:27 ` Ross Burton
2024-04-16 20:40 ` [OE-core] [PATCH 0/3] " Randy MacLeod
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=350638b5-9151-4e7a-993a-cdaf042f50f7@windriver.com \
--to=randy.macleod@windriver.com \
--cc=engineering@neighbourhood.ie \
--cc=ninette@thehoodiefirm.com \
--cc=openembedded-core@lists.openembedded.org \
--cc=richard.purdie@linuxfoundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).