From: Yoann CONGAL <yoann.congal@smile.fr>
To: yocto@lists.yoctoproject.org
Subject: Re: [yocto-autobuilder2][PATCH] schedulers: metrics: Start again with master for the daily full download
Date: Sat, 16 Mar 2024 09:25:44 +0100 [thread overview]
Message-ID: <CAMSfU+5HT6vYhbc-31PVa__m_kEPj4a_wVc_586JixwmdN5xAg@mail.gmail.com> (raw)
In-Reply-To: <20240315232921.2251458-1-yoann.congal@smile.fr>
[-- Attachment #1: Type: text/plain, Size: 4092 bytes --]
(Replying to myself to keep conclusion of an IRC discussion)
Le sam. 16 mars 2024 à 00:29, Yoann Congal <yoann.congal@smile.fr> a écrit :
> Change the order of metrics branches:
> * First, master at 5am which may do the full CVE download (might take
> 1 hour to finish).
> * Then, the other branches starting at 6am to allow them to use the
> fresh download.
> * And, finally, generate global charts data from individual branch
> metrics (at 8am)
>
That would not work since the master metrics can go well above 1h (e.g.
https://autobuilder.yoctoproject.org/typhoon/#/builders/138/builds/1343)
What we want is:
* A triggering builder (see
https://docs.buildbot.net/3.8.0/manual/configuration/steps/trigger.html)
that
* for each branch,
* trigger a builder that do the run-cvecheck (starting with master for
the NVD database download)
* Finish with the chart generation (run-cvechart)
That way, we will have proper ordering/dependencies between metrics
computing and chart generation.
NB: Trying to order builders on the AB based on estimated build duration
does not work because workers are quite loaded and the pressure regulation
can extend the build time quite a bit.
> Compared with commit 839c3cedbf6f ("schedulers: metrics: Start with master
> for the daily full download"),
> the last part is new.
>
> Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
> ---
>
> schedulers.py | 9 +++++++--
> 1 file changed, 7 insertions(+), 2 deletions(-)
>
> diff --git a/schedulers.py b/schedulers.py
> index a30383f..67cc739 100644
> --- a/schedulers.py
> +++ b/schedulers.py
> @@ -497,12 +497,17 @@
> schedulers.append(sched.Nightly(name='nightly-meta-oe-mirror',
> branch='master',
> schedulers.append(sched.Nightly(name='nightly-reproducible-meta-oe',
> branch='master', properties=parent_default_props('reproducible-meta-oe'),
> builderNames=['reproducible-meta-oe'], hour=2,
> minute=0, dayOfWeek=5))
>
> -# Run metrics at 3-7am each day
> +# Run metrics at 5-8am each day
> +# Starts with master which may do a full CVE database download and start
> other
> +# branches 1 hour later to allow them to re-use the fresh download.
> +schedulers.append(sched.Nightly(name='nightly-metrics', branch='master',
> properties=parent_default_props('metrics'), builderNames=['metrics'],
> hour=5, minute=0))
> schedulers.append(sched.Nightly(name='nightly-metrics-dunfell',
> branch='dunfell', properties=parent_default_props('metrics', 'dunfell'),
> builderNames=['metrics'], hour=6, minute=0, codebases = {'' : {'branch' :
> 'dunfell'}}))
> schedulers.append(sched.Nightly(name='nightly-metrics-kirkstone',
> branch='kirkstone', properties=parent_default_props('metrics',
> 'kirkstone'), builderNames=['metrics'], hour=6, minute=15, codebases = {''
> : {'branch' : 'kirkstone'}}))
> schedulers.append(sched.Nightly(name='nightly-metrics-mickledore',
> branch='mickledore', properties=parent_default_props('metrics',
> 'mickledore'), builderNames=['metrics'], hour=6, minute=30, codebases = {''
> : {'branch' : 'mickledore'}}))
> schedulers.append(sched.Nightly(name='nightly-metrics-nanbield',
> branch='nanbield', properties=parent_default_props('metrics', 'nanbield'),
> builderNames=['metrics'], hour=6, minute=45, codebases = {'' : {'branch' :
> 'nanbield'}}))
> -schedulers.append(sched.Nightly(name='nightly-metrics', branch='master',
> properties=parent_default_props('metrics'), builderNames=['metrics'],
> hour=7, minute=0))
> +# Generate charts after every branch had its metrics computed
> +schedulers.append(sched.Nightly(name='nightly-metrics-charts',
> branch='master', properties=parent_default_props('metrics'),
> builderNames=['metrics-charts'], hour=8, minute=0))
> +
>
> # Run check-layer-nightly twice a week for nanbield
> schedulers.append(sched.Nightly(name='nightly-check-layer-nanbield',
> properties=parent_default_props('check-layer-nightly', 'nanbield'),
> --
> 2.39.2
>
>
--
Yoann Congal
Smile ECS - Tech expert
[-- Attachment #2: Type: text/html, Size: 5710 bytes --]
prev parent reply other threads:[~2024-03-16 8:26 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-03-15 23:29 [yocto-autobuilder2][PATCH] schedulers: metrics: Start again with master for the daily full download Yoann Congal
2024-03-16 8:25 ` Yoann CONGAL [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAMSfU+5HT6vYhbc-31PVa__m_kEPj4a_wVc_586JixwmdN5xAg@mail.gmail.com \
--to=yoann.congal@smile.fr \
--cc=yocto@lists.yoctoproject.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).