From: Richard Purdie <richard.purdie@linuxfoundation.org>
To: Alexander Kanavin <alex.kanavin@gmail.com>
Cc: Khem Raj <raj.khem@gmail.com>,
bitbake-devel <bitbake-devel@lists.openembedded.org>,
Randy MacLeod <rwmacleod@gmail.com>,
Mark Hatle <mark.hatle@kernel.crashing.org>
Subject: Re: [bitbake-devel] regression of 'world' performance?
Date: Wed, 14 Feb 2024 08:05:52 +0000 [thread overview]
Message-ID: <2845e2a61d7383c4716e1881fe2e6e3feb16464c.camel@linuxfoundation.org> (raw)
In-Reply-To: <CANNYZj83d9i0prBSDBwLu=SSS57p7YQVyWKvXphBts_yHBghCg@mail.gmail.com>
On Wed, 2024-02-14 at 07:59 +0100, Alexander Kanavin wrote:
> On Tue, 13 Feb 2024 at 13:41, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > We can work out how it compares to what we'd expect. If you generate
> > task-depends.dot and count the number of dependencies between tasks,
> > I'd expect the number of appends in build_taskdepdata to be be
> > approximately:
> >
> > No. Tasks * No. Dependencies * 0.5
> >
> > The half is assuming that the number of dependencies a task as scales
> > linearly which it doesn't but should give a rough number. It certainly
> > shouldn't be more than tasks * dependencies as an upper bound.
>
> Here's what task-depends.dot contains:
>
> alex@Zen2:/srv/storage/alex/yocto/build-metaoe$ grep label task-depends.dot |wc
> 35783 107349 5655551
> alex@Zen2:/srv/storage/alex/yocto/build-metaoe$ grep -v label
> task-depends.dot |wc
> 145771 437311 9299716
>
> So 35000 tasks, and 145000 dependencies. If one is multiplied by the
> other it becomes 5 billion, which is much higher than 90 million
> appends. Not sure what to make of that.
Looked at another way, the 90 million calls equates to about 2500
dependencies per task, which probably isn't unreasonable when you
consider how many a single command line tool has (bash, sed or
whatever).
That taskdep data is passed into the task in question so it is allowed
to "see" any dependencies but not anything of any other task as that
would not be deterministic. That is why the list is rebuilt each call.
Whether there is a better way to do what that code is doing, I'm not
sure but the number of calls isn't a red flag in this case.
The amount of time spent in the next_task function is a lot more
significant though.
Cheers,
Richard
next prev parent reply other threads:[~2024-02-14 8:06 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-01-12 10:57 regression of 'world' performance? Alexander Kanavin
2024-01-12 12:13 ` [bitbake-devel] " Richard Purdie
2024-01-12 12:27 ` Alexander Kanavin
2024-01-12 22:19 ` Khem Raj
2024-02-09 15:23 ` Alexander Kanavin
2024-02-09 15:43 ` Richard Purdie
2024-02-09 15:59 ` Alexander Kanavin
2024-02-09 16:05 ` Richard Purdie
2024-02-09 16:16 ` Alexander Kanavin
2024-02-09 16:22 ` Richard Purdie
2024-02-09 17:02 ` Alexander Kanavin
[not found] ` <17B24084174823DE.15017@lists.openembedded.org>
2024-02-09 20:44 ` Alexander Kanavin
2024-02-09 21:43 ` chris.laplante
2024-02-09 22:50 ` chris.laplante
2024-02-09 23:33 ` Richard Purdie
[not found] ` <17B255DB2CE33879.588@lists.openembedded.org>
2024-02-09 23:59 ` Richard Purdie
[not found] ` <17B257441406400D.588@lists.openembedded.org>
2024-02-10 15:16 ` Richard Purdie
[not found] ` <17B2894C95E031D0.14481@lists.openembedded.org>
2024-02-10 20:42 ` Richard Purdie
[not found] ` <17B29B2072FF51A8.14481@lists.openembedded.org>
2024-02-10 20:52 ` Richard Purdie
2024-02-12 12:58 ` Alexander Kanavin
2024-02-13 8:57 ` Richard Purdie
2024-02-13 9:42 ` Alexander Kanavin
2024-02-13 12:41 ` Richard Purdie
2024-02-14 6:59 ` Alexander Kanavin
2024-02-14 8:05 ` Richard Purdie [this message]
2024-02-13 10:49 ` Ross Burton
2024-02-13 12:31 ` Richard Purdie
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=2845e2a61d7383c4716e1881fe2e6e3feb16464c.camel@linuxfoundation.org \
--to=richard.purdie@linuxfoundation.org \
--cc=alex.kanavin@gmail.com \
--cc=bitbake-devel@lists.openembedded.org \
--cc=mark.hatle@kernel.crashing.org \
--cc=raj.khem@gmail.com \
--cc=rwmacleod@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).