From: Richard Purdie <richard.purdie@linuxfoundation.org>
To: bitbake-devel@lists.openembedded.org
Subject: [PATCH 2/2] runqueue: Optimise performance of setscene task execution
Date: Sat, 10 Feb 2024 23:04:42 +0000 [thread overview]
Message-ID: <20240210230442.3633663-2-richard.purdie@linuxfoundation.org> (raw)
In-Reply-To: <20240210230442.3633663-1-richard.purdie@linuxfoundation.org>
"time bitbake world -n" shows a lot of time being spent trying to execute the
setscene tasks when many of them are being skipped due to not being present.
The challenge is the "return True" back into the mainloop restarts the search
back at the start of the task list particularly when there are large numbers of
deferred tasks due to hard dependencies.
In this case we can use continue to continue within the list at the expense of
not returning so quickly to loop back through the main loop. This does significantly
improve performance in real world usage scenarios and we can probably afford the
potential loop starvation.
Thanks for the suggestion from Alexander Kanavin.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
lib/bb/runqueue.py | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/lib/bb/runqueue.py b/lib/bb/runqueue.py
index 95cab5132f..85669b80ee 100644
--- a/lib/bb/runqueue.py
+++ b/lib/bb/runqueue.py
@@ -2204,7 +2204,9 @@ class RunQueueExecute:
if nexttask in self.sqdata.outrightfail:
logger.debug2('No package found, so skipping setscene task %s', nexttask)
self.sq_task_failoutright(nexttask)
- return True
+ # Technically we should return True here but that would then start the task list back at the start
+ # which is really bad for performance ("time bitbake world -n"). Instead, keep our place in the tasklist.
+ continue
if nexttask in self.sqdata.unskippable:
logger.debug2("Setscene task %s is unskippable" % nexttask)
task = nexttask
--
2.40.1
next prev parent reply other threads:[~2024-02-10 23:04 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-02-10 23:04 [PATCH 1/2] runqueue: Optimise taskname lookups in next_buildable_task Richard Purdie
2024-02-10 23:04 ` Richard Purdie [this message]
[not found] ` <17B2A2DF1E287224.588@lists.openembedded.org>
2024-02-11 6:17 ` [bitbake-devel] [PATCH 2/2] runqueue: Optimise performance of setscene task execution Richard Purdie
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240210230442.3633663-2-richard.purdie@linuxfoundation.org \
--to=richard.purdie@linuxfoundation.org \
--cc=bitbake-devel@lists.openembedded.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).