From: Roro Cui <roro.cui@broadcom.com>
To: roro.cui@broadcom.com
Cc: Richard Purdie <richard.purdie@linuxfoundation.org>,
bitbake-devel@lists.openembedded.org
Subject: Re: [bitbake-devel] Got "ValueError: filedescriptor out of range in select" on bitbake v1.46.0
Date: Fri, 12 Jan 2024 12:36:54 -0500 [thread overview]
Message-ID: <CAHBa6FPyPG9oHroszumiFT-TahsmPWV8a6NNJ4e8Di48608O4g@mail.gmail.com> (raw)
In-Reply-To: <17A9A9BBB897E15F.31870@lists.openembedded.org>
[-- Attachment #1.1: Type: text/plain, Size: 6559 bytes --]
hi, Richard
Setting BB_NUMBER_PARSE_THREADS to 256 fixes my issue.
I am going to tune that value to see which max number triggers this kind of
problem.
Thank you very much for your help.
Roro
On Fri, Jan 12, 2024 at 12:29 PM Roro Cui via lists.openembedded.org
<roro.cui=broadcom.com@lists.openembedded.org> wrote:
> hi, Richard
>
> Thank you for your reply.
> I tried BB_NUMBER_THREADS before, that does not work, since this issue
> happens during 'parsing recipes' stage.
> But I guess BB_NUMBER_PARSE_THREADS will work, let me try it.
> Will update you soon.
> roro
>
> On Fri, Jan 12, 2024 at 12:23 PM Richard Purdie <
> richard.purdie@linuxfoundation.org> wrote:
>
>> On Fri, 2024-01-12 at 11:40 -0500, Roro Cui via lists.openembedded.org
>> wrote:
>> > hi, Dear Sir or Madam
>> >
>> > I hit this issue when building the target in yocto dunfell on our new
>> server(512 cpu cores)
>> > The cooker daemon process quits abnormally. Here is the output
>> >
>> > Command Completed
>> > Running command ['getVariable', 'BBINCLUDELOGS']
>> > Command Completed
>> > Running command ['getVariable', 'BBINCLUDELOGS_LINES']
>> > Command Completed
>> > Running command ['getSetVariable', 'BB_CONSOLELOG']
>> > Command Completed
>> > Running command ['getSetVariable', 'BB_LOGCONFIG']
>> > Command Completed
>> > Running command ['getUIHandlerNum']
>> > Command Completed
>> > Running command ['setEventMask', 1, 20, {'BitBake.RunQueue.HashEquiv':
>> 19, 'BitBake.SigGen.HashEquiv': 19}, ['bb.runqueue.runQueueExitWait',
>> 'bb.event.LogExecTTY', 'logging.LogRecord', 'bb.build.TaskFailed',
>> 'bb.build.TaskBase', 'bb.event.ParseStarted', 'bb.event.ParseProgress',
>> 'bb.event.ParseCompleted', 'bb.event.CacheLoadStarted',
>> 'bb.event.CacheLoadProgress', 'bb.event.CacheLoadCompleted',
>> 'bb.command.CommandFailed', 'bb.command.CommandExit',
>> 'bb.command.CommandCompleted', 'bb.cooker.CookerExit',
>> 'bb.event.MultipleProviders', 'bb.event.NoProvider',
>> 'bb.runqueue.sceneQueueTaskStarted', 'bb.runqueue.runQueueTaskStarted',
>> 'bb.runqueue.runQueueTaskFailed', 'bb.runqueue.sceneQueueTaskFailed',
>> 'bb.event.BuildBase', 'bb.build.TaskStarted', 'bb.build.TaskSucceeded',
>> 'bb.build.TaskFailedSilent', 'bb.build.TaskProgress',
>> 'bb.event.ProcessStarted', 'bb.event.ProcessProgress',
>> 'bb.event.ProcessFinished']]
>> > Command Completed
>> > Running command ['setConfig', 'cmd', 'fetch']
>> > Command Completed
>> > Running command ['buildTargets', ['quilt-native'], 'fetch']
>> > Command Completed
>> > Process ProcessServer-1:
>> > Traceback (most recent call last):
>> > File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in
>> _bootstrap
>> > self.run()
>> > File
>> "/local/home1/rorocui/yocto/gr_ml_20240105/openembedded-core/bitbake/lib/bb/server/process.py",
>> line 115, in run
>> > ret = self.main()
>> > File
>> "/local/home1/rorocui/yocto/gr_ml_20240105/openembedded-core/bitbake/lib/bb/server/process.py",
>> line 232, in main
>> > ready = self.idle_commands(.1, fds)
>> > File
>> "/local/home1/rorocui/yocto/gr_ml_20240105/openembedded-core/bitbake/lib/bb/server/process.py",
>> line 334, in idle_commands
>> > return select.select(fds,[],[],nextsleep)[0]
>> > ValueError: filedescriptor out of range in select()
>> > Var <PATH[:=]>:1: SyntaxWarning: "is not" with a literal. Did you mean
>> "!="?
>> >
>> > Is this a known issue?
>>
>> It isn't an issue we've been tracking, no. It sounds like your 512 core
>> machine has probably run more tasks that bitbake can manage correctly.
>> Try lower values of BB_NUMBER_THREADS and BB_NUMBER_PARSE_THREADS?
>>
>> You could also open a bug in bugzilla. There are probably ways to
>> improve the code to avoid select() calls but those will be invasive.
>>
>> Cheers,
>>
>> Richard
>>
>
>
> --
>
> Regards
>
> Roro Cui
>
> Principal Software Engineer, Software Engineering | STB & CM
>
> *Broadcom*
>
>
>
> *office:* 678.475.2897(X72897) | *fax:* 678.475.2888
>
> 4385 River Green Parkway | Duluth, GA 30096
>
> roro.cui@broadcom.com <danny.zhang@broadcom.com> | broadcom.com
>
>
>
>
> This electronic communication and the information and any files
> transmitted with it, or attached to it, are confidential and are intended
> solely for the use of the individual or entity to whom it is addressed and
> may contain information that is confidential, legally privileged, protected
> by privacy laws, or otherwise restricted from disclosure to anyone else. If
> you are not the intended recipient or the person responsible for delivering
> the e-mail to the intended recipient, you are hereby notified that any use,
> copying, distributing, dissemination, forwarding, printing, or copying of
> this e-mail is strictly prohibited. If you received this e-mail in error,
> please return the e-mail to the sender, delete it from your computer, and
> destroy any printed copy of it.
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#15791):
> https://lists.openembedded.org/g/bitbake-devel/message/15791
> Mute This Topic: https://lists.openembedded.org/mt/103686457/8130911
> Group Owner: bitbake-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [
> roro.cui@broadcom.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>
--
Regards
Roro Cui
Principal Software Engineer, Software Engineering | STB & CM
*Broadcom*
*office:* 678.475.2897(X72897) | *fax:* 678.475.2888
4385 River Green Parkway | Duluth, GA 30096
roro.cui@broadcom.com <danny.zhang@broadcom.com> | broadcom.com
--
This electronic communication and the information and any files transmitted
with it, or attached to it, are confidential and are intended solely for
the use of the individual or entity to whom it is addressed and may contain
information that is confidential, legally privileged, protected by privacy
laws, or otherwise restricted from disclosure to anyone else. If you are
not the intended recipient or the person responsible for delivering the
e-mail to the intended recipient, you are hereby notified that any use,
copying, distributing, dissemination, forwarding, printing, or copying of
this e-mail is strictly prohibited. If you received this e-mail in error,
please return the e-mail to the sender, delete it from your computer, and
destroy any printed copy of it.
[-- Attachment #1.2: Type: text/html, Size: 11816 bytes --]
[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4197 bytes --]
prev parent reply other threads:[~2024-01-12 17:37 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-01-12 16:40 Got "ValueError: filedescriptor out of range in select" on bitbake v1.46.0 Roro Cui
2024-01-12 17:23 ` [bitbake-devel] " Richard Purdie
2024-01-12 17:29 ` Roro Cui
[not found] ` <17A9A9BBB897E15F.31870@lists.openembedded.org>
2024-01-12 17:36 ` Roro Cui [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAHBa6FPyPG9oHroszumiFT-TahsmPWV8a6NNJ4e8Di48608O4g@mail.gmail.com \
--to=roro.cui@broadcom.com \
--cc=bitbake-devel@lists.openembedded.org \
--cc=richard.purdie@linuxfoundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).