From: Matthew Brost <matthew.brost@intel.com>
To: Tvrtko Ursulin <tvrtko.ursulin@igalia.com>
Cc: Daniel Vetter <daniel@ffwll.ch>, Rob Clark <robdclark@gmail.com>,
"dri-devel@lists.freedesktop.org"
<dri-devel@lists.freedesktop.org>
Subject: Re: drm scheduler and wq flavours
Date: Wed, 8 May 2024 19:07:52 +0000 [thread overview]
Message-ID: <ZjvNiD4CL0fGX1w8@DUT025-TGLU.fm.intel.com> (raw)
In-Reply-To: <55e260b7-12d3-462b-aa08-a939a4ee67ef@igalia.com>
On Tue, May 07, 2024 at 10:09:18AM +0100, Tvrtko Ursulin wrote:
>
> On 07/05/2024 00:23, Matthew Brost wrote:
> > On Thu, May 02, 2024 at 03:33:50PM +0100, Tvrtko Ursulin wrote:
> > >
> > > Hi all,
> > >
> > > Continuing after the brief IRC discussion yesterday regarding work queues
> > > being prone to deadlocks or not, I had a browse around the code base and
> > > ended up a bit confused.
> > >
> > > When drm_sched_init documents and allocates an *ordered* wq, if no custom
> > > one was provided, could someone remind me was the ordered property
> > > fundamental for something to work correctly? Like run_job vs free_job
> > > ordering?
> > >
> >
> > Before the work queue (kthread design), run_job & free_job were ordered.
> > It was decided to not break this existing behavior.
>
> Simply for extra paranoia or you remember if there was a reason identified?
>
Not to break existing behavior. Can dig the entire thread if for
reference if needed.
> > > I ask because it appears different drivers to different things and at the
> > > moment it looks we have all possible combos or ordered/unordered, bound and
> > > unbound, shared or not shared with the timeout wq, or even unbound for the
> > > timeout wq.
> > >
> > > The drivers worth looking at in this respect are probably nouveau, panthor,
> > > pvr and xe.
> > >
> > > Nouveau also talks about a depency betwen run_job and free_job and goes to
> > > create two unordered wqs.
> > >
> > > Then xe looks a bit funky with the workaround/hack for lockep where it
> > > creates 512 work queues and hands them over to user queues in round-robin
> > > fashion. (Instead of default 1:1.) Which I suspect is a problem which should
> > > be applicable for any 1:1 driver given a thorough enough test suite.
> > >
> >
> > I think lockdep ran out of chains or something when executing some wild
> > IGT with 1:1. Yes, any driver with a wild enough test would likely hit
> > this lockdep splat too. Using a pool probably is not bad idea either.
>
> I wonder what is different between that and having a single shared unbound
> queue and let kernel manage the concurrency? Both this..
>
Each action (run_job, free_job, and Xe specific process msg) has its own
work item on the DRM scheduler work queue. In Xe, these options must be
ordered, or strictly speaking, not executed in parallel within the DRM
sched entity/scheduler. With a single shared unbound queue, this breaks.
> > > So anyway.. ordered vs unordered - drm sched dictated or at driver's choice?
> > >
> >
> > Default ordered, driver can override with unordered.
>
> .. and this, go back to my original question - whether the default queue
> must be ordered or not, or under which circustmances can drivers choose
> unordered. I think in drm_sched_init, where kerneldoc says it will create an
> ordered queue, it would be good to document the rules.
>
Sure. Let me write something up.
Matt
> Regards,
>
> Tvrtko
prev parent reply other threads:[~2024-05-08 19:08 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-05-02 14:33 drm scheduler and wq flavours Tvrtko Ursulin
2024-05-06 23:23 ` Matthew Brost
2024-05-07 9:09 ` Tvrtko Ursulin
2024-05-08 19:07 ` Matthew Brost [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ZjvNiD4CL0fGX1w8@DUT025-TGLU.fm.intel.com \
--to=matthew.brost@intel.com \
--cc=daniel@ffwll.ch \
--cc=dri-devel@lists.freedesktop.org \
--cc=robdclark@gmail.com \
--cc=tvrtko.ursulin@igalia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).