From: Jason Ekstrand <jason@jlekstrand.net>
To: Daniel Vetter <daniel@ffwll.ch>
Cc: Intel GFX <intel-gfx@lists.freedesktop.org>,
Maling list - DRI developers <dri-devel@lists.freedesktop.org>
Subject: Re: [Intel-gfx] [PATCH 16/31] drm/i915/gem: Add an intermediate proto_context struct (v4)
Date: Thu, 10 Jun 2021 11:53:48 -0500 [thread overview]
Message-ID: <CAOFGe95osa5vC3UQU6Rhchg7korhfXv0jYsTL0x5gD_-OWeEBA@mail.gmail.com> (raw)
In-Reply-To: <YMJBZXGL3spPGcJa@phenom.ffwll.local>
On Thu, Jun 10, 2021 at 11:44 AM Daniel Vetter <daniel@ffwll.ch> wrote:
>
> On Wed, Jun 09, 2021 at 11:00:26AM -0500, Jason Ekstrand wrote:
> > On Wed, Jun 9, 2021 at 6:28 AM Daniel Vetter <daniel@ffwll.ch> wrote:
> > >
> > > On Tue, Jun 08, 2021 at 11:35:58PM -0500, Jason Ekstrand wrote:
> > > > The current context uAPI allows for two methods of setting context
> > > > parameters: SET_CONTEXT_PARAM and CONTEXT_CREATE_EXT_SETPARAM. The
> > > > former is allowed to be called at any time while the later happens as
> > > > part of GEM_CONTEXT_CREATE. Currently, everything settable via one is
> > > > settable via the other. While some params are fairly simple and setting
> > > > them on a live context is harmless such the context priority, others are
> > > > far trickier such as the VM or the set of engines. In order to swap out
> > > > the VM, for instance, we have to delay until all current in-flight work
> > > > is complete, swap in the new VM, and then continue. This leads to a
> > > > plethora of potential race conditions we'd really rather avoid.
> > > >
> > > > Unfortunately, both methods of setting the VM and the engine set are in
> > > > active use today so we can't simply disallow setting the VM or engine
> > > > set vial SET_CONTEXT_PARAM. In order to work around this wart, this
> > > > commit adds a proto-context struct which contains all the context create
> > > > parameters.
> > > >
> > > > v2 (Daniel Vetter):
> > > > - Better commit message
> > > > - Use __set/clear_bit instead of set/clear_bit because there's no race
> > > > and we don't need the atomics
> > > >
> > > > v3 (Daniel Vetter):
> > > > - Use manual bitops and BIT() instead of __set_bit
> > > >
> > > > v4 (Daniel Vetter):
> > > > - Add a changelog to the commit message
> > > > - Better hyperlinking in docs
> > > > - Create the default PPGTT in i915_gem_create_context
> > > >
> > > > Signed-off-by: Jason Ekstrand <jason@jlekstrand.net>
> > > > ---
> > > > drivers/gpu/drm/i915/gem/i915_gem_context.c | 124 +++++++++++++++---
> > > > .../gpu/drm/i915/gem/i915_gem_context_types.h | 22 ++++
> > > > .../gpu/drm/i915/gem/selftests/mock_context.c | 16 ++-
> > > > 3 files changed, 145 insertions(+), 17 deletions(-)
> > > >
> > > > diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.c b/drivers/gpu/drm/i915/gem/i915_gem_context.c
> > > > index f9a6eac78c0ae..b5d8c1ff5d7b3 100644
> > > > --- a/drivers/gpu/drm/i915/gem/i915_gem_context.c
> > > > +++ b/drivers/gpu/drm/i915/gem/i915_gem_context.c
> > > > @@ -191,6 +191,83 @@ static int validate_priority(struct drm_i915_private *i915,
> > > > return 0;
> > > > }
> > > >
> > > > +static void proto_context_close(struct i915_gem_proto_context *pc)
> > > > +{
> > > > + if (pc->vm)
> > > > + i915_vm_put(pc->vm);
> > > > + kfree(pc);
> > > > +}
> > > > +
> > > > +static int proto_context_set_persistence(struct drm_i915_private *i915,
> > > > + struct i915_gem_proto_context *pc,
> > > > + bool persist)
> > > > +{
> > > > + if (persist) {
> > > > + /*
> > > > + * Only contexts that are short-lived [that will expire or be
> > > > + * reset] are allowed to survive past termination. We require
> > > > + * hangcheck to ensure that the persistent requests are healthy.
> > > > + */
> > > > + if (!i915->params.enable_hangcheck)
> > > > + return -EINVAL;
> > > > +
> > > > + pc->user_flags |= BIT(UCONTEXT_PERSISTENCE);
> > > > + } else {
> > > > + /* To cancel a context we use "preempt-to-idle" */
> > > > + if (!(i915->caps.scheduler & I915_SCHEDULER_CAP_PREEMPTION))
> > > > + return -ENODEV;
> > > > +
> > > > + /*
> > > > + * If the cancel fails, we then need to reset, cleanly!
> > > > + *
> > > > + * If the per-engine reset fails, all hope is lost! We resort
> > > > + * to a full GPU reset in that unlikely case, but realistically
> > > > + * if the engine could not reset, the full reset does not fare
> > > > + * much better. The damage has been done.
> > > > + *
> > > > + * However, if we cannot reset an engine by itself, we cannot
> > > > + * cleanup a hanging persistent context without causing
> > > > + * colateral damage, and we should not pretend we can by
> > > > + * exposing the interface.
> > > > + */
> > > > + if (!intel_has_reset_engine(&i915->gt))
> > > > + return -ENODEV;
> > > > +
> > > > + pc->user_flags &= ~BIT(UCONTEXT_PERSISTENCE);
> > > > + }
> > > > +
> > > > + return 0;
> > > > +}
> > > > +
> > > > +static struct i915_gem_proto_context *
> > > > +proto_context_create(struct drm_i915_private *i915, unsigned int flags)
> > > > +{
> > > > + struct i915_gem_proto_context *pc, *err;
> > > > +
> > > > + pc = kzalloc(sizeof(*pc), GFP_KERNEL);
> > > > + if (!pc)
> > > > + return ERR_PTR(-ENOMEM);
> > > > +
> > > > + pc->user_flags = BIT(UCONTEXT_BANNABLE) |
> > > > + BIT(UCONTEXT_RECOVERABLE);
> > > > + proto_context_set_persistence(i915, pc, true);
> > >
> > > I think we're down to the bikeshed here that I think should be open-coded
> > > for clarity of what exactly is going on. But maybe better in a distinct
> > > patch.
> >
> > Nope. I just failed at rebasing. :-) I put it in the 2nd big
> > proto-ctx patch instead of this one. Should be fixed now.
>
> Ah found it, and looks correct.
> >
> > > Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
>
> So r-b should hold once you've moved the hunk.
v5 has been posted and has the hunk moved. I left your RB
--Jason
> -Daniel
>
> >
> > Thanks!
> >
> > > > + pc->sched.priority = I915_PRIORITY_NORMAL;
> > > > +
> > > > + if (flags & I915_CONTEXT_CREATE_FLAGS_SINGLE_TIMELINE) {
> > > > + if (!HAS_EXECLISTS(i915)) {
> > > > + err = ERR_PTR(-EINVAL);
> > > > + goto proto_close;
> > > > + }
> > > > + pc->single_timeline = true;
> > > > + }
> > > > +
> > > > + return pc;
> > > > +
> > > > +proto_close:
> > > > + proto_context_close(pc);
> > > > + return err;
> > > > +}
> > > > +
> > > > static struct i915_address_space *
> > > > context_get_vm_rcu(struct i915_gem_context *ctx)
> > > > {
> > > > @@ -660,7 +737,8 @@ static int __context_set_persistence(struct i915_gem_context *ctx, bool state)
> > > > }
> > > >
> > > > static struct i915_gem_context *
> > > > -__create_context(struct drm_i915_private *i915)
> > > > +__create_context(struct drm_i915_private *i915,
> > > > + const struct i915_gem_proto_context *pc)
> > > > {
> > > > struct i915_gem_context *ctx;
> > > > struct i915_gem_engines *e;
> > > > @@ -673,7 +751,7 @@ __create_context(struct drm_i915_private *i915)
> > > >
> > > > kref_init(&ctx->ref);
> > > > ctx->i915 = i915;
> > > > - ctx->sched.priority = I915_PRIORITY_NORMAL;
> > > > + ctx->sched = pc->sched;
> > > > mutex_init(&ctx->mutex);
> > > > INIT_LIST_HEAD(&ctx->link);
> > > >
> > > > @@ -696,9 +774,7 @@ __create_context(struct drm_i915_private *i915)
> > > > * is no remap info, it will be a NOP. */
> > > > ctx->remap_slice = ALL_L3_SLICES(i915);
> > > >
> > > > - i915_gem_context_set_bannable(ctx);
> > > > - i915_gem_context_set_recoverable(ctx);
> > > > - __context_set_persistence(ctx, true /* cgroup hook? */);
> > > > + ctx->user_flags = pc->user_flags;
> > > >
> > > > for (i = 0; i < ARRAY_SIZE(ctx->hang_timestamp); i++)
> > > > ctx->hang_timestamp[i] = jiffies - CONTEXT_FAST_HANG_JIFFIES;
> > > > @@ -786,20 +862,22 @@ static void __assign_ppgtt(struct i915_gem_context *ctx,
> > > > }
> > > >
> > > > static struct i915_gem_context *
> > > > -i915_gem_create_context(struct drm_i915_private *i915, unsigned int flags)
> > > > +i915_gem_create_context(struct drm_i915_private *i915,
> > > > + const struct i915_gem_proto_context *pc)
> > > > {
> > > > struct i915_gem_context *ctx;
> > > > int ret;
> > > >
> > > > - if (flags & I915_CONTEXT_CREATE_FLAGS_SINGLE_TIMELINE &&
> > > > - !HAS_EXECLISTS(i915))
> > > > - return ERR_PTR(-EINVAL);
> > > > -
> > > > - ctx = __create_context(i915);
> > > > + ctx = __create_context(i915, pc);
> > > > if (IS_ERR(ctx))
> > > > return ctx;
> > > >
> > > > - if (HAS_FULL_PPGTT(i915)) {
> > > > + if (pc->vm) {
> > > > + /* __assign_ppgtt() requires this mutex to be held */
> > > > + mutex_lock(&ctx->mutex);
> > > > + __assign_ppgtt(ctx, pc->vm);
> > > > + mutex_unlock(&ctx->mutex);
> > > > + } else if (HAS_FULL_PPGTT(i915)) {
> > > > struct i915_ppgtt *ppgtt;
> > > >
> > > > ppgtt = i915_ppgtt_create(&i915->gt);
> > > > @@ -810,14 +888,16 @@ i915_gem_create_context(struct drm_i915_private *i915, unsigned int flags)
> > > > return ERR_CAST(ppgtt);
> > > > }
> > > >
> > > > + /* __assign_ppgtt() requires this mutex to be held */
> > > > mutex_lock(&ctx->mutex);
> > > > __assign_ppgtt(ctx, &ppgtt->vm);
> > > > mutex_unlock(&ctx->mutex);
> > > >
> > > > + /* __assign_ppgtt() takes another reference for us */
> > > > i915_vm_put(&ppgtt->vm);
> > > > }
> > > >
> > > > - if (flags & I915_CONTEXT_CREATE_FLAGS_SINGLE_TIMELINE) {
> > > > + if (pc->single_timeline) {
> > > > ret = drm_syncobj_create(&ctx->syncobj,
> > > > DRM_SYNCOBJ_CREATE_SIGNALED,
> > > > NULL);
> > > > @@ -883,6 +963,7 @@ int i915_gem_context_open(struct drm_i915_private *i915,
> > > > struct drm_file *file)
> > > > {
> > > > struct drm_i915_file_private *file_priv = file->driver_priv;
> > > > + struct i915_gem_proto_context *pc;
> > > > struct i915_gem_context *ctx;
> > > > int err;
> > > > u32 id;
> > > > @@ -892,7 +973,14 @@ int i915_gem_context_open(struct drm_i915_private *i915,
> > > > /* 0 reserved for invalid/unassigned ppgtt */
> > > > xa_init_flags(&file_priv->vm_xa, XA_FLAGS_ALLOC1);
> > > >
> > > > - ctx = i915_gem_create_context(i915, 0);
> > > > + pc = proto_context_create(i915, 0);
> > > > + if (IS_ERR(pc)) {
> > > > + err = PTR_ERR(pc);
> > > > + goto err;
> > > > + }
> > > > +
> > > > + ctx = i915_gem_create_context(i915, pc);
> > > > + proto_context_close(pc);
> > > > if (IS_ERR(ctx)) {
> > > > err = PTR_ERR(ctx);
> > > > goto err;
> > > > @@ -1884,6 +1972,7 @@ int i915_gem_context_create_ioctl(struct drm_device *dev, void *data,
> > > > {
> > > > struct drm_i915_private *i915 = to_i915(dev);
> > > > struct drm_i915_gem_context_create_ext *args = data;
> > > > + struct i915_gem_proto_context *pc;
> > > > struct create_ext ext_data;
> > > > int ret;
> > > > u32 id;
> > > > @@ -1906,7 +1995,12 @@ int i915_gem_context_create_ioctl(struct drm_device *dev, void *data,
> > > > return -EIO;
> > > > }
> > > >
> > > > - ext_data.ctx = i915_gem_create_context(i915, args->flags);
> > > > + pc = proto_context_create(i915, args->flags);
> > > > + if (IS_ERR(pc))
> > > > + return PTR_ERR(pc);
> > > > +
> > > > + ext_data.ctx = i915_gem_create_context(i915, pc);
> > > > + proto_context_close(pc);
> > > > if (IS_ERR(ext_data.ctx))
> > > > return PTR_ERR(ext_data.ctx);
> > > >
> > > > diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context_types.h b/drivers/gpu/drm/i915/gem/i915_gem_context_types.h
> > > > index 5f0673a2129f9..e0bdf3e298a6a 100644
> > > > --- a/drivers/gpu/drm/i915/gem/i915_gem_context_types.h
> > > > +++ b/drivers/gpu/drm/i915/gem/i915_gem_context_types.h
> > > > @@ -66,6 +66,28 @@ struct i915_gem_engines_iter {
> > > > const struct i915_gem_engines *engines;
> > > > };
> > > >
> > > > +/**
> > > > + * struct i915_gem_proto_context - prototype context
> > > > + *
> > > > + * The struct i915_gem_proto_context represents the creation parameters for
> > > > + * a struct i915_gem_context. This is used to gather parameters provided
> > > > + * either through creation flags or via SET_CONTEXT_PARAM so that, when we
> > > > + * create the final i915_gem_context, those parameters can be immutable.
> > > > + */
> > > > +struct i915_gem_proto_context {
> > > > + /** @vm: See &i915_gem_context.vm */
> > > > + struct i915_address_space *vm;
> > > > +
> > > > + /** @user_flags: See &i915_gem_context.user_flags */
> > > > + unsigned long user_flags;
> > > > +
> > > > + /** @sched: See &i915_gem_context.sched */
> > > > + struct i915_sched_attr sched;
> > > > +
> > > > + /** @single_timeline: See See &i915_gem_context.syncobj */
> > > > + bool single_timeline;
> > > > +};
> > > > +
> > > > /**
> > > > * struct i915_gem_context - client state
> > > > *
> > > > diff --git a/drivers/gpu/drm/i915/gem/selftests/mock_context.c b/drivers/gpu/drm/i915/gem/selftests/mock_context.c
> > > > index 51b5a3421b400..e0f512ef7f3c6 100644
> > > > --- a/drivers/gpu/drm/i915/gem/selftests/mock_context.c
> > > > +++ b/drivers/gpu/drm/i915/gem/selftests/mock_context.c
> > > > @@ -80,11 +80,17 @@ void mock_init_contexts(struct drm_i915_private *i915)
> > > > struct i915_gem_context *
> > > > live_context(struct drm_i915_private *i915, struct file *file)
> > > > {
> > > > + struct i915_gem_proto_context *pc;
> > > > struct i915_gem_context *ctx;
> > > > int err;
> > > > u32 id;
> > > >
> > > > - ctx = i915_gem_create_context(i915, 0);
> > > > + pc = proto_context_create(i915, 0);
> > > > + if (IS_ERR(pc))
> > > > + return ERR_CAST(pc);
> > > > +
> > > > + ctx = i915_gem_create_context(i915, pc);
> > > > + proto_context_close(pc);
> > > > if (IS_ERR(ctx))
> > > > return ctx;
> > > >
> > > > @@ -142,8 +148,14 @@ struct i915_gem_context *
> > > > kernel_context(struct drm_i915_private *i915)
> > > > {
> > > > struct i915_gem_context *ctx;
> > > > + struct i915_gem_proto_context *pc;
> > > > +
> > > > + pc = proto_context_create(i915, 0);
> > > > + if (IS_ERR(pc))
> > > > + return ERR_CAST(pc);
> > > >
> > > > - ctx = i915_gem_create_context(i915, 0);
> > > > + ctx = i915_gem_create_context(i915, pc);
> > > > + proto_context_close(pc);
> > > > if (IS_ERR(ctx))
> > > > return ctx;
> > > >
> > > > --
> > > > 2.31.1
> > > >
> > > > _______________________________________________
> > > > Intel-gfx mailing list
> > > > Intel-gfx@lists.freedesktop.org
> > > > https://lists.freedesktop.org/mailman/listinfo/intel-gfx
> > >
> > > --
> > > Daniel Vetter
> > > Software Engineer, Intel Corporation
> > > http://blog.ffwll.ch
>
> --
> Daniel Vetter
> Software Engineer, Intel Corporation
> http://blog.ffwll.ch
next prev parent reply other threads:[~2021-06-10 16:54 UTC|newest]
Thread overview: 47+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-06-09 4:35 [PATCH 00/31] drm/i915/gem: ioctl clean-ups (v6) Jason Ekstrand
2021-06-09 4:35 ` [PATCH 01/31] drm/i915: Drop I915_CONTEXT_PARAM_RINGSIZE Jason Ekstrand
2021-06-09 4:35 ` [PATCH 02/31] drm/i915: Stop storing the ring size in the ring pointer (v2) Jason Ekstrand
2021-06-09 4:35 ` [PATCH 03/31] drm/i915: Drop I915_CONTEXT_PARAM_NO_ZEROMAP Jason Ekstrand
2021-06-09 4:35 ` [PATCH 04/31] drm/i915/gem: Set the watchdog timeout directly in intel_context_set_gem (v2) Jason Ekstrand
2021-06-09 4:35 ` [PATCH 05/31] drm/i915/gem: Return void from context_apply_all Jason Ekstrand
2021-06-09 4:35 ` [PATCH 06/31] drm/i915: Drop the CONTEXT_CLONE API (v2) Jason Ekstrand
2021-06-09 4:35 ` [PATCH 07/31] drm/i915: Implement SINGLE_TIMELINE with a syncobj (v4) Jason Ekstrand
2021-06-09 4:35 ` [PATCH 08/31] drm/i915: Drop getparam support for I915_CONTEXT_PARAM_ENGINES Jason Ekstrand
2021-06-09 4:35 ` [PATCH 09/31] drm/i915/gem: Disallow bonding of virtual engines (v3) Jason Ekstrand
2021-06-09 4:35 ` [PATCH 10/31] drm/i915/gem: Remove engine auto-magic with FENCE_SUBMIT (v2) Jason Ekstrand
2021-06-09 4:35 ` [PATCH 11/31] drm/i915/request: Remove the hook from await_execution Jason Ekstrand
2021-06-09 4:35 ` [PATCH 12/31] drm/i915/gem: Disallow creating contexts with too many engines Jason Ekstrand
2021-06-09 4:35 ` [PATCH 13/31] drm/i915: Stop manually RCU banging in reset_stats_ioctl (v2) Jason Ekstrand
2021-06-09 4:35 ` [PATCH 14/31] drm/i915/gem: Add a separate validate_priority helper Jason Ekstrand
2021-06-09 4:35 ` [PATCH 15/31] drm/i915: Add gem/i915_gem_context.h to the docs Jason Ekstrand
2021-06-09 4:35 ` [PATCH 16/31] drm/i915/gem: Add an intermediate proto_context struct (v4) Jason Ekstrand
2021-06-09 11:28 ` [Intel-gfx] " Daniel Vetter
2021-06-09 16:00 ` Jason Ekstrand
2021-06-10 16:44 ` Daniel Vetter
2021-06-10 16:53 ` Jason Ekstrand [this message]
2021-06-09 4:35 ` [PATCH 17/31] drm/i915/gem: Rework error handling in default_engines Jason Ekstrand
2021-06-09 4:36 ` [PATCH 18/31] drm/i915/gem: Optionally set SSEU in intel_context_set_gem Jason Ekstrand
2021-06-09 4:36 ` [PATCH 19/31] drm/i915: Add an i915_gem_vm_lookup helper Jason Ekstrand
2021-06-09 4:36 ` [PATCH 20/31] drm/i915/gem: Make an alignment check more sensible Jason Ekstrand
2021-06-09 4:36 ` [PATCH 21/31] drm/i915/gem: Use the proto-context to handle create parameters (v3) Jason Ekstrand
2021-06-09 4:36 ` [PATCH 22/31] drm/i915/gem: Return an error ptr from context_lookup Jason Ekstrand
2021-06-09 4:36 ` [PATCH 23/31] drm/i915/gt: Drop i915_address_space::file (v2) Jason Ekstrand
2021-06-09 4:36 ` [PATCH 24/31] drm/i915/gem: Delay context creation (v2) Jason Ekstrand
2021-06-09 8:25 ` [Intel-gfx] " kernel test robot
2021-06-09 11:38 ` kernel test robot
2021-06-09 11:44 ` kernel test robot
2021-06-09 4:36 ` [PATCH 25/31] drm/i915/gem: Don't allow changing the VM on running contexts (v2) Jason Ekstrand
2021-06-09 7:39 ` [Intel-gfx] " kernel test robot
2021-06-09 11:34 ` Daniel Vetter
2021-06-09 11:34 ` Daniel Vetter
2021-06-09 16:06 ` Jason Ekstrand
2021-06-09 4:36 ` [PATCH 26/31] drm/i915/gem: Don't allow changing the engine set " Jason Ekstrand
2021-06-09 4:36 ` [PATCH 27/31] drm/i915/selftests: Take a VM in kernel_context() Jason Ekstrand
2021-06-09 4:36 ` [PATCH 28/31] i915/gem/selftests: Assign the VM at context creation in igt_shared_ctx_exec Jason Ekstrand
2021-06-09 4:36 ` [PATCH 29/31] drm/i915/gem: Roll all of context creation together Jason Ekstrand
2021-06-09 4:36 ` [PATCH 30/31] drm/i915: Finalize contexts in GEM_CONTEXT_CREATE on version 13+ Jason Ekstrand
2021-06-09 10:13 ` [Intel-gfx] " kernel test robot
2021-06-09 11:38 ` Daniel Vetter
2021-06-09 4:36 ` [PATCH 31/31] drm/i915: Drop some RCU usage around context VMs Jason Ekstrand
2021-06-09 11:41 ` [Intel-gfx] " Daniel Vetter
2021-06-09 16:07 ` Jason Ekstrand
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAOFGe95osa5vC3UQU6Rhchg7korhfXv0jYsTL0x5gD_-OWeEBA@mail.gmail.com \
--to=jason@jlekstrand.net \
--cc=daniel@ffwll.ch \
--cc=dri-devel@lists.freedesktop.org \
--cc=intel-gfx@lists.freedesktop.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).