From: Friedrich Vock <friedrich.vock@gmx.de>
To: Tvrtko Ursulin <tursulin@igalia.com>,
amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org
Cc: kernel-dev@igalia.com,
"Tvrtko Ursulin" <tvrtko.ursulin@igalia.com>,
"Christian König" <christian.koenig@amd.com>
Subject: Re: [RFC 1/5] drm/amdgpu: Fix migration rate limiting accounting
Date: Wed, 8 May 2024 21:08:57 +0200 [thread overview]
Message-ID: <146d615c-6eb1-491a-9494-cacb9337f13e@gmx.de> (raw)
In-Reply-To: <20240508180946.96863-2-tursulin@igalia.com>
On 08.05.24 20:09, Tvrtko Ursulin wrote:
> From: Tvrtko Ursulin <tvrtko.ursulin@igalia.com>
>
> The logic assumed any migration attempt worked and therefore would over-
> account the amount of data migrated during buffer re-validation. As a
> consequence client can be unfairly penalised by incorrectly considering
> its migration budget spent.
If the migration failed but data was still moved (which I think could be
the case when we try evicting everything but it still doesn't work?),
shouldn't the eviction movements count towards the ratelimit too?
>
> Fix it by looking at the before and after buffer object backing store and
> only account if there was a change.
>
> FIXME:
> I think this needs a better solution to account for migrations between
> VRAM visible and non-visible portions.
FWIW, I have some WIP patches (not posted on any MLs yet though) that
attempt to solve this issue (+actually enforcing ratelimits) by moving
the ratelimit accounting/enforcement to TTM entirely.
By moving the accounting to TTM we can count moved bytes when we move
them, and don't have to rely on comparing resources to determine whether
moving actually happened. This should address your FIXME as well.
Regards,
Friedrich
> Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@igalia.com>
> Cc: Christian König <christian.koenig@amd.com>
> Cc: Friedrich Vock <friedrich.vock@gmx.de>
> ---
> drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 26 +++++++++++++++++++++-----
> 1 file changed, 21 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
> index ec888fc6ead8..22708954ae68 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
> @@ -784,12 +784,15 @@ static int amdgpu_cs_bo_validate(void *param, struct amdgpu_bo *bo)
> .no_wait_gpu = false,
> .resv = bo->tbo.base.resv
> };
> + struct ttm_resource *old_res;
> uint32_t domain;
> int r;
>
> if (bo->tbo.pin_count)
> return 0;
>
> + old_res = bo->tbo.resource;
> +
> /* Don't move this buffer if we have depleted our allowance
> * to move it. Don't move anything if the threshold is zero.
> */
> @@ -817,16 +820,29 @@ static int amdgpu_cs_bo_validate(void *param, struct amdgpu_bo *bo)
> amdgpu_bo_placement_from_domain(bo, domain);
> r = ttm_bo_validate(&bo->tbo, &bo->placement, &ctx);
>
> - p->bytes_moved += ctx.bytes_moved;
> - if (!amdgpu_gmc_vram_full_visible(&adev->gmc) &&
> - amdgpu_res_cpu_visible(adev, bo->tbo.resource))
> - p->bytes_moved_vis += ctx.bytes_moved;
> -
> if (unlikely(r == -ENOMEM) && domain != bo->allowed_domains) {
> domain = bo->allowed_domains;
> goto retry;
> }
>
> + if (!r) {
> + struct ttm_resource *new_res = bo->tbo.resource;
> + bool moved = true;
> +
> + if (old_res == new_res)
> + moved = false;
> + else if (old_res && new_res &&
> + old_res->mem_type == new_res->mem_type)
> + moved = false;
> +
> + if (moved) {
> + p->bytes_moved += ctx.bytes_moved;
> + if (!amdgpu_gmc_vram_full_visible(&adev->gmc) &&
> + amdgpu_res_cpu_visible(adev, bo->tbo.resource))
> + p->bytes_moved_vis += ctx.bytes_moved;
> + }
> + }
> +
> return r;
> }
>
next prev parent reply other threads:[~2024-05-08 19:09 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-05-08 18:09 [RFC 0/5] Discussion around eviction improvements Tvrtko Ursulin
2024-05-08 18:09 ` [RFC 1/5] drm/amdgpu: Fix migration rate limiting accounting Tvrtko Ursulin
2024-05-08 19:08 ` Friedrich Vock [this message]
2024-05-09 9:19 ` Tvrtko Ursulin
2024-05-13 14:36 ` Friedrich Vock
2024-05-15 7:14 ` Christian König
2024-05-15 10:51 ` Tvrtko Ursulin
2024-05-08 18:09 ` [RFC 2/5] drm/amdgpu: Actually respect buffer migration budget Tvrtko Ursulin
2024-05-15 7:20 ` Christian König
2024-05-15 10:59 ` Tvrtko Ursulin
2024-05-15 14:31 ` Christian König
2024-05-15 15:13 ` Tvrtko Ursulin
2024-05-08 18:09 ` [RFC 3/5] drm/ttm: Add preferred placement flag Tvrtko Ursulin
2024-05-08 18:09 ` [RFC 4/5] drm/amdgpu: Use preferred placement for VRAM+GTT Tvrtko Ursulin
2024-05-08 18:09 ` [RFC 5/5] drm/amdgpu: Re-validate evicted buffers Tvrtko Ursulin
2024-05-09 12:40 ` [RFC 0/5] Discussion around eviction improvements Tvrtko Ursulin
2024-05-13 13:49 ` Tvrtko Ursulin
2024-05-14 15:14 ` Tvrtko Ursulin
2024-05-14 15:47 ` Christian König
2024-05-13 6:50 ` Christian König
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=146d615c-6eb1-491a-9494-cacb9337f13e@gmx.de \
--to=friedrich.vock@gmx.de \
--cc=amd-gfx@lists.freedesktop.org \
--cc=christian.koenig@amd.com \
--cc=dri-devel@lists.freedesktop.org \
--cc=kernel-dev@igalia.com \
--cc=tursulin@igalia.com \
--cc=tvrtko.ursulin@igalia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).