From: Dan Schatzberg <schatzberg.dan@gmail.com>
To: Waiman Long <longman@redhat.com>
Cc: Jens Axboe <axboe@kernel.dk>, Tejun Heo <tj@kernel.org>,
Josef Bacik <josef@toxicpanda.com>,
cgroups@vger.kernel.org, linux-block@vger.kernel.org,
linux-kernel@vger.kernel.org, Ming Lei <ming.lei@redhat.com>
Subject: Re: [PATCH] blk-cgroup: Properly propagate the iostat update up the hierarchy
Date: Wed, 15 May 2024 12:58:00 -0400 [thread overview]
Message-ID: <ZkTpmPoMb4PyPwCl@dschatzberg-fedora-PF3DHTBV> (raw)
In-Reply-To: <20240515143059.276677-1-longman@redhat.com>
On Wed, May 15, 2024 at 10:30:59AM -0400, Waiman Long wrote:
> During a cgroup_rstat_flush() call, the lowest level of nodes are flushed
> first before their parents. Since commit 3b8cc6298724 ("blk-cgroup:
> Optimize blkcg_rstat_flush()"), iostat propagation was still done to
> the parent. Grandparent, however, may not get the iostat update if the
> parent has no blkg_iostat_set queued in its lhead lockless list.
>
> Fix this iostat propagation problem by queuing the parent's global
> blkg->iostat into one of its percpu lockless lists to make sure that
> the delta will always be propagated up to the grandparent and so on
> toward the root blkcg.
>
> Note that successive calls to __blkcg_rstat_flush() are serialized by
> the cgroup_rstat_lock. So no special barrier is used in the reading
> and writing of blkg->iostat.lqueued.
>
> Fixes: 3b8cc6298724 ("blk-cgroup: Optimize blkcg_rstat_flush()")
> Reported-by: Dan Schatzberg <schatzberg.dan@gmail.com>
> Closes: https://lore.kernel.org/lkml/ZkO6l%2FODzadSgdhC@dschatzberg-fedora-PF3DHTBV/
> Signed-off-by: Waiman Long <longman@redhat.com>
> ---
> block/blk-cgroup.c | 19 ++++++++++++++++++-
> 1 file changed, 18 insertions(+), 1 deletion(-)
>
> diff --git a/block/blk-cgroup.c b/block/blk-cgroup.c
> index 059467086b13..2a7624c32a1a 100644
> --- a/block/blk-cgroup.c
> +++ b/block/blk-cgroup.c
> @@ -323,6 +323,7 @@ static struct blkcg_gq *blkg_alloc(struct blkcg *blkcg, struct gendisk *disk,
> blkg->q = disk->queue;
> INIT_LIST_HEAD(&blkg->q_node);
> blkg->blkcg = blkcg;
> + blkg->iostat.blkg = blkg;
> #ifdef CONFIG_BLK_CGROUP_PUNT_BIO
> spin_lock_init(&blkg->async_bio_lock);
> bio_list_init(&blkg->async_bios);
> @@ -1025,6 +1026,8 @@ static void __blkcg_rstat_flush(struct blkcg *blkcg, int cpu)
> unsigned int seq;
>
> WRITE_ONCE(bisc->lqueued, false);
> + if (bisc == &blkg->iostat)
> + goto propagate_up; /* propagate up to parent only */
>
> /* fetch the current per-cpu values */
> do {
> @@ -1034,10 +1037,24 @@ static void __blkcg_rstat_flush(struct blkcg *blkcg, int cpu)
>
> blkcg_iostat_update(blkg, &cur, &bisc->last);
>
> +propagate_up:
> /* propagate global delta to parent (unless that's root) */
> - if (parent && parent->parent)
> + if (parent && parent->parent) {
> blkcg_iostat_update(parent, &blkg->iostat.cur,
> &blkg->iostat.last);
> + /*
> + * Queue parent->iostat to its blkcg's lockless
> + * list to propagate up to the grandparent if the
> + * iostat hasn't been queued yet.
> + */
> + if (!parent->iostat.lqueued) {
> + struct llist_head *plhead;
> +
> + plhead = per_cpu_ptr(parent->blkcg->lhead, cpu);
> + llist_add(&parent->iostat.lnode, plhead);
> + parent->iostat.lqueued = true;
> + }
> + }
> }
> raw_spin_unlock_irqrestore(&blkg_stat_lock, flags);
> out:
> --
> 2.39.3
>
I've tested and confirmed this patch fixes the original issue. Thanks!
next prev parent reply other threads:[~2024-05-15 16:58 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-05-15 14:30 [PATCH] blk-cgroup: Properly propagate the iostat update up the hierarchy Waiman Long
2024-05-15 14:35 ` Waiman Long
2024-05-15 16:58 ` Dan Schatzberg [this message]
2024-05-15 17:00 ` Tejun Heo
2024-05-16 1:13 ` Ming Lei
2024-05-16 2:21 ` Jens Axboe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ZkTpmPoMb4PyPwCl@dschatzberg-fedora-PF3DHTBV \
--to=schatzberg.dan@gmail.com \
--cc=axboe@kernel.dk \
--cc=cgroups@vger.kernel.org \
--cc=josef@toxicpanda.com \
--cc=linux-block@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=longman@redhat.com \
--cc=ming.lei@redhat.com \
--cc=tj@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).