From: Waiman Long <longman@redhat.com>
To: Dan Schatzberg <schatzberg.dan@gmail.com>, Tejun Heo <tj@kernel.org>
Cc: linux-block@vger.kernel.org, cgroups@vger.kernel.org,
linux-kernel@vger.kernel.org
Subject: Re: cgroup io.stat propagation regression
Date: Wed, 15 May 2024 10:26:31 -0400 [thread overview]
Message-ID: <11b8c1e4-45a7-4895-a1f3-6626744cee1e@redhat.com> (raw)
In-Reply-To: <3ed32279-904a-411d-91a4-a62f4ca2dde2@redhat.com>
On 5/14/24 23:59, Waiman Long wrote:
> On 5/14/24 15:25, Dan Schatzberg wrote:
>> Hi Waiman,
>>
>> I've noticed that on recent kernels io.stat metrics don't propagate
>> all the way up the hierarchy. Specifically, io.stat metrics of some
>> leaf cgroup will be propagated to the parent, but not its grandparent.
>>
>> For a simple repro, run the following:
>>
>> systemd-run --slice test-test dd if=/dev/urandom of=/tmp/test bs=4096
>> count=1
>>
>> Then:
>>
>> cat /sys/fs/cgroup/test.slice/test-test.slice/io.stat
>>
>> Shows the parent cgroup stats and I see wbytes=4096 but the
>> grandparent cgroup:
>>
>> cat /sys/fs/cgroup/test.slice/io.stat
>>
>> shows no writes.
>>
>> I believe this was caused by the change in "blk-cgroup: Optimize
>> blkcg_rstat_flush()". When blkcg_rstat_flush is called on the parent
>> cgroup, it exits early because the lockless list is empty since the
>> parent cgroup never issued writes itself (e.g. in
>> blk_cgroup_bio_start). However, in doing so it never propagated stats
>> to its parent.
>>
>> Can you confirm if my understanding of the logic here is correct and
>> advise on a fix?
>
> Yes, I believe your analysis is correct. Thanks for spotting this
> iostat propagation problem.
>
> I am working on a fix to address this problem and will post a patch
> once I have finished my testing.
Actually, I can only reproduce the issue with a 3-level
(child-parent-grandparent) cgroup hierarchy below the root cgroup. The
dd command is run test.slice/test-test.slice. So both test.slice/io.stat
and test.slice/test-test.slice/io.stat are properly updated.
Cheers,
Longman
next prev parent reply other threads:[~2024-05-15 14:26 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-05-14 19:25 cgroup io.stat propagation regression Dan Schatzberg
2024-05-15 3:59 ` Waiman Long
2024-05-15 14:26 ` Waiman Long [this message]
2024-05-15 16:54 ` Dan Schatzberg
2024-05-15 17:51 ` Waiman Long
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=11b8c1e4-45a7-4895-a1f3-6626744cee1e@redhat.com \
--to=longman@redhat.com \
--cc=cgroups@vger.kernel.org \
--cc=linux-block@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=schatzberg.dan@gmail.com \
--cc=tj@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).