From: Qu Wenruo <quwenruo.btrfs@gmx.com>
To: Johannes Thumshirn <Johannes.Thumshirn@wdc.com>,
Hans Holmberg <Hans.Holmberg@wdc.com>,
Zorro Lang <zlang@redhat.com>
Cc: "Zorro Lang" <zlang@kernel.org>,
"Darrick J. Wong" <djwong@kernel.org>,
"linux-btrfs@vger.kernel.org" <linux-btrfs@vger.kernel.org>,
"Damien Le Moal" <Damien.LeMoal@wdc.com>,
"Matias Bjørling" <Matias.Bjorling@wdc.com>,
"Naohiro Aota" <Naohiro.Aota@wdc.com>, "hch@lst.de" <hch@lst.de>,
"fstests@vger.kernel.org" <fstests@vger.kernel.org>,
"Jaegeuk Kim" <jaegeuk@kernel.org>,
"bvanassche@acm.org" <bvanassche@acm.org>,
"daeho43@gmail.com" <daeho43@gmail.com>,
"Boris Burkov" <boris@bur.io>
Subject: Re: [PATCH] generic: add gc stress test
Date: Mon, 13 May 2024 17:03:40 +0930 [thread overview]
Message-ID: <b8723562-d154-4171-836c-6194cfd708a5@gmx.com> (raw)
In-Reply-To: <299bf8a4-b71e-4a05-8210-d52ea45d5329@wdc.com>
在 2024/5/13 02:26, Johannes Thumshirn 写道:
> [ +CC Boris ]
[...]
>> I was surprised to see the failure for brtrfs on a conventional block
>> device, but have not dug into it. I suspect/assume it's the same root
>> cause as the issue Johannes is looking into when using a zoned block
>> device as backing storage.
>>
>> I debugged that a bit with Johannes, and noticed that if I manually
>> kick btrfs rebalancing after each write via sysfs, the test progresses
>> further (but super slow).
>>
>> So *I think* that btrfs needs to:
>>
>> * tune the triggering of gc to kick in way before available free space
>> runs out
>> * start slowing down / blocking writes when reclaim pressure is high to
>> avoid premature -ENOSPC:es.
>
> Yes both Boris and I are working on different solutions to the GC
> problem. But apart from that, I have the feeling that using stat to
> check on the available space is not the best idea.
Although my previous workaround (fill to 100% then deleting 5%) is not
going to be feasible for zoned devices, what about two-run solution below?
- The first run to fill the whole fs until ENOSPC
Then calculate how many bytes we have really written. (du?)
- Recreate the fs and fill to 95% of above number and start the test
But with this workaround, I'm not 100% if this is a good idea for all
filesystems.
AFAIK ext4/xfs sometimes can under-report the available space (aka,
reporting no available bytes, but can still write new data).
If we always go ENOSPC to calculate the real available space, it may
cause too much pressure.
And it may be a good idea for us btrfs guys to implement a similar
under-reporting available space behavior?
Thanks,
Qu
>
>> It's a pretty nasty problem, as potentially any write could -ENOSPC
>> long before the reported available space runs out when a workload
>> ends up fragmenting the disk and write pressure is high..
>
>
next prev parent reply other threads:[~2024-05-13 7:34 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-04-15 11:23 [PATCH] generic: add gc stress test Hans Holmberg
2024-04-16 9:07 ` Hans Holmberg
2024-04-16 18:54 ` Darrick J. Wong
2024-04-17 12:43 ` Zorro Lang
2024-04-17 13:21 ` Hans Holmberg
2024-04-17 14:06 ` Zorro Lang
2024-04-17 14:45 ` Hans Holmberg
2024-05-08 7:08 ` Hans Holmberg
2024-05-08 8:51 ` Zorro Lang
2024-05-08 9:28 ` Qu Wenruo
2024-05-08 11:02 ` Johannes Thumshirn
2024-05-09 5:43 ` hch
2024-05-09 9:42 ` Zorro Lang
2024-05-09 12:54 ` hch
2024-05-10 3:21 ` Zorro Lang
2024-05-11 13:08 ` Hans Holmberg
2024-05-12 16:54 ` Johannes Thumshirn
2024-05-12 16:56 ` Johannes Thumshirn
2024-05-13 7:33 ` Qu Wenruo [this message]
2024-05-14 8:02 ` Hans Holmberg
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=b8723562-d154-4171-836c-6194cfd708a5@gmx.com \
--to=quwenruo.btrfs@gmx.com \
--cc=Damien.LeMoal@wdc.com \
--cc=Hans.Holmberg@wdc.com \
--cc=Johannes.Thumshirn@wdc.com \
--cc=Matias.Bjorling@wdc.com \
--cc=Naohiro.Aota@wdc.com \
--cc=boris@bur.io \
--cc=bvanassche@acm.org \
--cc=daeho43@gmail.com \
--cc=djwong@kernel.org \
--cc=fstests@vger.kernel.org \
--cc=hch@lst.de \
--cc=jaegeuk@kernel.org \
--cc=linux-btrfs@vger.kernel.org \
--cc=zlang@kernel.org \
--cc=zlang@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).