From: Johannes Thumshirn <Johannes.Thumshirn@wdc.com>
To: Boris Burkov <boris@bur.io>,
"linux-btrfs@vger.kernel.org" <linux-btrfs@vger.kernel.org>,
"kernel-team@fb.com" <kernel-team@fb.com>
Subject: Re: [PATCH 3/6] btrfs: dynamic block_group reclaim threshold
Date: Tue, 9 Apr 2024 11:52:30 +0000 [thread overview]
Message-ID: <e0daf386-d8bb-4bc6-b0e2-e8516788b592@wdc.com> (raw)
In-Reply-To: <198348dc7188360fe33083ce06421b1e7f7874d9.1712168477.git.boris@bur.io>
On 03.04.24 21:37, Boris Burkov wrote:
> +/*
> + * The fundamental goal of automatic reclaim is to protect the filesystem's
> + * unallocated space and thus minimize the probability of the filesystem going
> + * read only when a metadata allocation failure causes a transaction abort.
> + *
> + * However, relocations happen into the space_info's unused space, therefore
> + * automatic reclaim must also back off as that space runs low. There is no
> + * value in doing trivial "relocations" of re-writing the same block group
> + * into a fresh one.
> + *
> + * Furthermore, we want to avoid doing too much reclaim even if there are good
> + * candidates. This is because the allocator is pretty good at filling up the
> + * holes with writes. So we want to do just enough reclaim to try and stay
> + * safe from running out of unallocated space but not be wasteful about it.
> + *
> + * Therefore, the dynamic reclaim threshold is calculated as follows:
> + * - calculate a target unallocated amount of 5 block group sized chunks
> + * - ratchet up the intensity of reclaim depending on how far we are from
> + * that target by using a formula of unalloc / target to set the threshold.
> + *
> + * Typically with 10 block groups as the target, the discrete values this comes
> + * out to are 0, 10, 20, ... , 80, 90, and 99.
> + */
> +static int calc_dynamic_reclaim_threshold(struct btrfs_space_info *space_info)
> +{
> + struct btrfs_fs_info *fs_info = space_info->fs_info;
> + u64 unalloc = atomic64_read(&fs_info->free_chunk_space);
> + u64 target = calc_unalloc_target(fs_info);
> + u64 alloc = space_info->total_bytes;
> + u64 used = btrfs_space_info_used(space_info, false);
As btrfs_space_info_used() includes bytes_zone_unusable, I /think/ we
need to subtract it here again, or add it to unused.
> + u64 unused = alloc - used;
> + u64 want = target > unalloc ? target - unalloc : 0;
> + u64 data_chunk_size = calc_effective_data_chunk_size(fs_info);
> + /* Cast to int is OK because want <= target */
> + int ratio = calc_pct_ratio(want, target);
> +
> + /* If we have no unused space, don't bother, it won't work anyway */
> + if (unused < data_chunk_size)
> + return 0;
> +
> + return ratio;
> +}
next prev parent reply other threads:[~2024-04-09 11:52 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-04-03 19:38 [PATCH 0/6] btrfs: dynamic and periodic block_group reclaim Boris Burkov
2024-04-03 19:38 ` [PATCH 1/6] btrfs: report reclaim stats in sysfs Boris Burkov
2024-04-09 11:24 ` Johannes Thumshirn
2024-04-03 19:38 ` [PATCH 2/6] btrfs: store fs_info on space_info Boris Burkov
2024-04-09 11:25 ` Johannes Thumshirn
2024-04-03 19:38 ` [PATCH 3/6] btrfs: dynamic block_group reclaim threshold Boris Burkov
2024-04-09 11:52 ` Johannes Thumshirn [this message]
2024-04-03 19:38 ` [PATCH 4/6] btrfs: periodic block_group reclaim Boris Burkov
2024-04-03 19:38 ` [PATCH 5/6] btrfs: prevent pathological periodic reclaim loops Boris Burkov
2024-04-03 19:38 ` [PATCH 6/6] btrfs: urgent periodic reclaim pass Boris Burkov
-- strict thread matches above, loose matches on Subject: below --
2024-02-02 23:12 [RFC PATCH 0/6] btrfs: dynamic and periodic block_group reclaim Boris Burkov
2024-02-02 23:12 ` [PATCH 3/6] btrfs: dynamic block_group reclaim threshold Boris Burkov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=e0daf386-d8bb-4bc6-b0e2-e8516788b592@wdc.com \
--to=johannes.thumshirn@wdc.com \
--cc=boris@bur.io \
--cc=kernel-team@fb.com \
--cc=linux-btrfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).