From: Sandeep Dhavale via Linux-erofs <linux-erofs@lists.ozlabs.org>
To: Gao Xiang <hsiangkao@linux.alibaba.com>
Cc: kernel-team@android.com, linux-erofs@lists.ozlabs.org
Subject: Re: [PATCH 1/1] erofs-utils: lib: treat data blocks filled with 0s as a hole
Date: Thu, 4 Apr 2024 09:52:41 -0700 [thread overview]
Message-ID: <CAB=BE-RJq=Zt7X7BHfMuWU_w-ssRBWWdrt4+zWQNN-JZTdv7hA@mail.gmail.com> (raw)
In-Reply-To: <7681743f-ea33-49d4-9769-53895e5355dc@linux.alibaba.com>
On Thu, Apr 4, 2024 at 7:00 AM Gao Xiang <hsiangkao@linux.alibaba.com> wrote:
>
> Hi Sandeep,
>
> On 2024/4/4 07:57, Sandeep Dhavale wrote:
> > Add optimization to treat data blocks filled with 0s as a hole.
> > Even though diskspace savings are comparable to chunk based or dedupe,
> > having no block assigned saves us redundant disk IOs during read.
> >
> > This patch detects if the block is filled with zeros and marks
> > chunk as erofs_holechunk so there is no physical block assigned.
> >
> > Signed-off-by: Sandeep Dhavale <dhavale@google.com>
> > ---
> > lib/blobchunk.c | 25 ++++++++++++++++++++++++-
> > 1 file changed, 24 insertions(+), 1 deletion(-)
> >
> > diff --git a/lib/blobchunk.c b/lib/blobchunk.c
> > index 641e3d4..8535058 100644
> > --- a/lib/blobchunk.c
> > +++ b/lib/blobchunk.c
> > @@ -232,6 +232,21 @@ static void erofs_update_minextblks(struct erofs_sb_info *sbi,
> > *minextblks = lb;
> > }
> >
> > +static bool erofs_is_buf_zeros(void *buf, unsigned long len)
> > +{
> > + int i, words;
> > + const unsigned long *words_buf = buf;
> > + words = len / sizeof(unsigned long);
> > +
> > + DBG_BUGON(len % sizeof(unsigned long));
> > +
> > + for (i = 0; i < words; i++) {
> > + if (words_buf[i])
> > + return false;
> > + }
> > + return true;
> > +}
> > +
> > int erofs_blob_write_chunked_file(struct erofs_inode *inode, int fd,
> > erofs_off_t startoff)
> > {
> > @@ -323,7 +338,15 @@ int erofs_blob_write_chunked_file(struct erofs_inode *inode, int fd,
> > ret = -EIO;
> > goto err;
> > }
> > -
> > + if (len == chunksize && erofs_is_buf_zeros(chunkdata, len)) {
> > + /* if data is all zeros, treat this block as hole */
> > + *(void **)idx++ = &erofs_holechunk;
> > + erofs_update_minextblks(sbi, interval_start, pos,
> > + &minextblks);
> > + interval_start = pos + len;
> > + lastch = NULL;
> > + continue;
> > + }
> > chunk = erofs_blob_getchunk(sbi, chunkdata, len);
>
> Yes, it's a valid opt. Yet, I guess we could calculate the unique hash value
> of all zeroes (with chunksize) in advance (e.g. when initialization).
>
> And compare the buf hash and the all-zeroed hash in erofs_blob_getchunk(), if
> they are the same, let's just compare all data then (or not compare since
> it's little chance to have such collision. In that case, erofs_blob_getchunk
> returns erofs_holechunk.
>
> Does it sound a good idea to you?
>
Hi Gao,
Sure, I will re-work this in v2.
Thanks,
Sandeep.
> Thanks,
> Gao Xiang
>
> > if (IS_ERR(chunk)) {
> > ret = PTR_ERR(chunk);
prev parent reply other threads:[~2024-04-04 16:53 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-04-03 23:57 [PATCH 0/1] Opportunistically making files sparse Sandeep Dhavale via Linux-erofs
2024-04-03 23:57 ` [PATCH 1/1] erofs-utils: lib: treat data blocks filled with 0s as a hole Sandeep Dhavale via Linux-erofs
2024-04-04 14:00 ` Gao Xiang
2024-04-04 16:52 ` Sandeep Dhavale via Linux-erofs [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAB=BE-RJq=Zt7X7BHfMuWU_w-ssRBWWdrt4+zWQNN-JZTdv7hA@mail.gmail.com' \
--to=linux-erofs@lists.ozlabs.org \
--cc=dhavale@google.com \
--cc=hsiangkao@linux.alibaba.com \
--cc=kernel-team@android.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).