Linux-bcache Archive on lore.kernel.org
 help / color / mirror / Atom feed

From: 許耀宗 <yaozhonghsu@gmail.com>
To: linux-bcache@vger.kernel.org, kent.overstreet@gmail.com
Cc: everest@synology.com, norrishsu@synology.com
Subject: bcache: Data is inconsistent when the cache device failed in writearound mode
Date: Thu, 24 Nov 2022 15:57:38 +0800	[thread overview]
Message-ID: <CABeQj8kYqMFVZ3OOAT9svsXg-GHTJC+N6G4_gtGmBwoC0=xseg@mail.gmail.com> (raw)

Hi linux-bcache,

I was evaluating the performance and stability of bcache.
In my test environment, I was using bcache as a read cache in
writearound cache mode running on OS ubuntu 20.04 with linux kernel
5.15.
However, I found that sometimes the data was inconsistent, and here is
how to reproduce:

1. Set up bcache with 1 hard drive and 1 read cache

    $ make-bcache -B /dev/sdb -C /dev/sdc

2. Set as writearound mode

    $ echo "writearound" > /sys/block/bcache0/bcache/cache_mode

3. Create XFS on /dev/bcache0

    $ mkfs.xfs -m reflink=0 -f /dev/bcache0

4. Mount the file system

    $ mkdir /srv/node/ -p
    $ mount -t xfs /dev/bcache0 /srv/node -o
rw,nofail,noexec,nodev,noatime,nodiratime,inode64,logbsize=256k

5. Prepare a 50G file with random content

    $ head -c 50G < /dev/urandom > 50G

6. Get md5sum of this 50G file

    $ md5sum 50G
    99c91956af4eec68707b329902062bfd

7. Copy the 50G file to the mount point

    $ cp 50G /srv/node/

8. During the copy operation, remove the cache device manually

    $ echo 1 > /sys/block/sdc/device/delete

9. Wait for the copy operation done, then calculate md5sum of 50G file
in the destination

    $ md5sum /srv/node/50G
    bb178bbb9c0ca07c202580e3ea9e62c5

The result shows the md5sum of two 50G file are not the same.
As far as I know, no data is written to cache device in writearound
mode. Therefore a failed cache device should not cause any data
inconsistency.

There are some further observations,
1. I can reproduce it with kernel 5.04 / 5.14.
2. I can reproduce it with ubuntu 22.04.
3. I cannot reproduce if the file is not large enough, like 10G and 20G.

Please advise me how to fix it. Thank you.

Sincerely,
Norris Hsu

                 reply	other threads:[~2022-11-24  7:57 UTC|newest]

Thread overview: [no followups] expand[flat|nested]  mbox.gz  Atom feed

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CABeQj8kYqMFVZ3OOAT9svsXg-GHTJC+N6G4_gtGmBwoC0=xseg@mail.gmail.com' \
    --to=yaozhonghsu@gmail.com \
    --cc=everest@synology.com \
    --cc=kent.overstreet@gmail.com \
    --cc=linux-bcache@vger.kernel.org \
    --cc=norrishsu@synology.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).