From: Eric Van Hensbergen <ericvh@gmail.com>
To: Jeff Layton <jlayton@kernel.org>
Cc: Christian Schoenebeck <linux_oss@crudebyte.com>,
Luis Chamberlain <mcgrof@kernel.org>,
Dominique Martinet <asmadeus@codewreck.org>,
Josef Bacik <josef@toxicpanda.com>,
lucho@ionkov.net, v9fs-developer@lists.sourceforge.net,
linux-kernel@vger.kernel.org,
Amir Goldstein <amir73il@gmail.com>,
Pankaj Raghav <p.raghav@samsung.com>,
v9fs@lists.linux.dev
Subject: Re: 9p caching with cache=loose and cache=fscache
Date: Fri, 31 Mar 2023 11:47:12 -0500 [thread overview]
Message-ID: <CAFkjPT=gJBELBg1gCjYFrZKVp5fy1vmidByOstB8tfqcuCUvLA@mail.gmail.com> (raw)
In-Reply-To: <6f89f0ac34956e7f527c7efa3d162b4a1f5ea71a.camel@kernel.org>
I like the sliding window concept - I wasn't aware NFS was doing that,
I'll have a look as part of my rework.
The unmount/mount should indeed flush any cache (unless using
fscache), so that might be a good workaround if it can be automated in
the workflow.
-eric
On Wed, Mar 29, 2023 at 6:32 AM Jeff Layton <jlayton@kernel.org> wrote:
>
> On Wed, 2023-03-29 at 13:19 +0200, Christian Schoenebeck wrote:
> > On Wednesday, March 29, 2023 12:08:26 AM CEST Dominique Martinet wrote:
> > > Luis Chamberlain wrote on Tue, Mar 28, 2023 at 10:41:02AM -0700:
> > > > > "To speedup things you can also consider to use e.g. cache=loose instead.
> > > >
> > > > My experience is that cache=loose is totally useless.
> > >
> > > If the fs you mount isn't accessed by the host while the VM is up, and
> > > isn't shared with another guest (e.g. "exclusive share"), you'll get
> > > what you expect.
> > >
> > > I have no idea what people use qemu's virtfs for but this is apparently
> > > common enough that it was recommended before without anyone complaining
> > > since that started being recommended in 2011[1] until now?
> > >
> > > [1] https://wiki.qemu.org/index.php?title=Documentation/9psetup&diff=2178&oldid=2177
> > >
> > > (now I'm not arguing it should be recommended, my stance as a 9p
> > > maintainer is that the default should be used unless you know what
> > > you're doing, so the new code should just remove the 'cache=none'
> > > altogether as that's the default.
> > > With the new cache models Eric is preparing comes, we'll get a new safe
> > > default that will likely be better than cache=none, there is no reason
> > > to explicitly recommend the historic safe model as the default has
> > > always been on the safe side and we have no plan of changing that.)
> >
> > It's not that I receive a lot of feedback for what people use 9p for, nor am I
> > QEMU's 9p maintainer for a long time, but so far contributors cared more about
> > performance and other issues than propagating changes host -> guest without
> > reboot/remount/drop_caches. At least they did not care enough to work on
> > patches.
> >
> > Personally I also use cache=loose and only need to push changes host->guest
> > once in a while.
> >
> > > > > That will deploy a filesystem cache on guest side and reduces the amount of
> > > > > 9p requests to hosts. As a consequence however guest might not see file
> > > > > changes performed on host side *at* *all*
> > > >
> > > > I think that makes it pretty useless, aren't most setups on the guest read-only?
> > > >
> > > > It is not about "may not see", just won't. For example I modified the
> > > > Makefile and compiled a full kernel and even with those series of
> > > > changes, the guest *minutes later* never saw any updates.
> > >
> > > read-only on the guest has nothing to do with it, nor has time.
> > >
> > > If the directory is never accessed on the guest before the kernel has
> > > been built, you'll be able to make install on the guest -- once, even if
> > > the build was done after the VM booted and fs mounted.
> > >
> > > After it's been read once, it'll stay in cache until memory pressure (or
> > > an admin action like umount/mount or sysctl vm.drop_caches=3) clears it.
> > >
> > > I believe that's why it appeared to work until you noticed the issue and
> > > had to change the mount option -- I'd expect in most case you'll run
> > > make install once and reboot/kexec into the new kernel.
> > >
> > > It's not safe for your usecase and cache=none definitely sounds better
> > > to me, but people should use defaults make their own informed decision.
> >
> > It appears to me that read-only seems not to be the average use case for 9p,
> > at least from the command lines I received. It is often used in combination
> > with overlayfs though.
> >
> > I (think) the reason why cache=loose was recommended as default option on the
> > QEMU wiki page ages ago, was because of its really poor performance at that
> > point. I would personally not go that far and discourage people from using
> > cache=loose in general, as long as they get informed about the consequences.
> > You still get a great deal of performance boost, the rest is for each
> > individual to decide.
> >
> > Considering that Eric already has patches for revalidating the cache in the
> > works, I think the changes I made on the other QEMU wiki page are appropriate,
> > including the word "might" as it's soon only a matter of kernel version.
> >
> > > > > In the above example the folder /home/guest/9p_setup/ shared of the
> > > > > host is shared with the folder /tmp/shared on the guest. We use no
> > > > > cache because current caching mechanisms need more work and the
> > > > > results are not what you would expect."
> > > >
> > > > I got a wiki account now and I was the one who had clarified this.
> > >
> > > Thanks for helping making this clearer.
> >
> > Yep, and thanks for making a wiki account and improving the content there
> > directly. Always appreciated!
> >
>
> Catching up on this thread.
>
> Getting cache coherency right on a network filesystem is quite
> difficult. It's always a balance between correctness and performance.
>
> Some protocols (e.g. CIFS and Ceph) take a very heavy-handed approach to
> try ensure that the caches are always coherent. Basically, these clients
> are only allowed to cache when the server grants permission for it. That
> can have a negative effect on performance, of course.
>
> NFS as a protocol is more "loose", but we've generally beat its cache
> coherency mechanisms into shape over the years, so you don't see these
> sorts of problems there as much. FWIW, NFS uses a sliding time window to
> revalidate the cache, such that it'll revalidate frequently when an
> inodes is changing frequently, but less so when it's more stable.
>
> 9P I haven't worked with as much, but it sounds like it doesn't try to
> keep caches coherent (at least not with cache=loose).
>
> Probably the simplest solution here is to simply unmount/mount before
> you have the clients call "make modules_install && make install". That
> should ensure that the client doesn't have any stale data in the cache
> when the time comes to do the reads. A full reboot shouldn't be
> required.
>
> --
> Jeff Layton <jlayton@kernel.org>
parent reply other threads:[~2023-03-31 16:47 UTC|newest]
Thread overview: expand[flat|nested] mbox.gz Atom feed
[parent not found: <6f89f0ac34956e7f527c7efa3d162b4a1f5ea71a.camel@kernel.org>]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAFkjPT=gJBELBg1gCjYFrZKVp5fy1vmidByOstB8tfqcuCUvLA@mail.gmail.com' \
--to=ericvh@gmail.com \
--cc=amir73il@gmail.com \
--cc=asmadeus@codewreck.org \
--cc=jlayton@kernel.org \
--cc=josef@toxicpanda.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux_oss@crudebyte.com \
--cc=lucho@ionkov.net \
--cc=mcgrof@kernel.org \
--cc=p.raghav@samsung.com \
--cc=v9fs-developer@lists.sourceforge.net \
--cc=v9fs@lists.linux.dev \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).