From: Dave Chinner <david@fromorbit.com>
To: Hao Xu <hao.xu@linux.dev>
Cc: Wanpeng Li <wanpengli@tencent.com>,
"Darrick J . Wong" <djwong@kernel.org>,
Dominique Martinet <asmadeus@codewreck.org>,
linux-kernel@vger.kernel.org, linux-mm@kvack.org,
Stefan Roesch <shr@fb.com>, Clay Harris <bugs@claycon.org>,
linux-s390@vger.kernel.org, linux-nilfs@vger.kernel.org,
codalist@coda.cs.cmu.edu, cluster-devel@redhat.com,
linux-cachefs@redhat.com, linux-ext4@vger.kernel.org,
devel@lists.orangefs.org, linux-cifs@vger.kernel.org,
ecryptfs@vger.kernel.org, linux-nfs@vger.kernel.org,
linux-block@vger.kernel.org,
Alexander Viro <viro@zeniv.linux.org.uk>,
io-uring@vger.kernel.org, Jens Axboe <axboe@kernel.dk>,
Christian Brauner <brauner@kernel.org>,
netdev@vger.kernel.org, samba-technical@lists.samba.org,
linux-unionfs@vger.kernel.org,
linux-f2fs-devel@lists.sourceforge.net,
linux-xfs@vger.kernel.org, linux-fsdevel@vger.kernel.org,
linux-mtd@lists.infradead.org, bpf@vger.kernel.org,
Pavel Begunkov <asml.silence@gmail.com>,
linux-btrfs@vger.kernel.org
Subject: Re: [Cluster-devel] [PATCH RFC v5 00/29] io_uring getdents
Date: Sat, 26 Aug 2023 08:53:28 +1000 [thread overview]
Message-ID: <ZOkw6KkdP1UWPNBW@dread.disaster.area> (raw)
In-Reply-To: <20230825135431.1317785-1-hao.xu@linux.dev>
On Fri, Aug 25, 2023 at 09:54:02PM +0800, Hao Xu wrote:
> From: Hao Xu <howeyxu@tencent.com>
>
> This series introduce getdents64 to io_uring, the code logic is similar
> with the snychronized version's. It first try nowait issue, and offload
> it to io-wq threads if the first try fails.
>
> Patch1 and Patch2 are some preparation
> Patch3 supports nowait for xfs getdents code
> Patch4-11 are vfs change, include adding helpers and trylock for locks
> Patch12-29 supports nowait for involved xfs journal stuff
> note, Patch24 and 27 are actually two questions, might be removed later.
> an xfs test may come later.
You need to drop all the XFS journal stuff. It's fundamentally
broken as it stands, and we cannot support non-blocking
transactional changes without first putting a massive investment in
transaction and intent chain rollback to allow correctly undoing
partially complete modifications.
Regardless, non-blocking transactions are completely unnecessary for
a non-blocking readdir implementation. readdir should only be
touching atime, and with relatime it should only occur once every 24
hours per inode. If that's a problem, then we have noatime mount
options. Hence I just don't see any point in worrying about having a
timestamp update block occasionally...
I also don't really don't see why you need to fiddle with xfs buffer
cache semantics - it already has the functionality "nowait" buffer
reads require (i.e. XBF_INCORE|XBF_TRYLOCK).
However, the readahead IO that the xfs readdir code issues cannot
use your defined NOWAIT semantics - it must be able to allocate
memory and issue IO. Readahead already avoids blocking on memory
allocation and blocking on IO via the XBF_READ_AHEAD flag. This sets
__GFP_NORETRY for buffer allocation and REQ_RAHEAD for IO. Hence
readahead only needs the existing XBF_TRYLOCK flag to be set to be
compatible with the required NOWAIT semantics....
As for the NOIO memory allocation restrictions io_uring requires,
that should be enforced at the io_uring layer before calling into
the VFS using memalloc_noio_save/restore. At that point no memory
allocation will trigger IO and none of the code running under NOWAIT
conditions even needs to be aware that io_uring has a GFP_NOIO
restriction on memory allocation....
Please go back to the simple "do non-blocking buffer IO"
implementation we started with and don't try to solve every little
blocking problem that might exist in the VFS and filesystems...
-Dave
--
Dave Chinner
david@fromorbit.com
prev parent reply other threads:[~2023-08-25 22:53 UTC|newest]
Thread overview: 39+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-08-25 13:54 [Cluster-devel] [PATCH RFC v5 00/29] io_uring getdents Hao Xu
2023-08-25 13:54 ` [Cluster-devel] [PATCH 01/29] fs: split off vfs_getdents function of getdents64 syscall Hao Xu
2023-08-25 13:54 ` [Cluster-devel] [PATCH 02/29] xfs: rename XBF_TRYLOCK to XBF_NOWAIT Hao Xu
2023-08-25 21:39 ` Dave Chinner
2023-08-25 13:54 ` [Cluster-devel] [PATCH 03/29] xfs: add NOWAIT semantics for readdir Hao Xu
2023-08-25 13:54 ` [Cluster-devel] [PATCH 04/29] vfs: add nowait flag for struct dir_context Hao Xu
2023-08-25 13:54 ` [Cluster-devel] [PATCH 05/29] vfs: add a vfs helper for io_uring file pos lock Hao Xu
2023-08-25 13:54 ` [Cluster-devel] [PATCH 06/29] vfs: add file_pos_unlock() for io_uring usage Hao Xu
2023-08-25 13:54 ` [Cluster-devel] [PATCH 07/29] vfs: add a nowait parameter for touch_atime() Hao Xu
2023-08-25 13:54 ` [Cluster-devel] [PATCH 08/29] vfs: add nowait parameter for file_accessed() Hao Xu
2023-08-25 13:54 ` [Cluster-devel] [PATCH 09/29] vfs: move file_accessed() to the beginning of iterate_dir() Hao Xu
2023-08-25 13:54 ` [Cluster-devel] [PATCH 10/29] vfs: add S_NOWAIT for nowait time update Hao Xu
2023-08-25 13:54 ` [Cluster-devel] [PATCH 11/29] vfs: trylock inode->i_rwsem in iterate_dir() to support nowait Hao Xu
2023-08-25 13:54 ` [Cluster-devel] [PATCH 12/29] xfs: enforce GFP_NOIO implicitly during nowait time update Hao Xu
2023-08-25 14:20 ` Matthew Wilcox
2023-08-25 13:54 ` [Cluster-devel] [PATCH 13/29] xfs: make xfs_trans_alloc() support nowait semantics Hao Xu
2023-08-25 13:54 ` [Cluster-devel] [PATCH 14/29] xfs: support nowait for xfs_log_reserve() Hao Xu
2023-08-25 13:54 ` [Cluster-devel] [PATCH 15/29] xfs: don't wait for free space in xlog_grant_head_check() in nowait case Hao Xu
2023-08-25 13:54 ` [Cluster-devel] [PATCH 16/29] xfs: add nowait parameter for xfs_inode_item_init() Hao Xu
2023-08-25 13:54 ` [Cluster-devel] [PATCH 17/29] xfs: make xfs_trans_ijoin() error out -EAGAIN Hao Xu
2023-08-25 13:54 ` [Cluster-devel] [PATCH 18/29] xfs: set XBF_NOWAIT for xfs_buf_read_map if necessary Hao Xu
2023-08-25 13:54 ` [Cluster-devel] [PATCH 19/29] xfs: support nowait memory allocation in _xfs_buf_alloc() Hao Xu
2023-08-25 13:54 ` [Cluster-devel] [PATCH 20/29] xfs: distinguish error type of memory allocation failure for nowait case Hao Xu
2023-08-25 13:54 ` [Cluster-devel] [PATCH 21/29] xfs: return -EAGAIN when bulk memory allocation fails in " Hao Xu
2023-08-25 13:54 ` [Cluster-devel] [PATCH 22/29] xfs: comment page allocation for nowait case in xfs_buf_find_insert() Hao Xu
2023-08-25 14:09 ` Matthew Wilcox
2023-08-25 13:54 ` [Cluster-devel] [PATCH 23/29] xfs: don't print warn info for -EAGAIN error in xfs_buf_get_map() Hao Xu
2023-08-25 13:54 ` [Cluster-devel] [PATCH 24/29] xfs: support nowait for xfs_buf_read_map() Hao Xu
2023-08-25 21:53 ` Dave Chinner
2023-08-25 13:54 ` [Cluster-devel] [PATCH 25/29] xfs: support nowait for xfs_buf_item_init() Hao Xu
2023-08-25 22:16 ` Dave Chinner
2023-08-25 13:54 ` [Cluster-devel] [PATCH 26/29] xfs: return -EAGAIN when nowait meets sync in transaction commit Hao Xu
2023-08-25 21:58 ` Dave Chinner
2023-08-25 13:54 ` [Cluster-devel] [PATCH 27/29] xfs: add a comment for xlog_kvmalloc() Hao Xu
2023-08-25 13:54 ` [Cluster-devel] [PATCH 28/29] xfs: support nowait semantics for xc_ctx_lock in xlog_cil_commit() Hao Xu
2023-08-25 21:59 ` Dave Chinner
2023-08-25 13:54 ` [Cluster-devel] [PATCH 29/29] io_uring: add support for getdents Hao Xu
2023-08-25 15:11 ` [Cluster-devel] [PATCH RFC v5 00/29] io_uring getdents Darrick J. Wong
2023-08-25 22:53 ` Dave Chinner [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ZOkw6KkdP1UWPNBW@dread.disaster.area \
--to=david@fromorbit.com \
--cc=asmadeus@codewreck.org \
--cc=asml.silence@gmail.com \
--cc=axboe@kernel.dk \
--cc=bpf@vger.kernel.org \
--cc=brauner@kernel.org \
--cc=bugs@claycon.org \
--cc=cluster-devel@redhat.com \
--cc=codalist@coda.cs.cmu.edu \
--cc=devel@lists.orangefs.org \
--cc=djwong@kernel.org \
--cc=ecryptfs@vger.kernel.org \
--cc=hao.xu@linux.dev \
--cc=io-uring@vger.kernel.org \
--cc=linux-block@vger.kernel.org \
--cc=linux-btrfs@vger.kernel.org \
--cc=linux-cachefs@redhat.com \
--cc=linux-cifs@vger.kernel.org \
--cc=linux-ext4@vger.kernel.org \
--cc=linux-f2fs-devel@lists.sourceforge.net \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-mtd@lists.infradead.org \
--cc=linux-nfs@vger.kernel.org \
--cc=linux-nilfs@vger.kernel.org \
--cc=linux-s390@vger.kernel.org \
--cc=linux-unionfs@vger.kernel.org \
--cc=linux-xfs@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=samba-technical@lists.samba.org \
--cc=shr@fb.com \
--cc=viro@zeniv.linux.org.uk \
--cc=wanpengli@tencent.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).