From: Hao Xu <hao.xu@linux.dev>
To: io-uring@vger.kernel.org, Jens Axboe <axboe@kernel.dk>
Cc: Dominique Martinet <asmadeus@codewreck.org>,
Pavel Begunkov <asml.silence@gmail.com>,
Christian Brauner <brauner@kernel.org>,
Alexander Viro <viro@zeniv.linux.org.uk>,
Stefan Roesch <shr@fb.com>, Clay Harris <bugs@claycon.org>,
Dave Chinner <david@fromorbit.com>,
"Darrick J . Wong" <djwong@kernel.org>,
linux-fsdevel@vger.kernel.org, linux-xfs@vger.kernel.org,
linux-ext4@vger.kernel.org, linux-cachefs@redhat.com,
ecryptfs@vger.kernel.org, linux-nfs@vger.kernel.org,
linux-unionfs@vger.kernel.org, bpf@vger.kernel.org,
netdev@vger.kernel.org, linux-s390@vger.kernel.org,
linux-kernel@vger.kernel.org, linux-block@vger.kernel.org,
linux-btrfs@vger.kernel.org, codalist@coda.cs.cmu.edu,
linux-f2fs-devel@lists.sourceforge.net, cluster-devel@redhat.com,
linux-mm@kvack.org, linux-nilfs@vger.kernel.org,
devel@lists.orangefs.org, linux-cifs@vger.kernel.org,
samba-techni
Subject: [PATCH v6 00/11] io_uring getdents
Date: Sun, 27 Aug 2023 21:28:24 +0800 [thread overview]
Message-ID: <20230827132835.1373581-1-hao.xu@linux.dev> (raw)
From: Hao Xu <howeyxu@tencent.com>
This series introduce getdents64 to io_uring, the code logic is similar
with the snychronized version's. It first try nowait issue, and offload
it to io-wq threads if the first try fails.
Patch1 and Patch2 are some preparation
Patch3 supports nowait for xfs getdents code
Patch4-11 are vfs change, include adding helpers and trylock for locks
Tests I've done:
A liburing test case for functional test:
https://github.com/HowHsu/liburing/commit/39dc9a8e19c06a8cebf8c2301b85320eb45c061e?diff=unified
Tested it with a liburing performance test:
https://github.com/HowHsu/liburing/blob/getdents/test/getdents2.c
The test is controlled by the below script[2] which runs getdents2.t 100
times and calulate the avg.
The result show that io_uring version is about 2.6% faster:
note:
[1] the number of getdents call/request in io_uring and normal sync version
are made sure to be same beforehand.
[2] run_getdents.py
```python3
import subprocess
N = 100
sum = 0.0
args = ["/data/home/howeyxu/tmpdir", "sync"]
for i in range(N):
output = subprocess.check_output(["./liburing/test/getdents2.t"] + args)
sum += float(output)
average = sum / N
print("Average of sync:", average)
sum = 0.0
args = ["/data/home/howeyxu/tmpdir", "iouring"]
for i in range(N):
output = subprocess.check_output(["./liburing/test/getdents2.t"] + args)
sum += float(output)
average = sum / N
print("Average of iouring:", average)
```
v5->v6:
- remove xfs journal stuff since there are fundamental issues in the
design.
v4->v5:
- move atime update to the beginning of getdents operation
- trylock for i_rwsem
- nowait semantics for involved xfs journal stuff
v3->v4:
- add Dave's xfs nowait code and fix a deadlock problem, with some code
style tweak.
- disable fixed file to avoid a race problem for now
- add a test program.
v2->v3:
- removed the kernfs patches
- add f_pos_lock logic
- remove the "reduce last EOF getdents try" optimization since
Dominique reports that doesn't make difference
- remove the rewind logic, I think the right way is to introduce lseek
to io_uring not to patch this logic to getdents.
- add Singed-off-by of Stefan Roesch for patch 1 since checkpatch
complained that Co-developed-by someone should be accompanied with
Signed-off-by same person, I can remove them if Stefan thinks that's
not proper.
Dominique Martinet (1):
fs: split off vfs_getdents function of getdents64 syscall
Hao Xu (10):
xfs: add NOWAIT semantics for readdir
vfs: add nowait flag for struct dir_context
vfs: add a vfs helper for io_uring file pos lock
vfs: add file_pos_unlock() for io_uring usage
vfs: add a nowait parameter for touch_atime()
vfs: add nowait parameter for file_accessed()
vfs: move file_accessed() to the beginning of iterate_dir()
vfs: error out -EAGAIN if atime needs to be updated
vfs: trylock inode->i_rwsem in iterate_dir() to support nowait
io_uring: add support for getdents
arch/s390/hypfs/inode.c | 2 +-
block/fops.c | 2 +-
fs/btrfs/file.c | 2 +-
fs/btrfs/inode.c | 2 +-
fs/cachefiles/namei.c | 2 +-
fs/coda/dir.c | 4 +--
fs/ecryptfs/file.c | 4 +--
fs/ext2/file.c | 4 +--
fs/ext4/file.c | 6 ++--
fs/f2fs/file.c | 4 +--
fs/file.c | 13 ++++++++
fs/fuse/dax.c | 2 +-
fs/fuse/file.c | 4 +--
fs/gfs2/file.c | 2 +-
fs/hugetlbfs/inode.c | 2 +-
fs/inode.c | 10 ++++--
fs/internal.h | 8 +++++
fs/namei.c | 4 +--
fs/nfsd/vfs.c | 2 +-
fs/nilfs2/file.c | 2 +-
fs/orangefs/file.c | 2 +-
fs/orangefs/inode.c | 2 +-
fs/overlayfs/file.c | 2 +-
fs/overlayfs/inode.c | 2 +-
fs/pipe.c | 2 +-
fs/ramfs/file-nommu.c | 2 +-
fs/readdir.c | 61 ++++++++++++++++++++++++++--------
fs/smb/client/cifsfs.c | 2 +-
fs/splice.c | 2 +-
fs/stat.c | 2 +-
fs/ubifs/file.c | 2 +-
fs/udf/file.c | 2 +-
fs/xfs/libxfs/xfs_da_btree.c | 16 +++++++++
fs/xfs/libxfs/xfs_da_btree.h | 1 +
fs/xfs/libxfs/xfs_dir2_block.c | 7 ++--
fs/xfs/libxfs/xfs_dir2_priv.h | 2 +-
fs/xfs/scrub/dir.c | 2 +-
fs/xfs/scrub/readdir.c | 2 +-
fs/xfs/xfs_dir2_readdir.c | 49 +++++++++++++++++++++------
fs/xfs/xfs_file.c | 6 ++--
fs/xfs/xfs_inode.c | 27 +++++++++++++++
fs/xfs/xfs_inode.h | 17 ++++++----
fs/zonefs/file.c | 4 +--
include/linux/file.h | 7 ++++
include/linux/fs.h | 15 +++++++--
include/uapi/linux/io_uring.h | 1 +
io_uring/fs.c | 53 +++++++++++++++++++++++++++++
io_uring/fs.h | 3 ++
io_uring/opdef.c | 8 +++++
kernel/bpf/inode.c | 4 +--
mm/filemap.c | 8 ++---
mm/shmem.c | 6 ++--
net/unix/af_unix.c | 4 +--
53 files changed, 310 insertions(+), 96 deletions(-)
--
2.25.1
next reply other threads:[~2023-08-27 13:28 UTC|newest]
Thread overview: 28+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-08-27 13:28 Hao Xu [this message]
2023-08-27 13:28 ` [PATCH 01/11] fs: split off vfs_getdents function of getdents64 syscall Hao Xu
2023-08-27 13:28 ` [PATCH 02/11] xfs: add NOWAIT semantics for readdir Hao Xu
2023-08-27 20:44 ` Matthew Wilcox
2023-08-29 7:41 ` Hao Xu
2023-08-29 13:05 ` Matthew Wilcox
2023-09-04 1:02 ` Dave Chinner
2023-08-27 13:28 ` [PATCH 03/11] vfs: add nowait flag for struct dir_context Hao Xu
2023-08-27 13:28 ` [PATCH 04/11] vfs: add a vfs helper for io_uring file pos lock Hao Xu
2023-08-27 20:47 ` Matthew Wilcox
2023-08-27 13:28 ` [PATCH 05/11] vfs: add file_pos_unlock() for io_uring usage Hao Xu
2023-08-27 13:28 ` [PATCH 06/11] vfs: add a nowait parameter for touch_atime() Hao Xu
2023-08-27 13:28 ` [PATCH 07/11] vfs: add nowait parameter for file_accessed() Hao Xu
2023-08-27 21:32 ` Matthew Wilcox
2023-08-29 7:46 ` Hao Xu
2023-08-29 11:53 ` Matthew Wilcox
2023-08-30 6:11 ` Hao Xu
2023-09-03 22:30 ` Dave Chinner
2023-09-08 0:29 ` Pavel Begunkov
2023-09-10 22:01 ` Dave Chinner
2023-09-04 9:51 ` Christian Brauner
2023-08-27 13:28 ` [PATCH 08/11] vfs: move file_accessed() to the beginning of iterate_dir() Hao Xu
2023-08-27 13:28 ` [PATCH 09/11] vfs: error out -EAGAIN if atime needs to be updated Hao Xu
2023-08-27 20:51 ` Matthew Wilcox
2023-08-27 13:28 ` [PATCH 10/11] vfs: trylock inode->i_rwsem in iterate_dir() to support nowait Hao Xu
2023-09-04 9:37 ` Christian Brauner
2023-08-27 13:28 ` [PATCH 11/11] io_uring: add support for getdents Hao Xu
2023-09-04 9:57 ` [PATCH v6 00/11] io_uring getdents Christian Brauner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230827132835.1373581-1-hao.xu@linux.dev \
--to=hao.xu@linux.dev \
--cc=asmadeus@codewreck.org \
--cc=asml.silence@gmail.com \
--cc=axboe@kernel.dk \
--cc=bpf@vger.kernel.org \
--cc=brauner@kernel.org \
--cc=bugs@claycon.org \
--cc=cluster-devel@redhat.com \
--cc=codalist@coda.cs.cmu.edu \
--cc=david@fromorbit.com \
--cc=devel@lists.orangefs.org \
--cc=djwong@kernel.org \
--cc=ecryptfs@vger.kernel.org \
--cc=io-uring@vger.kernel.org \
--cc=linux-block@vger.kernel.org \
--cc=linux-btrfs@vger.kernel.org \
--cc=linux-cachefs@redhat.com \
--cc=linux-cifs@vger.kernel.org \
--cc=linux-ext4@vger.kernel.org \
--cc=linux-f2fs-devel@lists.sourceforge.net \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-nfs@vger.kernel.org \
--cc=linux-nilfs@vger.kernel.org \
--cc=linux-s390@vger.kernel.org \
--cc=linux-unionfs@vger.kernel.org \
--cc=linux-xfs@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=shr@fb.com \
--cc=viro@zeniv.linux.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).