From: Hannes Reinecke <hare@kernel.org>
To: Jens Axboe <axboe@kernel.dk>
Cc: Keith Busch <kbusch@kernel.org>, Christoph Hellwig <hch@lst.de>,
Sagi Grimberg <sagi@grimberg.me>,
linux-nvme@lists.infradead.org, linux-block@vger.kernel.org,
Hannes Reinecke <hare@kernel.org>
Subject: [PATCH RFC 0/2] block,nvme: latency-based I/O scheduler
Date: Tue, 26 Mar 2024 16:35:27 +0100 [thread overview]
Message-ID: <20240326153529.75989-1-hare@kernel.org> (raw)
Hi all,
there had been several attempts to implement a latency-based I/O
scheduler for native nvme multipath, all of which had its issues.
So time to start afresh, this time using the QoS framework
already present in the block layer.
It consists of two parts:
- a new 'blk-nodelat' QoS module, which is just a simple per-node
latency tracker
- a 'latency' nvme I/O policy
Using the 'tiobench' fio script I'm getting:
WRITE: bw=531MiB/s (556MB/s), 33.2MiB/s-52.4MiB/s
(34.8MB/s-54.9MB/s), io=4096MiB (4295MB), run=4888-7718msec
WRITE: bw=539MiB/s (566MB/s), 33.7MiB/s-50.9MiB/s
(35.3MB/s-53.3MB/s), io=4096MiB (4295MB), run=5033-7594msec
READ: bw=898MiB/s (942MB/s), 56.1MiB/s-75.4MiB/s
(58.9MB/s-79.0MB/s), io=4096MiB (4295MB), run=3397-4560msec
READ: bw=1023MiB/s (1072MB/s), 63.9MiB/s-75.1MiB/s
(67.0MB/s-78.8MB/s), io=4096MiB (4295MB), run=3408-4005msec
for 'round-robin' and
WRITE: bw=574MiB/s (601MB/s), 35.8MiB/s-45.5MiB/s
(37.6MB/s-47.7MB/s), io=4096MiB (4295MB), run=5629-7142msec
WRITE: bw=639MiB/s (670MB/s), 39.9MiB/s-47.5MiB/s
(41.9MB/s-49.8MB/s), io=4096MiB (4295MB), run=5388-6408msec
READ: bw=1024MiB/s (1074MB/s), 64.0MiB/s-73.7MiB/s
(67.1MB/s-77.2MB/s), io=4096MiB (4295MB), run=3475-4000msec
READ: bw=1013MiB/s (1063MB/s), 63.3MiB/s-72.6MiB/s
(66.4MB/s-76.2MB/s), io=4096MiB (4295MB), run=3524-4042msec
for 'latency' with 'decay' set to 10.
That's on a 32G FC testbed running against a brd target,
fio running with 16 thread.
As usual, comments and reviews are welcome.
Hannes Reinecke (2):
block: track per-node I/O latency
nvme: add 'latency' iopolicy
block/Kconfig | 7 +
block/Makefile | 1 +
block/blk-mq-debugfs.c | 2 +
block/blk-nodelat.c | 368 ++++++++++++++++++++++++++++++++++
block/blk-rq-qos.h | 6 +
drivers/nvme/host/multipath.c | 46 ++++-
drivers/nvme/host/nvme.h | 2 +
include/linux/blk-mq.h | 11 +
8 files changed, 439 insertions(+), 4 deletions(-)
create mode 100644 block/blk-nodelat.c
--
2.35.3
next reply other threads:[~2024-03-26 15:37 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-03-26 15:35 Hannes Reinecke [this message]
2024-03-26 15:35 ` [PATCH 1/2] block: track per-node I/O latency Hannes Reinecke
2024-03-27 18:03 ` kernel test robot
2024-03-27 20:59 ` kernel test robot
2024-03-26 15:35 ` [PATCH 2/2] nvme: add 'latency' iopolicy Hannes Reinecke
2024-03-28 10:38 ` [PATCH RFC 0/2] block,nvme: latency-based I/O scheduler Sagi Grimberg
2024-03-28 11:32 ` Hannes Reinecke
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240326153529.75989-1-hare@kernel.org \
--to=hare@kernel.org \
--cc=axboe@kernel.dk \
--cc=hch@lst.de \
--cc=kbusch@kernel.org \
--cc=linux-block@vger.kernel.org \
--cc=linux-nvme@lists.infradead.org \
--cc=sagi@grimberg.me \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).