From: Hannes Reinecke <hare@kernel.org>
To: Christoph Hellwig <hch@lst.de>
Cc: Keith Busch <kbusch@kernel.org>, Sagi Grimberg <sagi@grimberg.me>,
Jens Axboe <axboe@kernel.dk>,
linux-nvme@lists.infradead.org, linux-block@vger.kernel.org,
Hannes Reinecke <hare@kernel.org>
Subject: [PATCHv2 0/2] block,nvme: latency-based I/O scheduler
Date: Wed, 3 Apr 2024 16:17:54 +0200 [thread overview]
Message-ID: <20240403141756.88233-1-hare@kernel.org> (raw)
Hi all,
there had been several attempts to implement a latency-based I/O
scheduler for native nvme multipath, all of which had its issues.
So time to start afresh, this time using the QoS framework
already present in the block layer.
It consists of two parts:
- a new 'blk-nlatency' QoS module, which is just a simple per-node
latency tracker
- a 'latency' nvme I/O policy
Using the 'tiobench' fio script with 512 byte blocksize I'm getting
the following latencies (in usecs) as a baseline:
- seq write: avg 186 stddev 331
- rand write: avg 4598 stddev 7903
- seq read: avg 149 stddev 65
- rand read: avg 150 stddev 68
Enabling the 'latency' iopolicy:
- seq write: avg 178 stddev 113
- rand write: avg 3427 stddev 6703
- seq read: avg 140 stddev 59
- rand read: avg 141 stddev 58
Setting the 'decay' parameter to 10:
- seq write: avg 182 stddev 65
- rand write: avg 2619 stddev 5894
- seq read: avg 142 stddev 57
- rand read: avg 140 stddev 57
That's on a 32G FC testbed running against a brd target,
fio running with 48 threads. So promises are met: latency
goes down, and we're even able to control the standard
deviation via the 'decay' parameter.
As usual, comments and reviews are welcome.
Changes to the original version:
- split the rqos debugfs entries
- Modify commit message to indicate latency
- rename to blk-nlatency
Hannes Reinecke (2):
block: track per-node I/O latency
nvme: add 'latency' iopolicy
block/Kconfig | 6 +
block/Makefile | 1 +
block/blk-mq-debugfs.c | 2 +
block/blk-nlatency.c | 388 ++++++++++++++++++++++++++++++++++
block/blk-rq-qos.h | 6 +
drivers/nvme/host/multipath.c | 57 ++++-
drivers/nvme/host/nvme.h | 1 +
include/linux/blk-mq.h | 11 +
8 files changed, 465 insertions(+), 7 deletions(-)
create mode 100644 block/blk-nlatency.c
--
2.35.3
next reply other threads:[~2024-04-03 14:18 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-04-03 14:17 Hannes Reinecke [this message]
2024-04-03 14:17 ` [PATCH 1/2] block: track per-node I/O latency Hannes Reinecke
2024-04-04 2:22 ` kernel test robot
2024-04-04 2:55 ` kernel test robot
2024-04-04 18:47 ` kernel test robot
2024-04-03 14:17 ` [PATCH 2/2] nvme: add 'latency' iopolicy Hannes Reinecke
2024-04-04 21:14 ` [PATCHv2 0/2] block,nvme: latency-based I/O scheduler Keith Busch
2024-04-05 6:21 ` Hannes Reinecke
2024-04-05 15:03 ` Keith Busch
2024-04-05 15:36 ` Hannes Reinecke
2024-04-07 19:55 ` Sagi Grimberg
2024-05-09 20:43 ` [PATCH v3 0/3] " John Meneghini
2024-05-10 9:34 ` Niklas Cassel
2024-05-09 20:43 ` [PATCH v3 1/3] block: track per-node I/O latency John Meneghini
2024-05-10 7:11 ` Damien Le Moal
2024-05-10 9:28 ` Niklas Cassel
2024-05-10 10:00 ` Hannes Reinecke
2024-05-09 20:43 ` [PATCH v3 2/3] nvme: add 'latency' iopolicy John Meneghini
2024-05-10 7:17 ` Damien Le Moal
2024-05-10 10:03 ` Hannes Reinecke
2024-05-09 20:43 ` [PATCH v3 3/3] nvme: multipath: pr_notice when iopolicy changes John Meneghini
2024-05-10 7:19 ` Damien Le Moal
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240403141756.88233-1-hare@kernel.org \
--to=hare@kernel.org \
--cc=axboe@kernel.dk \
--cc=hch@lst.de \
--cc=kbusch@kernel.org \
--cc=linux-block@vger.kernel.org \
--cc=linux-nvme@lists.infradead.org \
--cc=sagi@grimberg.me \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).