From: Hitoshi Mitake <mitake.hitoshi@gmail.com>
To: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
Cc: stgt <stgt@vger.kernel.org>
Subject: Re: [PATCH v4 RFC 0/5] tgtd: offload iSCSI PDU send/recv to worker threads
Date: Wed, 19 Feb 2014 12:11:39 +0900 [thread overview]
Message-ID: <CAE1WaK+gMw1pxrB5h+_wr1ZM+gmKV-vCTv3EOOUG++cymgFgXg@mail.gmail.com> (raw)
In-Reply-To: <1392376893-26106-1-git-send-email-mitake.hitoshi@lab.ntt.co.jp>
Sorry, please discard this patchset. We found that the previous commit:
https://github.com/fujita/tgt/commit/0e23d5754d3bc3cf0328255fa7dab8547ecb6e60
improves throughput of tgtd enough.
We've evaluated this patchset based on RHEL's tgt, which doesn't have
the above commit.
Thanks,
Hitoshi
On Fri, Feb 14, 2014 at 8:21 PM, Hitoshi Mitake
<mitake.hitoshi@lab.ntt.co.jp> wrote:
> Current tgtd sends and receives iSCSI PDUs in its main event
> loop. This design can cause bottleneck when many iSCSI clients connect
> to single tgtd process. For example, we need multiple tgtd processes
> for utilizing fast network like 10 GbE because typical single
> processor core isn't fast enough for processing a bunch of requests.
>
> This patchset lets tgtd offload send/recv iSCSI PDUs and digest checking to
> worker threads. The basic strategy of this change is like below:
> 1. decompose iscsi_[rt]x_handler()
> 2. re-implement single threaded version of iscsi_tcp_event_handler()
> 3. implement multi-threaded version of iscsi_tcp_event_handler()
>
> This patch also adds a new option "-T" to specify a number of threads which
> send/recieve PDUs. When 1 is passed with the option, the above single threaded
> version will be used, because the multi-threaded version incurs overhead which
> comes from frequent communication between threads in some case. If users don't
> want to use the multi-threaded version, they don't have to use it.
>
> Below is a summary of our performance evaluation:
> - 4 physical hosts connected with 10Gbps ethernet
> - 1 tgtd process provides 1 logical unit
> - 16 VMs read 4GB iso file by dd command in parallel
> -- average time required to complete the dd command of 16 VMs is a score
> --- original tgtd: 93.718 second
> --- changed tgtd (with -T 16): 57.449 second
>
> The above scores show that this patchset can improve performance of parallel
> access of initiators. This patchset is not so heavily tested yet. I'd like to
> hear your opinion about the design.
>
> Hitoshi Mitake (5):
> tgtd: add helper functions for checking iostate of iscsi connections
> tgtd: decompose iscsi_[rt]x_handler()
> tgtd: add a new option "-T" for specifying a number of threads which
> send/recv iSCSI PDUs
> tgtd: implement a deferred event modification mechanism
> tgtd: offload iSCSI PDU send/recv to worker threads
>
> usr/iscsi/iscsi_tcp.c | 405 +++++++++++++++++++++++++++++++++++++++++++++++--
> usr/iscsi/iscsid.c | 101 ++++++++----
> usr/iscsi/iscsid.h | 11 +-
> usr/tgtd.c | 10 +-
> usr/tgtd.h | 1 +
> 5 files changed, 483 insertions(+), 45 deletions(-)
>
> --
> 1.7.10.4
>
prev parent reply other threads:[~2014-02-19 3:11 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-02-14 11:21 [PATCH v4 RFC 0/5] tgtd: offload iSCSI PDU send/recv to worker threads Hitoshi Mitake
2014-02-14 11:21 ` [PATCH v4 RFC 1/5] tgtd: add helper functions for checking iostate of iscsi connections Hitoshi Mitake
2014-02-14 11:21 ` [PATCH v4 RFC 2/5] tgtd: decompose iscsi_[rt]x_handler() Hitoshi Mitake
2014-02-14 11:21 ` [PATCH v4 RFC 3/5] tgtd: add a new option "-T" for specifying a number of threads which send/recv iSCSI PDUs Hitoshi Mitake
2014-02-14 11:21 ` [PATCH v4 RFC 4/5] tgtd: implement a deferred event modification mechanism Hitoshi Mitake
2014-02-14 11:21 ` [PATCH v4 RFC 5/5] tgtd: offload iSCSI PDU send/recv to worker threads Hitoshi Mitake
2014-02-19 3:11 ` Hitoshi Mitake [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAE1WaK+gMw1pxrB5h+_wr1ZM+gmKV-vCTv3EOOUG++cymgFgXg@mail.gmail.com \
--to=mitake.hitoshi@gmail.com \
--cc=mitake.hitoshi@lab.ntt.co.jp \
--cc=stgt@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).