From: Zhu Yanjun <yanjun.zhu@linux.dev>
To: Chuck Lever III <chuck.lever@oracle.com>
Cc: Chuck Lever <cel@kernel.org>,
Linux NFS Mailing List <linux-nfs@vger.kernel.org>,
"linux-rdma@vger.kernel.org" <linux-rdma@vger.kernel.org>
Subject: Re: [RFC PATCH 0/4] NFS: Fix another 'check_flush_dependency' splat
Date: Tue, 30 Apr 2024 16:57:47 +0200 [thread overview]
Message-ID: <873cfaf7-fc0f-4809-8439-7a67c1c4f5a0@linux.dev> (raw)
In-Reply-To: <3B48378E-CE68-47FC-A8D3-7A7AEE383B25@oracle.com>
On 30.04.24 16:52, Chuck Lever III wrote:
>
>> On Apr 30, 2024, at 10:45 AM, Zhu Yanjun <zyjzyj2000@gmail.com> wrote:
>>
>> On 30.04.24 16:13, Chuck Lever III wrote:
>>>> On Apr 30, 2024, at 9:58 AM, Zhu Yanjun <yanjun.zhu@linux.dev> wrote:
>>>>
>>>>
>>>> On 30.04.24 15:42, Chuck Lever III wrote:
>>>>>> On Apr 30, 2024, at 3:26 AM, Zhu Yanjun <zyjzyj2000@gmail.com> wrote:
>>>>>>
>>>>>> On 29.04.24 17:25, cel@kernel.org wrote:
>>>>>>> From: Chuck Lever <chuck.lever@oracle.com>
>>>>>>> Avoid getting work queue splats in the system journal by moving
>>>>>>> client-side RPC/RDMA transport tear-down into a background process.
>>>>>>> I've done some testing of this series, now looking for review
>>>>>>> comments.
>>>>>> How to make tests with nfs && rdma? Can you provide some steps or tools?
>>>>> We are building NFS tests into kdevops:
>>>>>
>>>>> https://github.com/linux-kdevops/kdevops.git
>>>>>
>>>>> and there is a config option to use soft iWARP instead of TCP.
>>>> Thanks a lot. It is interesting. Have you made tests with RXE instead of iWARP?
>>>>
>>>> If yes, does nfs work well with RXE? I am just curious with nfs && RXE.
>>>>
>>>> Normally nfs works with TCP. Now nfs will use RDMA instead of TCP.
>>>>
>>>> The popular RDMA implementation is RoCEv2 which is based on UDP protocol.
>>>>
>>>> So I am curious if NFS can work well with RXE (RoCEv2 emulation driver) or not.
>>>>
>>>> If the user wants to use nfs in his production hosts, it is possible that nfs will work with RoCEv2 (UDP).
>>> Yes, NFS/RDMA works with rxe and even with rxe mixed with
>>> hardware RoCE. Someone else will have to step in and say
>>> whether it works "well" since I don't use rxe, only CX-5
>>> and newer on 100GbE.
>>> Generally we use siw because our testing environment varies
>>> between all systems on a single local network or hypervisor,
>>> all the way up to NFS/RDMA on VPN and WAN. The rxe driver
>>> doesn't support operation over tunnels, currently.
>> Thanks a lot. "The rxe driver doesn't support operation over tunnels, currently." Do you mean that rxe can not work well with tun/tap device?
> No, rxe cannot be configured to use tunnel devices, AFAIK.
>
>
>>> It is possible to add rxe as a second option in kdevops,
>>> but siw has worked for our purposes so far, and the NFS
>>> test matrix is already enormous.
>> Thanks. If rxe can be as a second option in kdevops, I will make tests with kdevops to check rxe work well or not in the future kernel version.
> No new tests are necessary. The only thing missing right
> now is the ability to set up rxe devices on all the test
> systems.
Got it. Thanks.
Zhu Yanjun
>
>
>> Best Regards,
>> Zhu Yanjun
>>
>>>> Best Regards,
>>>>
>>>> Zhu Yanjun
>>>>
>>>>> kdevops includes workflows for fstests, Mora's nfstest, the
>>>>> git regression suite, and ltp, all of which we use regularly
>>>>> to test the Linux NFS client and server implementations.
>>>>>
>>>>>
>>>>>> I am interested in nfs && rdma.
>>>>>>
>>>>>> Thanks,
>>>>>> Zhu Yanjun
>>>>>>
>>>>>>> Chuck Lever (4):
>>>>>>> xprtrdma: Remove temp allocation of rpcrdma_rep objects
>>>>>>> xprtrdma: Clean up synopsis of frwr_mr_unmap()
>>>>>>> xprtrdma: Delay releasing connection hardware resources
>>>>>>> xprtrdma: Move MRs to struct rpcrdma_ep
>>>>>>> net/sunrpc/xprtrdma/frwr_ops.c | 13 ++-
>>>>>>> net/sunrpc/xprtrdma/rpc_rdma.c | 3 +-
>>>>>>> net/sunrpc/xprtrdma/transport.c | 20 +++-
>>>>>>> net/sunrpc/xprtrdma/verbs.c | 173 ++++++++++++++++----------------
>>>>>>> net/sunrpc/xprtrdma/xprt_rdma.h | 21 ++--
>>>>>>> 5 files changed, 125 insertions(+), 105 deletions(-)
>>>>>>> base-commit: e67572cd2204894179d89bd7b984072f19313b03
>>>>> --
>>>>> Chuck Lever
>>>>>
>>>>>
>>>> --
>>>> Best Regards,
>>>> Yanjun.Zhu
>>> --
>>> Chuck Lever
>
> --
> Chuck Lever
>
>
--
Best Regards,
Yanjun.Zhu
next prev parent reply other threads:[~2024-04-30 14:57 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-04-29 15:25 [RFC PATCH 0/4] NFS: Fix another 'check_flush_dependency' splat cel
2024-04-29 15:25 ` [RFC PATCH 1/4] xprtrdma: Remove temp allocation of rpcrdma_rep objects cel
2024-04-29 15:25 ` [RFC PATCH 2/4] xprtrdma: Clean up synopsis of frwr_mr_unmap() cel
2024-04-29 15:25 ` [RFC PATCH 3/4] xprtrdma: Delay releasing connection hardware resources cel
2024-04-29 15:25 ` [RFC PATCH 4/4] xprtrdma: Move MRs to struct rpcrdma_ep cel
2024-04-30 7:26 ` [RFC PATCH 0/4] NFS: Fix another 'check_flush_dependency' splat Zhu Yanjun
2024-04-30 13:42 ` Chuck Lever III
2024-04-30 13:58 ` Zhu Yanjun
2024-04-30 14:13 ` Chuck Lever III
2024-04-30 14:45 ` Zhu Yanjun
2024-04-30 14:52 ` Chuck Lever III
2024-04-30 14:57 ` Zhu Yanjun [this message]
2024-06-02 15:40 ` Chuck Lever III
2024-06-02 18:14 ` Zhu Yanjun
2024-06-03 15:59 ` Chuck Lever III
2024-06-03 16:54 ` Zhu Yanjun
2024-06-03 17:06 ` Chuck Lever III
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=873cfaf7-fc0f-4809-8439-7a67c1c4f5a0@linux.dev \
--to=yanjun.zhu@linux.dev \
--cc=cel@kernel.org \
--cc=chuck.lever@oracle.com \
--cc=linux-nfs@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).