From: Jason Wang <jasowang@redhat.com>
To: Heng Qi <hengqi@linux.alibaba.com>
Cc: Paolo Abeni <pabeni@redhat.com>,
"netdev@vger.kernel.org" <netdev@vger.kernel.org>,
"mst@redhat.com" <mst@redhat.com>,
"xuanzhuo@linux.alibaba.com" <xuanzhuo@linux.alibaba.com>,
"virtualization@lists.linux.dev"
<virtualization@lists.linux.dev>,
"davem@davemloft.net" <davem@davemloft.net>,
"edumazet@google.com" <edumazet@google.com>,
"kuba@kernel.org" <kuba@kernel.org>,
Jiri Pirko <jiri@nvidia.com>, Dan Jurgens <danielj@nvidia.com>
Subject: Re: [PATCH net-next v4 3/6] virtio_net: Add a lock for the command VQ.
Date: Fri, 19 Apr 2024 08:28:03 +0800 [thread overview]
Message-ID: <CACGkMEsy54akTSCEeOqFiC39097F82kARmZv3F42xf2Xmq0-pA@mail.gmail.com> (raw)
In-Reply-To: <a8ffbe97-22d1-4afe-bc6a-b4f9e7a8089a@linux.alibaba.com>
On Fri, Apr 19, 2024 at 12:12 AM Heng Qi <hengqi@linux.alibaba.com> wrote:
>
>
>
> 在 2024/4/18 下午11:48, Paolo Abeni 写道:
> > On Thu, 2024-04-18 at 15:38 +0000, Dan Jurgens wrote:
> >>> From: Paolo Abeni <pabeni@redhat.com>
> >>> Sent: Thursday, April 18, 2024 5:57 AM
> >>> On Thu, 2024-04-18 at 15:36 +0800, Heng Qi wrote:
> >>>> 在 2024/4/18 下午2:42, Jason Wang 写道:
> >>>>> On Wed, Apr 17, 2024 at 3:31 AM Daniel Jurgens <danielj@nvidia.com>
> >>> wrote:
> >>>>>> The command VQ will no longer be protected by the RTNL lock. Use a
> >>>>>> spinlock to protect the control buffer header and the VQ.
> >>>>>>
> >>>>>> Signed-off-by: Daniel Jurgens <danielj@nvidia.com>
> >>>>>> Reviewed-by: Jiri Pirko <jiri@nvidia.com>
> >>>>>> ---
> >>>>>> drivers/net/virtio_net.c | 6 +++++-
> >>>>>> 1 file changed, 5 insertions(+), 1 deletion(-)
> >>>>>>
> >>>>>> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> >>>>>> index 0ee192b45e1e..d02f83a919a7 100644
> >>>>>> --- a/drivers/net/virtio_net.c
> >>>>>> +++ b/drivers/net/virtio_net.c
> >>>>>> @@ -282,6 +282,7 @@ struct virtnet_info {
> >>>>>>
> >>>>>> /* Has control virtqueue */
> >>>>>> bool has_cvq;
> >>>>>> + spinlock_t cvq_lock;
> >>>>> Spinlock is instead of mutex which is problematic as there's no
> >>>>> guarantee on when the driver will get a reply. And it became even
> >>>>> more serious after 0d197a147164 ("virtio-net: add cond_resched() to
> >>>>> the command waiting loop").
> >>>>>
> >>>>> Any reason we can't use mutex?
> >>>> Hi Jason,
> >>>>
> >>>> I made a patch set to enable ctrlq's irq on top of this patch set,
> >>>> which removes cond_resched().
> >>>>
> >>>> But I need a little time to test, this is close to fast. So could the
> >>>> topic about cond_resched + spin lock or mutex lock be wait?
> >>> The big problem is that until the cond_resched() is there, replacing the
> >>> mutex with a spinlock can/will lead to scheduling while atomic splats. We
> >>> can't intentionally introduce such scenario.
> >> When I created the series set_rx_mode wasn't moved to a work queue,
> >> and the cond_resched wasn't there.
> > Unfortunately cond_resched() is there right now.
>
> YES.
>
> >
> >> Mutex wasn't possible, then. If the CVQ is made to be event driven, then
> >> the lock can be released right after posting the work to the VQ.
> > That should work.
>
> Yes, I will test my new patches (ctrlq with irq enabled) soon, then the
> combination
> of the this set and mine MAY make deciding between mutex or spin lock
> easier.
>
> Thanks.
So I guess the plan is to let your series come first?
Thanks
>
> >
> >>> Side note: the compiler apparently does not like guard() construct, leading to
> >>> new warning, here and in later patches. I'm unsure if the code simplification
> >>> is worthy.
> >> I didn't see any warnings with GCC or clang. This is used other places in the kernel as well.
> >> gcc version 13.2.1 20230918 (Red Hat 13.2.1-3) (GCC)
> >> clang version 17.0.6 (Fedora 17.0.6-2.fc39)
> >>
> > See:
> >
> > https://patchwork.kernel.org/project/netdevbpf/patch/20240416193039.272997-4-danielj@nvidia.com/
> > https://netdev.bots.linux.dev/static/nipa/845178/13632442/build_32bit/stderr
> > https://netdev.bots.linux.dev/static/nipa/845178/13632442/build_allmodconfig_warn/stderr
> >
> > Cheers,
> >
> > Paolo
>
next prev parent reply other threads:[~2024-04-19 0:28 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-04-16 19:30 [PATCH net-next v4 0/6] Remove RTNL lock protection of CVQ Daniel Jurgens
2024-04-16 19:30 ` [PATCH net-next v4 1/6] virtio_net: Store RSS setting in virtnet_info Daniel Jurgens
2024-04-16 19:30 ` [PATCH net-next v4 2/6] virtio_net: Remove command data from control_buf Daniel Jurgens
2024-04-16 19:30 ` [PATCH net-next v4 3/6] virtio_net: Add a lock for the command VQ Daniel Jurgens
2024-04-18 6:42 ` Jason Wang
2024-04-18 7:36 ` Heng Qi
2024-04-18 10:56 ` Paolo Abeni
2024-04-18 15:38 ` Dan Jurgens
2024-04-18 15:48 ` Paolo Abeni
2024-04-18 16:01 ` Jakub Kicinski
2024-04-18 16:12 ` Heng Qi
2024-04-19 0:28 ` Jason Wang [this message]
2024-04-18 16:06 ` Heng Qi
2024-04-16 19:30 ` [PATCH net-next v4 4/6] virtio_net: Do DIM update for specified queue only Daniel Jurgens
2024-04-16 19:30 ` [PATCH net-next v4 5/6] virtio_net: Add a lock for per queue RX coalesce Daniel Jurgens
2024-04-16 19:30 ` [PATCH net-next v4 6/6] virtio_net: Remove rtnl lock protection of command buffers Daniel Jurgens
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CACGkMEsy54akTSCEeOqFiC39097F82kARmZv3F42xf2Xmq0-pA@mail.gmail.com \
--to=jasowang@redhat.com \
--cc=danielj@nvidia.com \
--cc=davem@davemloft.net \
--cc=edumazet@google.com \
--cc=hengqi@linux.alibaba.com \
--cc=jiri@nvidia.com \
--cc=kuba@kernel.org \
--cc=mst@redhat.com \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=virtualization@lists.linux.dev \
--cc=xuanzhuo@linux.alibaba.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).