* [PATCH] vhost-vdpa: log warning message if vhost_vdpa_remove gets blocked
@ 2021-06-06 13:29 Gautam Dawar
2021-06-15 14:33 ` Jason Wang
2021-07-03 19:01 ` gautam.dawar
0 siblings, 2 replies; 5+ messages in thread
From: Gautam Dawar @ 2021-06-06 13:29 UTC (permalink / raw
Cc: martinh, hanand, gdawar, Michael S. Tsirkin, Jason Wang, kvm,
virtualization, netdev, linux-kernel
From: Gautam Dawar <gdawar@xilinx.com>
If some module invokes vdpa_device_unregister (usually in the module
unload function) when the userspace app (eg. QEMU) which had opened
the vhost-vdpa character device is still running, vhost_vdpa_remove()
function will block indefinitely in call to wait_for_completion().
This causes the vdpa_device_unregister caller to hang and with a
usual side-effect of rmmod command not returning when this call
is in the module_exit function.
This patch converts the wait_for_completion call to its timeout based
counterpart (wait_for_completion_timeout) and also adds a warning
message to alert the user/administrator about this hang situation.
To eventually fix this problem, a mechanism will be required to let
vhost-vdpa module inform the userspace of this situation and
userspace will close the descriptor of vhost-vdpa char device.
This will enable vhost-vdpa to continue with graceful clean-up.
Signed-off-by: Gautam Dawar <gdawar@xilinx.com>
---
drivers/vhost/vdpa.c | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c
index bfa4c6ef554e..572b64d09b06 100644
--- a/drivers/vhost/vdpa.c
+++ b/drivers/vhost/vdpa.c
@@ -1091,7 +1091,11 @@ static void vhost_vdpa_remove(struct vdpa_device *vdpa)
opened = atomic_cmpxchg(&v->opened, 0, 1);
if (!opened)
break;
- wait_for_completion(&v->completion);
+ wait_for_completion_timeout(&v->completion,
+ msecs_to_jiffies(1000));
+ dev_warn_ratelimited(&v->dev,
+ "%s waiting for /dev/%s to be closed\n",
+ __func__, dev_name(&v->dev));
} while (1);
put_device(&v->dev);
--
2.30.1
^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH] vhost-vdpa: log warning message if vhost_vdpa_remove gets blocked
2021-06-06 13:29 [PATCH] vhost-vdpa: log warning message if vhost_vdpa_remove gets blocked Gautam Dawar
@ 2021-06-15 14:33 ` Jason Wang
2021-07-03 8:11 ` Michael S. Tsirkin
2021-07-03 19:01 ` gautam.dawar
1 sibling, 1 reply; 5+ messages in thread
From: Jason Wang @ 2021-06-15 14:33 UTC (permalink / raw
To: Gautam Dawar
Cc: martinh, hanand, gdawar, Michael S. Tsirkin, kvm, virtualization,
netdev, linux-kernel
在 2021/6/6 下午9:29, Gautam Dawar 写道:
> From: Gautam Dawar <gdawar@xilinx.com>
>
> If some module invokes vdpa_device_unregister (usually in the module
> unload function) when the userspace app (eg. QEMU) which had opened
> the vhost-vdpa character device is still running, vhost_vdpa_remove()
> function will block indefinitely in call to wait_for_completion().
>
> This causes the vdpa_device_unregister caller to hang and with a
> usual side-effect of rmmod command not returning when this call
> is in the module_exit function.
>
> This patch converts the wait_for_completion call to its timeout based
> counterpart (wait_for_completion_timeout) and also adds a warning
> message to alert the user/administrator about this hang situation.
>
> To eventually fix this problem, a mechanism will be required to let
> vhost-vdpa module inform the userspace of this situation and
> userspace will close the descriptor of vhost-vdpa char device.
> This will enable vhost-vdpa to continue with graceful clean-up.
>
> Signed-off-by: Gautam Dawar <gdawar@xilinx.com>
> ---
> drivers/vhost/vdpa.c | 6 +++++-
> 1 file changed, 5 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c
> index bfa4c6ef554e..572b64d09b06 100644
> --- a/drivers/vhost/vdpa.c
> +++ b/drivers/vhost/vdpa.c
> @@ -1091,7 +1091,11 @@ static void vhost_vdpa_remove(struct vdpa_device *vdpa)
> opened = atomic_cmpxchg(&v->opened, 0, 1);
> if (!opened)
> break;
> - wait_for_completion(&v->completion);
> + wait_for_completion_timeout(&v->completion,
> + msecs_to_jiffies(1000));
> + dev_warn_ratelimited(&v->dev,
> + "%s waiting for /dev/%s to be closed\n",
> + __func__, dev_name(&v->dev));
> } while (1);
>
> put_device(&v->dev);
Acked-by: Jason Wang <jasowang@redhat.com>
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] vhost-vdpa: log warning message if vhost_vdpa_remove gets blocked
2021-06-15 14:33 ` Jason Wang
@ 2021-07-03 8:11 ` Michael S. Tsirkin
2021-07-03 19:08 ` Gautam Dawar
0 siblings, 1 reply; 5+ messages in thread
From: Michael S. Tsirkin @ 2021-07-03 8:11 UTC (permalink / raw
To: Jason Wang
Cc: Gautam Dawar, martinh, hanand, gdawar, kvm, virtualization,
netdev, linux-kernel
On Tue, Jun 15, 2021 at 10:33:22PM +0800, Jason Wang wrote:
>
> 在 2021/6/6 下午9:29, Gautam Dawar 写道:
> > From: Gautam Dawar <gdawar@xilinx.com>
> >
> > If some module invokes vdpa_device_unregister (usually in the module
> > unload function) when the userspace app (eg. QEMU) which had opened
> > the vhost-vdpa character device is still running, vhost_vdpa_remove()
> > function will block indefinitely in call to wait_for_completion().
> >
> > This causes the vdpa_device_unregister caller to hang and with a
> > usual side-effect of rmmod command not returning when this call
> > is in the module_exit function.
> >
> > This patch converts the wait_for_completion call to its timeout based
> > counterpart (wait_for_completion_timeout) and also adds a warning
> > message to alert the user/administrator about this hang situation.
> >
> > To eventually fix this problem, a mechanism will be required to let
> > vhost-vdpa module inform the userspace of this situation and
> > userspace will close the descriptor of vhost-vdpa char device.
> > This will enable vhost-vdpa to continue with graceful clean-up.
> >
> > Signed-off-by: Gautam Dawar <gdawar@xilinx.com>
> > ---
> > drivers/vhost/vdpa.c | 6 +++++-
> > 1 file changed, 5 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c
> > index bfa4c6ef554e..572b64d09b06 100644
> > --- a/drivers/vhost/vdpa.c
> > +++ b/drivers/vhost/vdpa.c
> > @@ -1091,7 +1091,11 @@ static void vhost_vdpa_remove(struct vdpa_device *vdpa)
> > opened = atomic_cmpxchg(&v->opened, 0, 1);
> > if (!opened)
> > break;
> > - wait_for_completion(&v->completion);
> > + wait_for_completion_timeout(&v->completion,
> > + msecs_to_jiffies(1000));
> > + dev_warn_ratelimited(&v->dev,
> > + "%s waiting for /dev/%s to be closed\n",
> > + __func__, dev_name(&v->dev));
Can fill up the kernel log in this case ... dev_warn_once seems more
appropriate.
> > } while (1);
> > put_device(&v->dev);
>
>
> Acked-by: Jason Wang <jasowang@redhat.com>
>
^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH] vhost-vdpa: log warning message if vhost_vdpa_remove gets blocked
2021-06-06 13:29 [PATCH] vhost-vdpa: log warning message if vhost_vdpa_remove gets blocked Gautam Dawar
2021-06-15 14:33 ` Jason Wang
@ 2021-07-03 19:01 ` gautam.dawar
1 sibling, 0 replies; 5+ messages in thread
From: gautam.dawar @ 2021-07-03 19:01 UTC (permalink / raw
Cc: martinh, hanand, gdawar, Jason Wang, Michael S. Tsirkin, kvm,
virtualization, netdev, linux-kernel
From: Gautam Dawar <gdawar@xilinx.com>
If some module invokes vdpa_device_unregister (usually in the module
unload function) when the userspace app (eg. QEMU) which had opened
the vhost-vdpa character device is still running, vhost_vdpa_remove()
function will block indefinitely in call to wait_for_completion().
This causes the vdpa_device_unregister caller to hang and with a
usual side-effect of rmmod command not returning when this call
is in the module_exit function.
This patch converts the wait_for_completion call to its timeout based
counterpart (wait_for_completion_timeout) and also adds a warning
message to alert the user/administrator about this hang situation.
To eventually fix this problem, a mechanism will be required to let
vhost-vdpa module inform the userspace of this situation and
userspace will close the descriptor of vhost-vdpa char device.
This will enable vhost-vdpa to continue with graceful clean-up.
Signed-off-by: Gautam Dawar <gdawar@xilinx.com>
Acked-by: Jason Wang <jasowang@redhat.com>
---
drivers/vhost/vdpa.c | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c
index bfa4c6ef554e..e4b7d26649d8 100644
--- a/drivers/vhost/vdpa.c
+++ b/drivers/vhost/vdpa.c
@@ -1091,7 +1091,11 @@ static void vhost_vdpa_remove(struct vdpa_device *vdpa)
opened = atomic_cmpxchg(&v->opened, 0, 1);
if (!opened)
break;
- wait_for_completion(&v->completion);
+ wait_for_completion_timeout(&v->completion,
+ msecs_to_jiffies(1000));
+ dev_warn_once(&v->dev,
+ "%s waiting for /dev/%s to be closed\n",
+ __func__, dev_name(&v->dev));
} while (1);
put_device(&v->dev);
--
2.30.1
^ permalink raw reply related [flat|nested] 5+ messages in thread
* RE: [PATCH] vhost-vdpa: log warning message if vhost_vdpa_remove gets blocked
2021-07-03 8:11 ` Michael S. Tsirkin
@ 2021-07-03 19:08 ` Gautam Dawar
0 siblings, 0 replies; 5+ messages in thread
From: Gautam Dawar @ 2021-07-03 19:08 UTC (permalink / raw
To: Michael S. Tsirkin, Jason Wang
Cc: Gautam Dawar, Martin Petrus Hubertus Habets, Harpreet Singh Anand,
kvm@vger.kernel.org, virtualization@lists.linux-foundation.org,
netdev@vger.kernel.org, linux-kernel@vger.kernel.org
Pls see inline [GD>>]
-----Original Message-----
From: Michael S. Tsirkin <mst@redhat.com>
Sent: Saturday, July 3, 2021 1:42 PM
To: Jason Wang <jasowang@redhat.com>
Cc: Gautam Dawar <gdawar.xilinx@gmail.com>; Martin Petrus Hubertus Habets <martinh@xilinx.com>; Harpreet Singh Anand <hanand@xilinx.com>; Gautam Dawar <gdawar@xilinx.com>; kvm@vger.kernel.org; virtualization@lists.linux-foundation.org; netdev@vger.kernel.org; linux-kernel@vger.kernel.org
Subject: Re: [PATCH] vhost-vdpa: log warning message if vhost_vdpa_remove gets blocked
On Tue, Jun 15, 2021 at 10:33:22PM +0800, Jason Wang wrote:
>
> 在 2021/6/6 下午9:29, Gautam Dawar 写道:
> > From: Gautam Dawar <gdawar@xilinx.com>
> >
> > If some module invokes vdpa_device_unregister (usually in the module
> > unload function) when the userspace app (eg. QEMU) which had opened
> > the vhost-vdpa character device is still running,
> > vhost_vdpa_remove() function will block indefinitely in call to wait_for_completion().
> >
> > This causes the vdpa_device_unregister caller to hang and with a
> > usual side-effect of rmmod command not returning when this call is
> > in the module_exit function.
> >
> > This patch converts the wait_for_completion call to its timeout
> > based counterpart (wait_for_completion_timeout) and also adds a
> > warning message to alert the user/administrator about this hang situation.
> >
> > To eventually fix this problem, a mechanism will be required to let
> > vhost-vdpa module inform the userspace of this situation and
> > userspace will close the descriptor of vhost-vdpa char device.
> > This will enable vhost-vdpa to continue with graceful clean-up.
> >
> > Signed-off-by: Gautam Dawar <gdawar@xilinx.com>
> > ---
> > drivers/vhost/vdpa.c | 6 +++++-
> > 1 file changed, 5 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c index
> > bfa4c6ef554e..572b64d09b06 100644
> > --- a/drivers/vhost/vdpa.c
> > +++ b/drivers/vhost/vdpa.c
> > @@ -1091,7 +1091,11 @@ static void vhost_vdpa_remove(struct vdpa_device *vdpa)
> > opened = atomic_cmpxchg(&v->opened, 0, 1);
> > if (!opened)
> > break;
> > - wait_for_completion(&v->completion);
> > + wait_for_completion_timeout(&v->completion,
> > + msecs_to_jiffies(1000));
> > + dev_warn_ratelimited(&v->dev,
> > + "%s waiting for /dev/%s to be closed\n",
> > + __func__, dev_name(&v->dev));
Can fill up the kernel log in this case ... dev_warn_once seems more appropriate.
[GD>>] Submitted the patch with suggested modification.
> > } while (1);
> > put_device(&v->dev);
>
>
> Acked-by: Jason Wang <jasowang@redhat.com>
>
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2021-07-03 19:21 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2021-06-06 13:29 [PATCH] vhost-vdpa: log warning message if vhost_vdpa_remove gets blocked Gautam Dawar
2021-06-15 14:33 ` Jason Wang
2021-07-03 8:11 ` Michael S. Tsirkin
2021-07-03 19:08 ` Gautam Dawar
2021-07-03 19:01 ` gautam.dawar
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).