* [PATCH v2 0/4] dax/bus.c: Fixups for dax-bus locking
@ 2024-04-16 21:46 Vishal Verma
2024-04-16 21:46 ` [PATCH v2 1/4] dax/bus.c: replace WARN_ON_ONCE() with lockdep asserts Vishal Verma
` (3 more replies)
0 siblings, 4 replies; 11+ messages in thread
From: Vishal Verma @ 2024-04-16 21:46 UTC (permalink / raw
To: Dan Williams, Dave Jiang, Alison Schofield, Andrew Morton
Cc: linux-mm, nvdimm, linux-cxl, linux-kernel, Vishal Verma
Commit Fixes: c05ae9d85b47 ("dax/bus.c: replace driver-core lock usage by a local rwsem")
introduced a few problems that this series aims to fix. Add back
device_lock() where it was correctly used (during device manipulation
operations), remove conditional locking in unregister_dax_dev() and
unregister_dax_mapping(), use non-interruptible versions of rwsem
locks when not called from a user process, and fix up a write vs.
read usage of an rwsem.
Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
---
Changes in v2:
- Add back valid device_lock uses (Dan)
- Remove conditional locking (Dan)
- Use non-interruptible versions of rwsem locks when not called from a
user process (Dan)
- Fix up a write vs. read usage of an rwsem
- Link to v1: https://lore.kernel.org/r/20240402-vv-dax_abi_fixes-v1-1-c3e0fdbafba5@intel.com
---
Vishal Verma (4):
dax/bus.c: replace WARN_ON_ONCE() with lockdep asserts
dax/bus.c: fix locking for unregister_dax_dev / unregister_dax_mapping paths
dax/bus.c: Don't use down_write_killable for non-user processes
dax/bus.c: Use the right locking mode (read vs write) in size_show
drivers/dax/bus.c | 68 ++++++++++++++++---------------------------------------
1 file changed, 20 insertions(+), 48 deletions(-)
---
base-commit: 39cd87c4eb2b893354f3b850f916353f2658ae6f
change-id: 20240402-vv-dax_abi_fixes-8af3b6ff2e5a
Best regards,
--
Vishal Verma <vishal.l.verma@intel.com>
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v2 1/4] dax/bus.c: replace WARN_ON_ONCE() with lockdep asserts
2024-04-16 21:46 [PATCH v2 0/4] dax/bus.c: Fixups for dax-bus locking Vishal Verma
@ 2024-04-16 21:46 ` Vishal Verma
2024-04-30 1:23 ` Dan Williams
2024-04-16 21:46 ` [PATCH v2 2/4] dax/bus.c: fix locking for unregister_dax_dev / unregister_dax_mapping paths Vishal Verma
` (2 subsequent siblings)
3 siblings, 1 reply; 11+ messages in thread
From: Vishal Verma @ 2024-04-16 21:46 UTC (permalink / raw
To: Dan Williams, Dave Jiang, Alison Schofield, Andrew Morton
Cc: linux-mm, nvdimm, linux-cxl, linux-kernel, Vishal Verma
In [1], Dan points out that all of the WARN_ON_ONCE() usage in the
referenced patch should be replaced with lockdep_assert_held, or
lockdep_held_assert_write(). Replace these as appropriate.
Link: https://lore.kernel.org/r/65f0b5ef41817_aa222941a@dwillia2-mobl3.amr.corp.intel.com.notmuch [1]
Fixes: c05ae9d85b47 ("dax/bus.c: replace driver-core lock usage by a local rwsem")
Cc: Andrew Morton <akpm@linux-foundation.org>
Reported-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
---
drivers/dax/bus.c | 16 ++++++++--------
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/drivers/dax/bus.c b/drivers/dax/bus.c
index 797e1ebff299..7924dd542a13 100644
--- a/drivers/dax/bus.c
+++ b/drivers/dax/bus.c
@@ -192,7 +192,7 @@ static u64 dev_dax_size(struct dev_dax *dev_dax)
u64 size = 0;
int i;
- WARN_ON_ONCE(!rwsem_is_locked(&dax_dev_rwsem));
+ lockdep_assert_held(&dax_dev_rwsem);
for (i = 0; i < dev_dax->nr_range; i++)
size += range_len(&dev_dax->ranges[i].range);
@@ -302,7 +302,7 @@ static unsigned long long dax_region_avail_size(struct dax_region *dax_region)
resource_size_t size = resource_size(&dax_region->res);
struct resource *res;
- WARN_ON_ONCE(!rwsem_is_locked(&dax_region_rwsem));
+ lockdep_assert_held(&dax_region_rwsem);
for_each_dax_region_resource(dax_region, res)
size -= resource_size(res);
@@ -447,7 +447,7 @@ static void trim_dev_dax_range(struct dev_dax *dev_dax)
struct range *range = &dev_dax->ranges[i].range;
struct dax_region *dax_region = dev_dax->region;
- WARN_ON_ONCE(!rwsem_is_locked(&dax_region_rwsem));
+ lockdep_assert_held_write(&dax_region_rwsem);
dev_dbg(&dev_dax->dev, "delete range[%d]: %#llx:%#llx\n", i,
(unsigned long long)range->start,
(unsigned long long)range->end);
@@ -507,7 +507,7 @@ static int __free_dev_dax_id(struct dev_dax *dev_dax)
struct dax_region *dax_region;
int rc = dev_dax->id;
- WARN_ON_ONCE(!rwsem_is_locked(&dax_dev_rwsem));
+ lockdep_assert_held_write(&dax_dev_rwsem);
if (!dev_dax->dyn_id || dev_dax->id < 0)
return -1;
@@ -713,7 +713,7 @@ static void __unregister_dax_mapping(void *data)
dev_dbg(dev, "%s\n", __func__);
- WARN_ON_ONCE(!rwsem_is_locked(&dax_region_rwsem));
+ lockdep_assert_held_write(&dax_region_rwsem);
dev_dax->ranges[mapping->range_id].mapping = NULL;
mapping->range_id = -1;
@@ -830,7 +830,7 @@ static int devm_register_dax_mapping(struct dev_dax *dev_dax, int range_id)
struct device *dev;
int rc;
- WARN_ON_ONCE(!rwsem_is_locked(&dax_region_rwsem));
+ lockdep_assert_held_write(&dax_region_rwsem);
if (dev_WARN_ONCE(&dev_dax->dev, !dax_region->dev->driver,
"region disabled\n"))
@@ -876,7 +876,7 @@ static int alloc_dev_dax_range(struct dev_dax *dev_dax, u64 start,
struct resource *alloc;
int i, rc;
- WARN_ON_ONCE(!rwsem_is_locked(&dax_region_rwsem));
+ lockdep_assert_held_write(&dax_region_rwsem);
/* handle the seed alloc special case */
if (!size) {
@@ -935,7 +935,7 @@ static int adjust_dev_dax_range(struct dev_dax *dev_dax, struct resource *res, r
struct device *dev = &dev_dax->dev;
int rc;
- WARN_ON_ONCE(!rwsem_is_locked(&dax_region_rwsem));
+ lockdep_assert_held_write(&dax_region_rwsem);
if (dev_WARN_ONCE(dev, !size, "deletion is handled by dev_dax_shrink\n"))
return -EINVAL;
--
2.44.0
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v2 2/4] dax/bus.c: fix locking for unregister_dax_dev / unregister_dax_mapping paths
2024-04-16 21:46 [PATCH v2 0/4] dax/bus.c: Fixups for dax-bus locking Vishal Verma
2024-04-16 21:46 ` [PATCH v2 1/4] dax/bus.c: replace WARN_ON_ONCE() with lockdep asserts Vishal Verma
@ 2024-04-16 21:46 ` Vishal Verma
2024-04-30 1:25 ` Dan Williams
2024-04-16 21:46 ` [PATCH v2 3/4] dax/bus.c: Don't use down_write_killable for non-user processes Vishal Verma
2024-04-16 21:46 ` [PATCH v2 4/4] dax/bus.c: Use the right locking mode (read vs write) in size_show Vishal Verma
3 siblings, 1 reply; 11+ messages in thread
From: Vishal Verma @ 2024-04-16 21:46 UTC (permalink / raw
To: Dan Williams, Dave Jiang, Alison Schofield, Andrew Morton
Cc: linux-mm, nvdimm, linux-cxl, linux-kernel, Vishal Verma
Commit c05ae9d85b47 ("dax/bus.c: replace driver-core lock usage by a local rwsem")
was a bit overzealous in eliminating device_lock() usage, and ended up
removing a couple of lock acquisitions which were needed, and as a
result, fix some of the conditional locking missteps that the above
commit introduced in unregister_dax_dev() and unregister_dax_mapping().
Fixes: c05ae9d85b47 ("dax/bus.c: replace driver-core lock usage by a local rwsem")
Reported-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
---
drivers/dax/bus.c | 44 ++++++++++----------------------------------
1 file changed, 10 insertions(+), 34 deletions(-)
diff --git a/drivers/dax/bus.c b/drivers/dax/bus.c
index 7924dd542a13..4e04b228b080 100644
--- a/drivers/dax/bus.c
+++ b/drivers/dax/bus.c
@@ -465,26 +465,17 @@ static void free_dev_dax_ranges(struct dev_dax *dev_dax)
trim_dev_dax_range(dev_dax);
}
-static void __unregister_dev_dax(void *dev)
+static void unregister_dev_dax(void *dev)
{
struct dev_dax *dev_dax = to_dev_dax(dev);
dev_dbg(dev, "%s\n", __func__);
+ down_write(&dax_region_rwsem);
kill_dev_dax(dev_dax);
device_del(dev);
free_dev_dax_ranges(dev_dax);
put_device(dev);
-}
-
-static void unregister_dev_dax(void *dev)
-{
- if (rwsem_is_locked(&dax_region_rwsem))
- return __unregister_dev_dax(dev);
-
- if (WARN_ON_ONCE(down_write_killable(&dax_region_rwsem) != 0))
- return;
- __unregister_dev_dax(dev);
up_write(&dax_region_rwsem);
}
@@ -560,15 +551,12 @@ static ssize_t delete_store(struct device *dev, struct device_attribute *attr,
if (!victim)
return -ENXIO;
- rc = down_write_killable(&dax_region_rwsem);
- if (rc)
- return rc;
- rc = down_write_killable(&dax_dev_rwsem);
- if (rc) {
- up_write(&dax_region_rwsem);
- return rc;
- }
+ device_lock(dev);
+ device_lock(victim);
dev_dax = to_dev_dax(victim);
+ rc = down_write_killable(&dax_dev_rwsem);
+ if (rc)
+ return rc;
if (victim->driver || dev_dax_size(dev_dax))
rc = -EBUSY;
else {
@@ -589,11 +577,12 @@ static ssize_t delete_store(struct device *dev, struct device_attribute *attr,
rc = -EBUSY;
}
up_write(&dax_dev_rwsem);
+ device_unlock(victim);
/* won the race to invalidate the device, clean it up */
if (do_del)
devm_release_action(dev, unregister_dev_dax, victim);
- up_write(&dax_region_rwsem);
+ device_unlock(dev);
put_device(victim);
return rc;
@@ -705,7 +694,7 @@ static void dax_mapping_release(struct device *dev)
put_device(parent);
}
-static void __unregister_dax_mapping(void *data)
+static void unregister_dax_mapping(void *data)
{
struct device *dev = data;
struct dax_mapping *mapping = to_dax_mapping(dev);
@@ -713,25 +702,12 @@ static void __unregister_dax_mapping(void *data)
dev_dbg(dev, "%s\n", __func__);
- lockdep_assert_held_write(&dax_region_rwsem);
-
dev_dax->ranges[mapping->range_id].mapping = NULL;
mapping->range_id = -1;
device_unregister(dev);
}
-static void unregister_dax_mapping(void *data)
-{
- if (rwsem_is_locked(&dax_region_rwsem))
- return __unregister_dax_mapping(data);
-
- if (WARN_ON_ONCE(down_write_killable(&dax_region_rwsem) != 0))
- return;
- __unregister_dax_mapping(data);
- up_write(&dax_region_rwsem);
-}
-
static struct dev_dax_range *get_dax_range(struct device *dev)
{
struct dax_mapping *mapping = to_dax_mapping(dev);
--
2.44.0
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v2 3/4] dax/bus.c: Don't use down_write_killable for non-user processes
2024-04-16 21:46 [PATCH v2 0/4] dax/bus.c: Fixups for dax-bus locking Vishal Verma
2024-04-16 21:46 ` [PATCH v2 1/4] dax/bus.c: replace WARN_ON_ONCE() with lockdep asserts Vishal Verma
2024-04-16 21:46 ` [PATCH v2 2/4] dax/bus.c: fix locking for unregister_dax_dev / unregister_dax_mapping paths Vishal Verma
@ 2024-04-16 21:46 ` Vishal Verma
2024-04-30 1:25 ` Dan Williams
2024-04-16 21:46 ` [PATCH v2 4/4] dax/bus.c: Use the right locking mode (read vs write) in size_show Vishal Verma
3 siblings, 1 reply; 11+ messages in thread
From: Vishal Verma @ 2024-04-16 21:46 UTC (permalink / raw
To: Dan Williams, Dave Jiang, Alison Schofield, Andrew Morton
Cc: linux-mm, nvdimm, linux-cxl, linux-kernel, Vishal Verma
Change an instance of down_write_killable() to a simple down_write() where
there is no user process that might want to interrupt the operation.
Fixes: c05ae9d85b47 ("dax/bus.c: replace driver-core lock usage by a local rwsem")
Reported-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
---
drivers/dax/bus.c | 6 +-----
1 file changed, 1 insertion(+), 5 deletions(-)
diff --git a/drivers/dax/bus.c b/drivers/dax/bus.c
index 4e04b228b080..db183eb5ce3a 100644
--- a/drivers/dax/bus.c
+++ b/drivers/dax/bus.c
@@ -1542,12 +1542,8 @@ static struct dev_dax *__devm_create_dev_dax(struct dev_dax_data *data)
struct dev_dax *devm_create_dev_dax(struct dev_dax_data *data)
{
struct dev_dax *dev_dax;
- int rc;
-
- rc = down_write_killable(&dax_region_rwsem);
- if (rc)
- return ERR_PTR(rc);
+ down_write(&dax_region_rwsem);
dev_dax = __devm_create_dev_dax(data);
up_write(&dax_region_rwsem);
--
2.44.0
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v2 4/4] dax/bus.c: Use the right locking mode (read vs write) in size_show
2024-04-16 21:46 [PATCH v2 0/4] dax/bus.c: Fixups for dax-bus locking Vishal Verma
` (2 preceding siblings ...)
2024-04-16 21:46 ` [PATCH v2 3/4] dax/bus.c: Don't use down_write_killable for non-user processes Vishal Verma
@ 2024-04-16 21:46 ` Vishal Verma
2024-04-30 1:26 ` Dan Williams
3 siblings, 1 reply; 11+ messages in thread
From: Vishal Verma @ 2024-04-16 21:46 UTC (permalink / raw
To: Dan Williams, Dave Jiang, Alison Schofield, Andrew Morton
Cc: linux-mm, nvdimm, linux-cxl, linux-kernel, Vishal Verma
In size_show(), the dax_dev_rwsem only needs a read lock, but was
acquiring a write lock. Change it to down_read_interruptible() so it
doesn't unnecessarily hold a write lock.
Cc: Dan Williams <dan.j.williams@intel.com>
Fixes: c05ae9d85b47 ("dax/bus.c: replace driver-core lock usage by a local rwsem")
Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
---
drivers/dax/bus.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/dax/bus.c b/drivers/dax/bus.c
index db183eb5ce3a..66095e60a279 100644
--- a/drivers/dax/bus.c
+++ b/drivers/dax/bus.c
@@ -939,11 +939,11 @@ static ssize_t size_show(struct device *dev,
unsigned long long size;
int rc;
- rc = down_write_killable(&dax_dev_rwsem);
+ rc = down_read_interruptible(&dax_dev_rwsem);
if (rc)
return rc;
size = dev_dax_size(dev_dax);
- up_write(&dax_dev_rwsem);
+ up_read(&dax_dev_rwsem);
return sysfs_emit(buf, "%llu\n", size);
}
--
2.44.0
^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH v2 1/4] dax/bus.c: replace WARN_ON_ONCE() with lockdep asserts
2024-04-16 21:46 ` [PATCH v2 1/4] dax/bus.c: replace WARN_ON_ONCE() with lockdep asserts Vishal Verma
@ 2024-04-30 1:23 ` Dan Williams
0 siblings, 0 replies; 11+ messages in thread
From: Dan Williams @ 2024-04-30 1:23 UTC (permalink / raw
To: Vishal Verma, Dan Williams, Dave Jiang, Alison Schofield,
Andrew Morton
Cc: linux-mm, nvdimm, linux-cxl, linux-kernel, Vishal Verma
Vishal Verma wrote:
> In [1], Dan points out that all of the WARN_ON_ONCE() usage in the
> referenced patch should be replaced with lockdep_assert_held, or
> lockdep_held_assert_write(). Replace these as appropriate.
>
> Link: https://lore.kernel.org/r/65f0b5ef41817_aa222941a@dwillia2-mobl3.amr.corp.intel.com.notmuch [1]
> Fixes: c05ae9d85b47 ("dax/bus.c: replace driver-core lock usage by a local rwsem")
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Reported-by: Dan Williams <dan.j.williams@intel.com>
> Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
> ---
> drivers/dax/bus.c | 16 ++++++++--------
> 1 file changed, 8 insertions(+), 8 deletions(-)
Looks good,
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2 2/4] dax/bus.c: fix locking for unregister_dax_dev / unregister_dax_mapping paths
2024-04-16 21:46 ` [PATCH v2 2/4] dax/bus.c: fix locking for unregister_dax_dev / unregister_dax_mapping paths Vishal Verma
@ 2024-04-30 1:25 ` Dan Williams
2024-04-30 4:11 ` Verma, Vishal L
0 siblings, 1 reply; 11+ messages in thread
From: Dan Williams @ 2024-04-30 1:25 UTC (permalink / raw
To: Vishal Verma, Dan Williams, Dave Jiang, Alison Schofield,
Andrew Morton
Cc: linux-mm, nvdimm, linux-cxl, linux-kernel, Vishal Verma
Vishal Verma wrote:
> Commit c05ae9d85b47 ("dax/bus.c: replace driver-core lock usage by a local rwsem")
> was a bit overzealous in eliminating device_lock() usage, and ended up
> removing a couple of lock acquisitions which were needed, and as a
> result, fix some of the conditional locking missteps that the above
> commit introduced in unregister_dax_dev() and unregister_dax_mapping().
I think it makes sense to tell the story a bit about why the
delete_store() conversion was problematic, because the
unregister_dev_dax() changes were just a knock-on effect to fixing the
delete_store() flow.
Something like:
---
commit c05ae9d85b47 ("dax/bus.c: replace driver-core lock usage by a local rwsem")
aimed to undo device_lock() abuses for protecting changes to dax-driver
internal data-structures like the dax_region resource tree to
device-dax-instance range structures. However, the device_lock() was legitamately
enforcing that devices to be deleted were not current actively attached
to any driver nor assigned any capacity from the region.
---
...you can fill in a couple notes about the knock-on fixups after that
was restored.
> Fixes: c05ae9d85b47 ("dax/bus.c: replace driver-core lock usage by a local rwsem")
> Reported-by: Dan Williams <dan.j.williams@intel.com>
> Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
> ---
> drivers/dax/bus.c | 44 ++++++++++----------------------------------
> 1 file changed, 10 insertions(+), 34 deletions(-)
>
> diff --git a/drivers/dax/bus.c b/drivers/dax/bus.c
> index 7924dd542a13..4e04b228b080 100644
> --- a/drivers/dax/bus.c
> +++ b/drivers/dax/bus.c
> @@ -465,26 +465,17 @@ static void free_dev_dax_ranges(struct dev_dax *dev_dax)
> trim_dev_dax_range(dev_dax);
> }
>
> -static void __unregister_dev_dax(void *dev)
> +static void unregister_dev_dax(void *dev)
> {
> struct dev_dax *dev_dax = to_dev_dax(dev);
>
> dev_dbg(dev, "%s\n", __func__);
>
> + down_write(&dax_region_rwsem);
> kill_dev_dax(dev_dax);
> device_del(dev);
> free_dev_dax_ranges(dev_dax);
> put_device(dev);
> -}
> -
> -static void unregister_dev_dax(void *dev)
> -{
> - if (rwsem_is_locked(&dax_region_rwsem))
> - return __unregister_dev_dax(dev);
> -
> - if (WARN_ON_ONCE(down_write_killable(&dax_region_rwsem) != 0))
> - return;
> - __unregister_dev_dax(dev);
> up_write(&dax_region_rwsem);
> }
>
> @@ -560,15 +551,12 @@ static ssize_t delete_store(struct device *dev, struct device_attribute *attr,
> if (!victim)
> return -ENXIO;
>
> - rc = down_write_killable(&dax_region_rwsem);
> - if (rc)
> - return rc;
> - rc = down_write_killable(&dax_dev_rwsem);
> - if (rc) {
> - up_write(&dax_region_rwsem);
> - return rc;
> - }
> + device_lock(dev);
> + device_lock(victim);
> dev_dax = to_dev_dax(victim);
> + rc = down_write_killable(&dax_dev_rwsem);
This begs the question, why down_write_killable(), but not
device_lock_interruptible()?
I do not expect any of this is long running so likely down_write() is
sufficient here, especially since the heaviest locks to acquire are
already held by the time rwsem is considered.
Other than that this looks good to me:
You can include my Reviewed-by on the next posting.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2 3/4] dax/bus.c: Don't use down_write_killable for non-user processes
2024-04-16 21:46 ` [PATCH v2 3/4] dax/bus.c: Don't use down_write_killable for non-user processes Vishal Verma
@ 2024-04-30 1:25 ` Dan Williams
0 siblings, 0 replies; 11+ messages in thread
From: Dan Williams @ 2024-04-30 1:25 UTC (permalink / raw
To: Vishal Verma, Dan Williams, Dave Jiang, Alison Schofield,
Andrew Morton
Cc: linux-mm, nvdimm, linux-cxl, linux-kernel, Vishal Verma
Vishal Verma wrote:
> Change an instance of down_write_killable() to a simple down_write() where
> there is no user process that might want to interrupt the operation.
>
> Fixes: c05ae9d85b47 ("dax/bus.c: replace driver-core lock usage by a local rwsem")
> Reported-by: Dan Williams <dan.j.williams@intel.com>
> Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
> ---
> drivers/dax/bus.c | 6 +-----
> 1 file changed, 1 insertion(+), 5 deletions(-)
Looks good,
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2 4/4] dax/bus.c: Use the right locking mode (read vs write) in size_show
2024-04-16 21:46 ` [PATCH v2 4/4] dax/bus.c: Use the right locking mode (read vs write) in size_show Vishal Verma
@ 2024-04-30 1:26 ` Dan Williams
0 siblings, 0 replies; 11+ messages in thread
From: Dan Williams @ 2024-04-30 1:26 UTC (permalink / raw
To: Vishal Verma, Dan Williams, Dave Jiang, Alison Schofield,
Andrew Morton
Cc: linux-mm, nvdimm, linux-cxl, linux-kernel, Vishal Verma
Vishal Verma wrote:
> In size_show(), the dax_dev_rwsem only needs a read lock, but was
> acquiring a write lock. Change it to down_read_interruptible() so it
> doesn't unnecessarily hold a write lock.
>
> Cc: Dan Williams <dan.j.williams@intel.com>
> Fixes: c05ae9d85b47 ("dax/bus.c: replace driver-core lock usage by a local rwsem")
> Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
> ---
> drivers/dax/bus.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
Looks good,
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2 2/4] dax/bus.c: fix locking for unregister_dax_dev / unregister_dax_mapping paths
2024-04-30 1:25 ` Dan Williams
@ 2024-04-30 4:11 ` Verma, Vishal L
2024-04-30 5:25 ` Dan Williams
0 siblings, 1 reply; 11+ messages in thread
From: Verma, Vishal L @ 2024-04-30 4:11 UTC (permalink / raw
To: Williams, Dan J, Jiang, Dave, Schofield, Alison,
akpm@linux-foundation.org
Cc: linux-mm@kvack.org, linux-cxl@vger.kernel.org,
nvdimm@lists.linux.dev, linux-kernel@vger.kernel.org
On Mon, 2024-04-29 at 18:25 -0700, Dan Williams wrote:
> Vishal Verma wrote:
> > Commit c05ae9d85b47 ("dax/bus.c: replace driver-core lock usage by a local rwsem")
> > was a bit overzealous in eliminating device_lock() usage, and ended up
> > removing a couple of lock acquisitions which were needed, and as a
> > result, fix some of the conditional locking missteps that the above
> > commit introduced in unregister_dax_dev() and unregister_dax_mapping().
>
> I think it makes sense to tell the story a bit about why the
> delete_store() conversion was problematic, because the
> unregister_dev_dax() changes were just a knock-on effect to fixing the
> delete_store() flow.
>
> Something like:
>
> ---
> commit c05ae9d85b47 ("dax/bus.c: replace driver-core lock usage by a local rwsem")
> aimed to undo device_lock() abuses for protecting changes to dax-driver
> internal data-structures like the dax_region resource tree to
> device-dax-instance range structures. However, the device_lock() was legitamately
> enforcing that devices to be deleted were not current actively attached
> to any driver nor assigned any capacity from the region.
> ---
>
> ...you can fill in a couple notes about the knock-on fixups after that
> was restored.
Sounds good, updated!
>
> >
> > @@ -560,15 +551,12 @@ static ssize_t delete_store(struct device *dev, struct device_attribute *attr,
> > if (!victim)
> > return -ENXIO;
> >
> > - rc = down_write_killable(&dax_region_rwsem);
> > - if (rc)
> > - return rc;
> > - rc = down_write_killable(&dax_dev_rwsem);
> > - if (rc) {
> > - up_write(&dax_region_rwsem);
> > - return rc;
> > - }
> > + device_lock(dev);
> > + device_lock(victim);
> > dev_dax = to_dev_dax(victim);
> > + rc = down_write_killable(&dax_dev_rwsem);
>
> This begs the question, why down_write_killable(), but not
> device_lock_interruptible()?
Do you mean change the device_lock()s to device_lock_interruptible() in
addition to the taking the rwsem (i.e. not instead of the rwsem..)?
I guess I just restored what was there previously - but the
interruptible variant makes sense, I can make that change.
>
> I do not expect any of this is long running so likely down_write() is
> sufficient here, especially since the heaviest locks to acquire are
> already held by the time rwsem is considered.
>
> Other than that this looks good to me:
>
> You can include my Reviewed-by on the next posting.
Thanks for the review Dan!
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2 2/4] dax/bus.c: fix locking for unregister_dax_dev / unregister_dax_mapping paths
2024-04-30 4:11 ` Verma, Vishal L
@ 2024-04-30 5:25 ` Dan Williams
0 siblings, 0 replies; 11+ messages in thread
From: Dan Williams @ 2024-04-30 5:25 UTC (permalink / raw
To: Verma, Vishal L, Williams, Dan J, Jiang, Dave, Schofield, Alison,
akpm@linux-foundation.org
Cc: linux-mm@kvack.org, linux-cxl@vger.kernel.org,
nvdimm@lists.linux.dev, linux-kernel@vger.kernel.org
Verma, Vishal L wrote:
> > > @@ -560,15 +551,12 @@ static ssize_t delete_store(struct device *dev, struct device_attribute *attr,
> > > if (!victim)
> > > return -ENXIO;
> > >
> > > - rc = down_write_killable(&dax_region_rwsem);
> > > - if (rc)
> > > - return rc;
> > > - rc = down_write_killable(&dax_dev_rwsem);
> > > - if (rc) {
> > > - up_write(&dax_region_rwsem);
> > > - return rc;
> > > - }
> > > + device_lock(dev);
> > > + device_lock(victim);
> > > dev_dax = to_dev_dax(victim);
> > > + rc = down_write_killable(&dax_dev_rwsem);
> >
> > This begs the question, why down_write_killable(), but not
> > device_lock_interruptible()?
>
> Do you mean change the device_lock()s to device_lock_interruptible() in
> addition to the taking the rwsem (i.e. not instead of the rwsem..)?
I mean convert the rwsem to drop _killable.
> I guess I just restored what was there previously - but the
> interruptible variant makes sense, I can make that change.
So the original code did device_lock(), then the rework added killable
rwsem (deleted device_lock()), and now the fixes add device_lock() back.
So now that there is a mix of killable/interruptible lock usage all the
locks should agree.
Since there really is no risk of these operations being long running
there is no driving need to make them killable/interruptible, so go with
the simple option.
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2024-04-30 5:26 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-04-16 21:46 [PATCH v2 0/4] dax/bus.c: Fixups for dax-bus locking Vishal Verma
2024-04-16 21:46 ` [PATCH v2 1/4] dax/bus.c: replace WARN_ON_ONCE() with lockdep asserts Vishal Verma
2024-04-30 1:23 ` Dan Williams
2024-04-16 21:46 ` [PATCH v2 2/4] dax/bus.c: fix locking for unregister_dax_dev / unregister_dax_mapping paths Vishal Verma
2024-04-30 1:25 ` Dan Williams
2024-04-30 4:11 ` Verma, Vishal L
2024-04-30 5:25 ` Dan Williams
2024-04-16 21:46 ` [PATCH v2 3/4] dax/bus.c: Don't use down_write_killable for non-user processes Vishal Verma
2024-04-30 1:25 ` Dan Williams
2024-04-16 21:46 ` [PATCH v2 4/4] dax/bus.c: Use the right locking mode (read vs write) in size_show Vishal Verma
2024-04-30 1:26 ` Dan Williams
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.