* [PATCH] selftests/mm: run_vmtests.sh: Fix hugetlb mem size calculation
@ 2024-03-21 21:50 peterx
2024-03-22 3:50 ` Muchun Song
` (3 more replies)
0 siblings, 4 replies; 6+ messages in thread
From: peterx @ 2024-03-21 21:50 UTC (permalink / raw
To: linux-mm, linux-kernel
Cc: Andrew Morton, peterx, Muhammad Usama Anjum, David Hildenbrand,
Nico Pache, Muchun Song
From: Peter Xu <peterx@redhat.com>
The script calculates a mininum required size of hugetlb memories, but
it'll stop working with <1MB huge page sizes, reporting all zeros even if
huge pages are available.
In reality, the calculation doesn't really need to be as comlicated either.
Make it simpler and work for KB-level hugepages too.
Cc: Muhammad Usama Anjum <usama.anjum@collabora.com>
Cc: David Hildenbrand <david@redhat.com>
Cc: Nico Pache <npache@redhat.com>
Cc: Muchun Song <muchun.song@linux.dev>
Signed-off-by: Peter Xu <peterx@redhat.com>
---
tools/testing/selftests/mm/run_vmtests.sh | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)
diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
index c2c542fe7b17..b1b78e45d613 100755
--- a/tools/testing/selftests/mm/run_vmtests.sh
+++ b/tools/testing/selftests/mm/run_vmtests.sh
@@ -152,9 +152,13 @@ done < /proc/meminfo
# both of these requirements into account and attempt to increase
# number of huge pages available.
nr_cpus=$(nproc)
-hpgsize_MB=$((hpgsize_KB / 1024))
-half_ufd_size_MB=$((((nr_cpus * hpgsize_MB + 127) / 128) * 128))
-needmem_KB=$((half_ufd_size_MB * 2 * 1024))
+uffd_min_KB=$((hpgsize_KB * nr_cpus * 2))
+hugetlb_min_KB=$((256 * 1024))
+if [[ $uffd_min_KB -gt $hugetlb_min_KB ]]; then
+ needmem_KB=$uffd_min_KB
+else
+ needmem_KB=$hugetlb_min_KB
+fi
# set proper nr_hugepages
if [ -n "$freepgs" ] && [ -n "$hpgsize_KB" ]; then
--
2.44.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH] selftests/mm: run_vmtests.sh: Fix hugetlb mem size calculation
2024-03-21 21:50 [PATCH] selftests/mm: run_vmtests.sh: Fix hugetlb mem size calculation peterx
@ 2024-03-22 3:50 ` Muchun Song
2024-03-22 17:54 ` David Hildenbrand
` (2 subsequent siblings)
3 siblings, 0 replies; 6+ messages in thread
From: Muchun Song @ 2024-03-22 3:50 UTC (permalink / raw
To: Peter Xu
Cc: Linux-MM, LKML, Andrew Morton, Muhammad Usama Anjum,
David Hildenbrand, Nico Pache
> On Mar 22, 2024, at 05:50, peterx@redhat.com wrote:
>
> From: Peter Xu <peterx@redhat.com>
>
> The script calculates a mininum required size of hugetlb memories, but
> it'll stop working with <1MB huge page sizes, reporting all zeros even if
> huge pages are available.
>
> In reality, the calculation doesn't really need to be as comlicated either.
^
complicated?
> Make it simpler and work for KB-level hugepages too.
>
> Cc: Muhammad Usama Anjum <usama.anjum@collabora.com>
> Cc: David Hildenbrand <david@redhat.com>
> Cc: Nico Pache <npache@redhat.com>
> Cc: Muchun Song <muchun.song@linux.dev>
> Signed-off-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Muchun Song <muchun.song@linux.dev>
Thanks.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] selftests/mm: run_vmtests.sh: Fix hugetlb mem size calculation
2024-03-21 21:50 [PATCH] selftests/mm: run_vmtests.sh: Fix hugetlb mem size calculation peterx
2024-03-22 3:50 ` Muchun Song
@ 2024-03-22 17:54 ` David Hildenbrand
2024-03-24 17:56 ` Muhammad Usama Anjum
2024-04-03 11:04 ` Ryan Roberts
3 siblings, 0 replies; 6+ messages in thread
From: David Hildenbrand @ 2024-03-22 17:54 UTC (permalink / raw
To: peterx, linux-mm, linux-kernel
Cc: Andrew Morton, Muhammad Usama Anjum, Nico Pache, Muchun Song
On 21.03.24 22:50, peterx@redhat.com wrote:
> From: Peter Xu <peterx@redhat.com>
>
> The script calculates a mininum required size of hugetlb memories, but
> it'll stop working with <1MB huge page sizes, reporting all zeros even if
> huge pages are available.
>
> In reality, the calculation doesn't really need to be as comlicated either.
> Make it simpler and work for KB-level hugepages too.
>
> Cc: Muhammad Usama Anjum <usama.anjum@collabora.com>
> Cc: David Hildenbrand <david@redhat.com>
> Cc: Nico Pache <npache@redhat.com>
> Cc: Muchun Song <muchun.song@linux.dev>
> Signed-off-by: Peter Xu <peterx@redhat.com>
> ---
> tools/testing/selftests/mm/run_vmtests.sh | 10 +++++++---
> 1 file changed, 7 insertions(+), 3 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
> index c2c542fe7b17..b1b78e45d613 100755
> --- a/tools/testing/selftests/mm/run_vmtests.sh
> +++ b/tools/testing/selftests/mm/run_vmtests.sh
> @@ -152,9 +152,13 @@ done < /proc/meminfo
> # both of these requirements into account and attempt to increase
> # number of huge pages available.
> nr_cpus=$(nproc)
> -hpgsize_MB=$((hpgsize_KB / 1024))
> -half_ufd_size_MB=$((((nr_cpus * hpgsize_MB + 127) / 128) * 128))
> -needmem_KB=$((half_ufd_size_MB * 2 * 1024))
> +uffd_min_KB=$((hpgsize_KB * nr_cpus * 2))
> +hugetlb_min_KB=$((256 * 1024))
> +if [[ $uffd_min_KB -gt $hugetlb_min_KB ]]; then
> + needmem_KB=$uffd_min_KB
> +else
> + needmem_KB=$hugetlb_min_KB
> +fi
>
> # set proper nr_hugepages
> if [ -n "$freepgs" ] && [ -n "$hpgsize_KB" ]; then
Reviewed-by: David Hildenbrand <david@redhat.com>
--
Cheers,
David / dhildenb
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] selftests/mm: run_vmtests.sh: Fix hugetlb mem size calculation
2024-03-21 21:50 [PATCH] selftests/mm: run_vmtests.sh: Fix hugetlb mem size calculation peterx
2024-03-22 3:50 ` Muchun Song
2024-03-22 17:54 ` David Hildenbrand
@ 2024-03-24 17:56 ` Muhammad Usama Anjum
2024-04-03 11:04 ` Ryan Roberts
3 siblings, 0 replies; 6+ messages in thread
From: Muhammad Usama Anjum @ 2024-03-24 17:56 UTC (permalink / raw
To: peterx, linux-mm, linux-kernel
Cc: Muhammad Usama Anjum, Andrew Morton, David Hildenbrand,
Nico Pache, Muchun Song
On 3/22/24 2:50 AM, peterx@redhat.com wrote:
> From: Peter Xu <peterx@redhat.com>
>
> The script calculates a mininum required size of hugetlb memories, but
> it'll stop working with <1MB huge page sizes, reporting all zeros even if
> huge pages are available.
>
> In reality, the calculation doesn't really need to be as comlicated either.
> Make it simpler and work for KB-level hugepages too.
>
> Cc: Muhammad Usama Anjum <usama.anjum@collabora.com>
> Cc: David Hildenbrand <david@redhat.com>
> Cc: Nico Pache <npache@redhat.com>
> Cc: Muchun Song <muchun.song@linux.dev>
> Signed-off-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Muhammad Usama Anjum <usama.anjum@collabora.com>
> ---
> tools/testing/selftests/mm/run_vmtests.sh | 10 +++++++---
> 1 file changed, 7 insertions(+), 3 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
> index c2c542fe7b17..b1b78e45d613 100755
> --- a/tools/testing/selftests/mm/run_vmtests.sh
> +++ b/tools/testing/selftests/mm/run_vmtests.sh
> @@ -152,9 +152,13 @@ done < /proc/meminfo
> # both of these requirements into account and attempt to increase
> # number of huge pages available.
> nr_cpus=$(nproc)
> -hpgsize_MB=$((hpgsize_KB / 1024))
> -half_ufd_size_MB=$((((nr_cpus * hpgsize_MB + 127) / 128) * 128))
> -needmem_KB=$((half_ufd_size_MB * 2 * 1024))
> +uffd_min_KB=$((hpgsize_KB * nr_cpus * 2))
> +hugetlb_min_KB=$((256 * 1024))
> +if [[ $uffd_min_KB -gt $hugetlb_min_KB ]]; then
> + needmem_KB=$uffd_min_KB
> +else
> + needmem_KB=$hugetlb_min_KB
> +fi
>
> # set proper nr_hugepages
> if [ -n "$freepgs" ] && [ -n "$hpgsize_KB" ]; then
--
BR,
Muhammad Usama Anjum
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] selftests/mm: run_vmtests.sh: Fix hugetlb mem size calculation
2024-03-21 21:50 [PATCH] selftests/mm: run_vmtests.sh: Fix hugetlb mem size calculation peterx
` (2 preceding siblings ...)
2024-03-24 17:56 ` Muhammad Usama Anjum
@ 2024-04-03 11:04 ` Ryan Roberts
2024-04-03 18:19 ` Peter Xu
3 siblings, 1 reply; 6+ messages in thread
From: Ryan Roberts @ 2024-04-03 11:04 UTC (permalink / raw
To: peterx, linux-mm, linux-kernel
Cc: Andrew Morton, Muhammad Usama Anjum, David Hildenbrand,
Nico Pache, Muchun Song
Hi Peter,
On 21/03/2024 21:50, peterx@redhat.com wrote:
> From: Peter Xu <peterx@redhat.com>
>
> The script calculates a mininum required size of hugetlb memories, but
> it'll stop working with <1MB huge page sizes, reporting all zeros even if
> huge pages are available.
>
> In reality, the calculation doesn't really need to be as comlicated either.
> Make it simpler and work for KB-level hugepages too.
>
> Cc: Muhammad Usama Anjum <usama.anjum@collabora.com>
> Cc: David Hildenbrand <david@redhat.com>
> Cc: Nico Pache <npache@redhat.com>
> Cc: Muchun Song <muchun.song@linux.dev>
> Signed-off-by: Peter Xu <peterx@redhat.com>
> ---
> tools/testing/selftests/mm/run_vmtests.sh | 10 +++++++---
> 1 file changed, 7 insertions(+), 3 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
> index c2c542fe7b17..b1b78e45d613 100755
> --- a/tools/testing/selftests/mm/run_vmtests.sh
> +++ b/tools/testing/selftests/mm/run_vmtests.sh
> @@ -152,9 +152,13 @@ done < /proc/meminfo
> # both of these requirements into account and attempt to increase
> # number of huge pages available.
> nr_cpus=$(nproc)
> -hpgsize_MB=$((hpgsize_KB / 1024))
> -half_ufd_size_MB=$((((nr_cpus * hpgsize_MB + 127) / 128) * 128))
Removing this has broken the uffd-stress "hugetlb" and "hugetlb-private" tests
(further down the file), which rely on $half_ufd_size_MB. Now that this is not
defined, they are called with too few params:
# # ---------------------------------
# # running ./uffd-stress hugetlb 32
# # ---------------------------------
# # ERROR: invalid MiB (errno=0, @uffd-stress.c:454)
# #
# # Usage: ./uffd-stress <test type> <MiB> <bounces>
# #
# # Supported <test type>: anon, hugetlb, hugetlb-private, shmem, shmem-private
# #
# # Examples:
# #
# # # Run anonymous memory test on 100MiB region with 99999 bounces:
# # ./uffd-stress anon 100 99999
# #
# # # Run share memory test on 1GiB region with 99 bounces:
# # ./uffd-stress shmem 1000 99
# #
# # # Run hugetlb memory test on 256MiB region with 50 bounces:
# # ./uffd-stress hugetlb 256 50
# #
# # # Run the same hugetlb test but using private file:
# # ./uffd-stress hugetlb-private 256 50
# #
# # # 10MiB-~6GiB 999 bounces anonymous test, continue forever unless an error
triggers
# # while ./uffd-stress anon $[RANDOM % 6000 + 10] 999; do true; done
# #
# # [FAIL]
# not ok 16 uffd-stress hugetlb 32 # exit=1
# # -----------------------------------------
# # running ./uffd-stress hugetlb-private 32
# # -----------------------------------------
# # ERROR: invalid MiB (errno=0, @uffd-stress.c:454)
# #
# # Usage: ./uffd-stress <test type> <MiB> <bounces>
# #
# # Supported <test type>: anon, hugetlb, hugetlb-private, shmem, shmem-private
# #
# # Examples:
# #
# # # Run anonymous memory test on 100MiB region with 99999 bounces:
# # ./uffd-stress anon 100 99999
# #
# # # Run share memory test on 1GiB region with 99 bounces:
# # ./uffd-stress shmem 1000 99
# #
# # # Run hugetlb memory test on 256MiB region with 50 bounces:
# # ./uffd-stress hugetlb 256 50
# #
# # # Run the same hugetlb test but using private file:
# # ./uffd-stress hugetlb-private 256 50
# #
# # # 10MiB-~6GiB 999 bounces anonymous test, continue forever unless an error
triggers
# # while ./uffd-stress anon $[RANDOM % 6000 + 10] 999; do true; done
# #
# # [FAIL]
# not ok 17 uffd-stress hugetlb-private 32 # exit=1
Thanks,
Ryan
> -needmem_KB=$((half_ufd_size_MB * 2 * 1024))
> +uffd_min_KB=$((hpgsize_KB * nr_cpus * 2))
> +hugetlb_min_KB=$((256 * 1024))
> +if [[ $uffd_min_KB -gt $hugetlb_min_KB ]]; then
> + needmem_KB=$uffd_min_KB
> +else
> + needmem_KB=$hugetlb_min_KB
> +fi
>
> # set proper nr_hugepages
> if [ -n "$freepgs" ] && [ -n "$hpgsize_KB" ]; then
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] selftests/mm: run_vmtests.sh: Fix hugetlb mem size calculation
2024-04-03 11:04 ` Ryan Roberts
@ 2024-04-03 18:19 ` Peter Xu
0 siblings, 0 replies; 6+ messages in thread
From: Peter Xu @ 2024-04-03 18:19 UTC (permalink / raw
To: Ryan Roberts
Cc: linux-mm, linux-kernel, Andrew Morton, Muhammad Usama Anjum,
David Hildenbrand, Nico Pache, Muchun Song
On Wed, Apr 03, 2024 at 12:04:00PM +0100, Ryan Roberts wrote:
> > diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
> > index c2c542fe7b17..b1b78e45d613 100755
> > --- a/tools/testing/selftests/mm/run_vmtests.sh
> > +++ b/tools/testing/selftests/mm/run_vmtests.sh
> > @@ -152,9 +152,13 @@ done < /proc/meminfo
> > # both of these requirements into account and attempt to increase
> > # number of huge pages available.
> > nr_cpus=$(nproc)
> > -hpgsize_MB=$((hpgsize_KB / 1024))
> > -half_ufd_size_MB=$((((nr_cpus * hpgsize_MB + 127) / 128) * 128))
>
> Removing this has broken the uffd-stress "hugetlb" and "hugetlb-private" tests
> (further down the file), which rely on $half_ufd_size_MB. Now that this is not
> defined, they are called with too few params:
Those FAILs can be burried in some other libc mismatch issues for me so I
overlooked.. My apologies.
I'll send a fixup soon, thank you Ryan!
--
Peter Xu
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2024-04-03 18:19 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-03-21 21:50 [PATCH] selftests/mm: run_vmtests.sh: Fix hugetlb mem size calculation peterx
2024-03-22 3:50 ` Muchun Song
2024-03-22 17:54 ` David Hildenbrand
2024-03-24 17:56 ` Muhammad Usama Anjum
2024-04-03 11:04 ` Ryan Roberts
2024-04-03 18:19 ` Peter Xu
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).