From: Zhongkun He <hezhongkun.hzk@bytedance.com>
To: Johannes Weiner <hannes@cmpxchg.org>,
Yosry Ahmed <yosryahmed@google.com>,
Andrew Morton <akpm@linux-foundation.org>
Cc: linux-mm <linux-mm@kvack.org>,
wuyun.abel@bytedance.com, zhouchengming@bytedance.com,
Nhat Pham <nphamcs@gmail.com>
Subject: [bug report] mm/zswap :memory corruption after zswap_load().
Date: Thu, 21 Mar 2024 12:34:41 +0800 [thread overview]
Message-ID: <CACSyD1N+dUvsu8=zV9P691B9bVq33erwOXNTmEaUbi9DrDeJzw@mail.gmail.com> (raw)
Hey folks,
Recently, I tested the zswap with memory reclaiming in the mainline
(6.8) and found a memory corruption issue related to exclusive loads.
root@**:/sys/fs/cgroup/zz# stress --vm 5 --vm-bytes 1g --vm-hang 3 --vm-keep
stress: info: [31753] dispatching hogs: 0 cpu, 0 io, 5 vm, 0 hdd
stress: FAIL: [31758] (522) memory corruption at: 0x7f347ed1a010
stress: FAIL: [31753] (394) <-- worker 31758 returned error 1
stress: WARN: [31753] (396) now reaping child worker processes
stress: FAIL: [31753] (451) failed run completed in 14s
1. Test step(the frequency of memory reclaiming has been accelerated):
-------------------------
a. set up the zswap, zram and cgroup V2
b. echo 0 > /sys/kernel/mm/lru_gen/enabled
(Increase the probability of problems occurring)
c. mkdir /sys/fs/cgroup/zz
echo $$ > /sys/fs/cgroup/zz/cgroup.procs
cd /sys/fs/cgroup/zz/
stress --vm 5 --vm-bytes 1g --vm-hang 3 --vm-keep
e. in other shell:
while :;do for i in {1..5};do echo 20g >
/sys/fs/cgroup/zz/memory.reclaim & done;sleep 1;done
2. Root cause:
--------------------------
With a small probability, the page fault will occur twice with the
original pte, even if a new pte has been successfully set.
Unfortunately, zswap_entry has been released during the first page fault
with exclusive loads, so zswap_load will fail, and there is no corresponding
data in swap space, memory corruption occurs.
bpftrace -e'k:zswap_load {printf("%lld, %lld\n", ((struct page
*)arg0)->private,nsecs)}'
--include linux/mm_types.h > a.txt
look up the same index:
index nsecs
1318876, 8976040736819
1318876, 8976040746078
4123110, 8976234682970
4123110, 8976234689736
2268896, 8976660124792
2268896, 8976660130607
4634105, 8976662117938
4634105, 8976662127596
3. Solution
Should we free zswap_entry in batches so that zswap_entry will be
valid when the next page fault occurs with the
original pte? It would be great if there are other better solutions.
next reply other threads:[~2024-03-21 4:34 UTC|newest]
Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-03-21 4:34 Zhongkun He [this message]
2024-03-21 4:42 ` [bug report] mm/zswap :memory corruption after zswap_load() Chengming Zhou
2024-03-21 5:09 ` [External] " Zhongkun He
2024-03-21 5:24 ` Chengming Zhou
2024-03-21 6:36 ` Zhongkun He
2024-03-21 9:28 ` Chengming Zhou
2024-03-21 15:25 ` Nhat Pham
2024-03-21 18:32 ` Yosry Ahmed
2024-03-22 3:27 ` Chengming Zhou
2024-03-22 3:16 ` Zhongkun He
2024-03-22 3:04 ` Zhongkun He
2024-03-22 19:34 ` Yosry Ahmed
2024-03-22 23:04 ` Barry Song
2024-03-22 23:08 ` Yosry Ahmed
2024-03-22 23:18 ` Barry Song
2024-03-22 23:22 ` Yosry Ahmed
2024-03-22 23:32 ` Barry Song
2024-03-22 23:34 ` Yosry Ahmed
2024-03-22 23:38 ` Barry Song
2024-03-22 23:41 ` Yosry Ahmed
2024-03-23 0:34 ` Barry Song
2024-03-23 0:42 ` Yosry Ahmed
2024-03-23 10:48 ` Chris Li
2024-03-23 11:27 ` Chris Li
2024-03-23 12:41 ` Zhongkun He
2024-03-23 1:34 ` Zhongkun He
2024-03-23 1:36 ` Yosry Ahmed
2024-03-23 10:52 ` Chris Li
2024-03-23 10:55 ` Barry Song
-- strict thread matches above, loose matches on Subject: below --
2024-03-28 15:45 Trisha Busch
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CACSyD1N+dUvsu8=zV9P691B9bVq33erwOXNTmEaUbi9DrDeJzw@mail.gmail.com' \
--to=hezhongkun.hzk@bytedance.com \
--cc=akpm@linux-foundation.org \
--cc=hannes@cmpxchg.org \
--cc=linux-mm@kvack.org \
--cc=nphamcs@gmail.com \
--cc=wuyun.abel@bytedance.com \
--cc=yosryahmed@google.com \
--cc=zhouchengming@bytedance.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.