From: Zhongkun He <hezhongkun.hzk@bytedance.com>
To: Chengming Zhou <chengming.zhou@linux.dev>
Cc: Johannes Weiner <hannes@cmpxchg.org>,
Yosry Ahmed <yosryahmed@google.com>,
Andrew Morton <akpm@linux-foundation.org>,
linux-mm <linux-mm@kvack.org>,
wuyun.abel@bytedance.com, zhouchengming@bytedance.com,
Nhat Pham <nphamcs@gmail.com>
Subject: Re: [External] Re: [bug report] mm/zswap :memory corruption after zswap_load().
Date: Thu, 21 Mar 2024 13:09:22 +0800 [thread overview]
Message-ID: <CACSyD1PJGFar315Y79MmrFN0-chzWtm15GuZDKtw7qN9pQpdaA@mail.gmail.com> (raw)
In-Reply-To: <01b0b8e8-af1d-4fbe-951e-278e882283fd@linux.dev>
On Thu, Mar 21, 2024 at 12:42 PM Chengming Zhou
<chengming.zhou@linux.dev> wrote:
>
> On 2024/3/21 12:34, Zhongkun He wrote:
> > Hey folks,
> >
> > Recently, I tested the zswap with memory reclaiming in the mainline
> > (6.8) and found a memory corruption issue related to exclusive loads.
>
> Is this fix included? 13ddaf26be32 ("mm/swap: fix race when skipping swapcache")
> This fix avoids concurrent swapin using the same swap entry.
>
Yes, This fix avoids concurrent swapin from different cpu, but the
reported issue occurs
on the same cpu.
Thanks.
> Thanks.
>
> >
> >
> > root@**:/sys/fs/cgroup/zz# stress --vm 5 --vm-bytes 1g --vm-hang 3 --vm-keep
> > stress: info: [31753] dispatching hogs: 0 cpu, 0 io, 5 vm, 0 hdd
> > stress: FAIL: [31758] (522) memory corruption at: 0x7f347ed1a010
> > stress: FAIL: [31753] (394) <-- worker 31758 returned error 1
> > stress: WARN: [31753] (396) now reaping child worker processes
> > stress: FAIL: [31753] (451) failed run completed in 14s
> >
> >
> > 1. Test step(the frequency of memory reclaiming has been accelerated):
> > -------------------------
> > a. set up the zswap, zram and cgroup V2
> > b. echo 0 > /sys/kernel/mm/lru_gen/enabled
> > (Increase the probability of problems occurring)
> > c. mkdir /sys/fs/cgroup/zz
> > echo $$ > /sys/fs/cgroup/zz/cgroup.procs
> > cd /sys/fs/cgroup/zz/
> > stress --vm 5 --vm-bytes 1g --vm-hang 3 --vm-keep
> >
> > e. in other shell:
> > while :;do for i in {1..5};do echo 20g >
> > /sys/fs/cgroup/zz/memory.reclaim & done;sleep 1;done
> >
> > 2. Root cause:
> > --------------------------
> > With a small probability, the page fault will occur twice with the
> > original pte, even if a new pte has been successfully set.
> > Unfortunately, zswap_entry has been released during the first page fault
> > with exclusive loads, so zswap_load will fail, and there is no corresponding
> > data in swap space, memory corruption occurs.
> >
> > bpftrace -e'k:zswap_load {printf("%lld, %lld\n", ((struct page
> > *)arg0)->private,nsecs)}'
> > --include linux/mm_types.h > a.txt
> >
> > look up the same index:
> >
> > index nsecs
> > 1318876, 8976040736819
> > 1318876, 8976040746078
> >
> > 4123110, 8976234682970
> > 4123110, 8976234689736
> >
> > 2268896, 8976660124792
> > 2268896, 8976660130607
> >
> > 4634105, 8976662117938
> > 4634105, 8976662127596
> >
> > 3. Solution
> >
> > Should we free zswap_entry in batches so that zswap_entry will be
> > valid when the next page fault occurs with the
> > original pte? It would be great if there are other better solutions.
> >
>
next prev parent reply other threads:[~2024-03-21 5:09 UTC|newest]
Thread overview: 29+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-03-21 4:34 [bug report] mm/zswap :memory corruption after zswap_load() Zhongkun He
2024-03-21 4:42 ` Chengming Zhou
2024-03-21 5:09 ` Zhongkun He [this message]
2024-03-21 5:24 ` [External] " Chengming Zhou
2024-03-21 6:36 ` Zhongkun He
2024-03-21 9:28 ` Chengming Zhou
2024-03-21 15:25 ` Nhat Pham
2024-03-21 18:32 ` Yosry Ahmed
2024-03-22 3:27 ` Chengming Zhou
2024-03-22 3:16 ` Zhongkun He
2024-03-22 3:04 ` Zhongkun He
2024-03-22 19:34 ` Yosry Ahmed
2024-03-22 23:04 ` Barry Song
2024-03-22 23:08 ` Yosry Ahmed
2024-03-22 23:18 ` Barry Song
2024-03-22 23:22 ` Yosry Ahmed
2024-03-22 23:32 ` Barry Song
2024-03-22 23:34 ` Yosry Ahmed
2024-03-22 23:38 ` Barry Song
2024-03-22 23:41 ` Yosry Ahmed
2024-03-23 0:34 ` Barry Song
2024-03-23 0:42 ` Yosry Ahmed
2024-03-23 10:48 ` Chris Li
2024-03-23 11:27 ` Chris Li
2024-03-23 12:41 ` Zhongkun He
2024-03-23 1:34 ` Zhongkun He
2024-03-23 1:36 ` Yosry Ahmed
2024-03-23 10:52 ` Chris Li
2024-03-23 10:55 ` Barry Song
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CACSyD1PJGFar315Y79MmrFN0-chzWtm15GuZDKtw7qN9pQpdaA@mail.gmail.com \
--to=hezhongkun.hzk@bytedance.com \
--cc=akpm@linux-foundation.org \
--cc=chengming.zhou@linux.dev \
--cc=hannes@cmpxchg.org \
--cc=linux-mm@kvack.org \
--cc=nphamcs@gmail.com \
--cc=wuyun.abel@bytedance.com \
--cc=yosryahmed@google.com \
--cc=zhouchengming@bytedance.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.