From: "Matthew Wilcox (Oracle)" <willy@infradead.org>
To: stable@vger.kernel.org
Cc: "Matthew Wilcox (Oracle)" <willy@infradead.org>,
Andrew Morton <akpm@linux-foundation.org>,
Song Liu <songliubraving@fb.com>,
"Kirill A . Shutemov" <kirill@shutemov.name>,
Qian Cai <cai@lca.pw>,
Linus Torvalds <torvalds@linux-foundation.org>
Subject: [PATCH 4/4] mm/filemap: fix storing to a THP shadow entry
Date: Mon, 7 Jun 2021 21:08:45 +0100 [thread overview]
Message-ID: <20210607200845.3860579-5-willy@infradead.org> (raw)
In-Reply-To: <20210607200845.3860579-1-willy@infradead.org>
commit 198b62f83eef1d605d70eca32759c92cdcc14175 upstream
When a THP is removed from the page cache by reclaim, we replace it with a
shadow entry that occupies all slots of the XArray previously occupied by
the THP. If the user then accesses that page again, we only allocate a
single page, but storing it into the shadow entry replaces all entries
with that one page. That leads to bugs like
page dumped because: VM_BUG_ON_PAGE(page_to_pgoff(page) != offset)
------------[ cut here ]------------
kernel BUG at mm/filemap.c:2529!
https://bugzilla.kernel.org/show_bug.cgi?id=206569
This is hard to reproduce with mainline, but happens regularly with the
THP patchset (as so many more THPs are created). This solution is take
from the THP patchset. It splits the shadow entry into order-0 pieces at
the time that we bring a new page into cache.
Fixes: 99cb0dbd47a1 ("mm,thp: add read-only THP support for (non-shmem) FS")
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Cc: Song Liu <songliubraving@fb.com>
Cc: "Kirill A . Shutemov" <kirill@shutemov.name>
Cc: Qian Cai <cai@lca.pw>
Link: https://lkml.kernel.org/r/20200903183029.14930-4-willy@infradead.org
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
---
mm/filemap.c | 37 ++++++++++++++++++++++++++++---------
1 file changed, 28 insertions(+), 9 deletions(-)
diff --git a/mm/filemap.c b/mm/filemap.c
index db542b494883..c10e237cc2c6 100644
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -856,7 +856,6 @@ noinline int __add_to_page_cache_locked(struct page *page,
int huge = PageHuge(page);
struct mem_cgroup *memcg;
int error;
- void *old;
VM_BUG_ON_PAGE(!PageLocked(page), page);
VM_BUG_ON_PAGE(PageSwapBacked(page), page);
@@ -872,21 +871,41 @@ noinline int __add_to_page_cache_locked(struct page *page,
get_page(page);
page->mapping = mapping;
page->index = offset;
+ gfp_mask &= GFP_RECLAIM_MASK;
do {
+ unsigned int order = xa_get_order(xas.xa, xas.xa_index);
+ void *entry, *old = NULL;
+
+ if (order > thp_order(page))
+ xas_split_alloc(&xas, xa_load(xas.xa, xas.xa_index),
+ order, gfp_mask);
xas_lock_irq(&xas);
- old = xas_load(&xas);
- if (old && !xa_is_value(old))
- xas_set_err(&xas, -EEXIST);
+ xas_for_each_conflict(&xas, entry) {
+ old = entry;
+ if (!xa_is_value(entry)) {
+ xas_set_err(&xas, -EEXIST);
+ goto unlock;
+ }
+ }
+
+ if (old) {
+ if (shadowp)
+ *shadowp = old;
+ /* entry may have been split before we acquired lock */
+ order = xa_get_order(xas.xa, xas.xa_index);
+ if (order > thp_order(page)) {
+ xas_split(&xas, old, order);
+ xas_reset(&xas);
+ }
+ }
+
xas_store(&xas, page);
if (xas_error(&xas))
goto unlock;
- if (xa_is_value(old)) {
+ if (old)
mapping->nrexceptional--;
- if (shadowp)
- *shadowp = old;
- }
mapping->nrpages++;
/* hugetlb pages do not participate in page cache accounting */
@@ -894,7 +913,7 @@ noinline int __add_to_page_cache_locked(struct page *page,
__inc_node_page_state(page, NR_FILE_PAGES);
unlock:
xas_unlock_irq(&xas);
- } while (xas_nomem(&xas, gfp_mask & GFP_RECLAIM_MASK));
+ } while (xas_nomem(&xas, gfp_mask));
if (xas_error(&xas))
goto error;
--
2.30.2
next prev parent reply other threads:[~2021-06-07 20:11 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-06-07 20:08 [5.4 PATCH 0/4] Fix a CONFIG_READ_ONLY_THP_FOR_FS bug Matthew Wilcox (Oracle)
2021-06-07 20:08 ` [PATCH 1/4] mm: add thp_order Matthew Wilcox (Oracle)
2021-06-07 20:08 ` [PATCH 2/4] XArray: add xa_get_order Matthew Wilcox (Oracle)
2021-06-07 20:08 ` [PATCH 3/4] XArray: add xas_split Matthew Wilcox (Oracle)
2021-06-07 20:08 ` Matthew Wilcox (Oracle) [this message]
2021-06-08 14:58 ` [5.4 PATCH 0/4] Fix a CONFIG_READ_ONLY_THP_FOR_FS bug Greg KH
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210607200845.3860579-5-willy@infradead.org \
--to=willy@infradead.org \
--cc=akpm@linux-foundation.org \
--cc=cai@lca.pw \
--cc=kirill@shutemov.name \
--cc=songliubraving@fb.com \
--cc=stable@vger.kernel.org \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.