From: Yang Shi <shy828301@gmail.com>
To: Hugh Dickins <hughd@google.com>
Cc: Andrew Morton <akpm@linux-foundation.org>,
"Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>,
Wang Yugui <wangyugui@e16-tech.com>,
Matthew Wilcox <willy@infradead.org>,
Naoya Horiguchi <naoya.horiguchi@nec.com>,
Alistair Popple <apopple@nvidia.com>,
Ralph Campbell <rcampbell@nvidia.com>, Zi Yan <ziy@nvidia.com>,
Miaohe Lin <linmiaohe@huawei.com>,
Minchan Kim <minchan@kernel.org>, Jue Wang <juew@google.com>,
Peter Xu <peterx@redhat.com>, Jan Kara <jack@suse.cz>,
Shakeel Butt <shakeelb@google.com>,
Oscar Salvador <osalvador@suse.de>, Linux MM <linux-mm@kvack.org>,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v2 02/10] mm/thp: make is_huge_zero_pmd() safe and quicker
Date: Wed, 9 Jun 2021 09:56:55 -0700 [thread overview]
Message-ID: <CAHbLzkqFba6c7=PM39sPqBdZyc+Yqp=9v2SxdwWM-4Z946ZsRw@mail.gmail.com> (raw)
In-Reply-To: <21ea9ca-a1f5-8b90-5e88-95fb1c49bbfa@google.com>
On Tue, Jun 8, 2021 at 9:08 PM Hugh Dickins <hughd@google.com> wrote:
>
> Most callers of is_huge_zero_pmd() supply a pmd already verified present;
> but a few (notably zap_huge_pmd()) do not - it might be a pmd migration
> entry, in which the pfn is encoded differently from a present pmd: which
> might pass the is_huge_zero_pmd() test (though not on x86, since L1TF
> forced us to protect against that); or perhaps even crash in pmd_page()
> applied to a swap-like entry.
>
> Make it safe by adding pmd_present() check into is_huge_zero_pmd() itself;
> and make it quicker by saving huge_zero_pfn, so that is_huge_zero_pmd()
> will not need to do that pmd_page() lookup each time.
>
> __split_huge_pmd_locked() checked pmd_trans_huge() before: that worked,
> but is unnecessary now that is_huge_zero_pmd() checks present.
>
> Fixes: e71769ae5260 ("mm: enable thp migration for shmem thp")
> Signed-off-by: Hugh Dickins <hughd@google.com>
> Cc: <stable@vger.kernel.org>
Reviewed-by: Yang Shi <shy828301@gmail.com>
> ---
> Patch added (replacing part of first) since the v1 series was posted.
>
> include/linux/huge_mm.h | 8 +++++++-
> mm/huge_memory.c | 5 ++++-
> 2 files changed, 11 insertions(+), 2 deletions(-)
>
> diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
> index 9626fda5efce..2a8ebe6c222e 100644
> --- a/include/linux/huge_mm.h
> +++ b/include/linux/huge_mm.h
> @@ -286,6 +286,7 @@ struct page *follow_devmap_pud(struct vm_area_struct *vma, unsigned long addr,
> vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf, pmd_t orig_pmd);
>
> extern struct page *huge_zero_page;
> +extern unsigned long huge_zero_pfn;
>
> static inline bool is_huge_zero_page(struct page *page)
> {
> @@ -294,7 +295,7 @@ static inline bool is_huge_zero_page(struct page *page)
>
> static inline bool is_huge_zero_pmd(pmd_t pmd)
> {
> - return is_huge_zero_page(pmd_page(pmd));
> + return READ_ONCE(huge_zero_pfn) == pmd_pfn(pmd) && pmd_present(pmd);
> }
>
> static inline bool is_huge_zero_pud(pud_t pud)
> @@ -440,6 +441,11 @@ static inline bool is_huge_zero_page(struct page *page)
> return false;
> }
>
> +static inline bool is_huge_zero_pmd(pmd_t pmd)
> +{
> + return false;
> +}
> +
> static inline bool is_huge_zero_pud(pud_t pud)
> {
> return false;
> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
> index 42cfefc6e66e..5885c5f5836f 100644
> --- a/mm/huge_memory.c
> +++ b/mm/huge_memory.c
> @@ -62,6 +62,7 @@ static struct shrinker deferred_split_shrinker;
>
> static atomic_t huge_zero_refcount;
> struct page *huge_zero_page __read_mostly;
> +unsigned long huge_zero_pfn __read_mostly = ~0UL;
>
> bool transparent_hugepage_enabled(struct vm_area_struct *vma)
> {
> @@ -98,6 +99,7 @@ static bool get_huge_zero_page(void)
> __free_pages(zero_page, compound_order(zero_page));
> goto retry;
> }
> + WRITE_ONCE(huge_zero_pfn, page_to_pfn(zero_page));
>
> /* We take additional reference here. It will be put back by shrinker */
> atomic_set(&huge_zero_refcount, 2);
> @@ -147,6 +149,7 @@ static unsigned long shrink_huge_zero_page_scan(struct shrinker *shrink,
> if (atomic_cmpxchg(&huge_zero_refcount, 1, 0) == 1) {
> struct page *zero_page = xchg(&huge_zero_page, NULL);
> BUG_ON(zero_page == NULL);
> + WRITE_ONCE(huge_zero_pfn, ~0UL);
> __free_pages(zero_page, compound_order(zero_page));
> return HPAGE_PMD_NR;
> }
> @@ -2071,7 +2074,7 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd,
> return;
> }
>
> - if (pmd_trans_huge(*pmd) && is_huge_zero_pmd(*pmd)) {
> + if (is_huge_zero_pmd(*pmd)) {
> /*
> * FIXME: Do we want to invalidate secondary mmu by calling
> * mmu_notifier_invalidate_range() see comments below inside
> --
> 2.26.2
>
next prev parent reply other threads:[~2021-06-09 16:57 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <af88612-1473-2eaa-903-8d1a448b26@google.com>
2021-06-09 4:08 ` [PATCH v2 02/10] mm/thp: make is_huge_zero_pmd() safe and quicker Hugh Dickins
2021-06-09 10:22 ` Kirill A. Shutemov
2021-06-09 16:56 ` Yang Shi [this message]
2021-06-09 4:14 ` [PATCH v2 04/10] mm/thp: fix vma_address() if virtual address below file offset Hugh Dickins
2021-06-09 4:16 ` [PATCH v2 05/10] mm/thp: fix page_address_in_vma() on file THP tails Hugh Dickins
2021-06-09 4:19 ` [PATCH v2 06/10] mm/thp: unmap_mapping_page() to fix THP truncate_cleanup_page() Hugh Dickins
2021-06-09 17:02 ` Yang Shi
2021-06-09 21:11 ` Hugh Dickins
2021-06-09 21:16 ` [PATCH v3 " Hugh Dickins
2021-06-09 21:51 ` Yang Shi
2021-06-09 4:22 ` [PATCH v2 07/10] mm: thp: replace DEBUG_VM BUG with VM_WARN when unmap fails for split Hugh Dickins
2021-06-09 4:25 ` [PATCH v2 08/10] mm: rmap: make try_to_unmap() void function Hugh Dickins
2021-06-10 7:57 ` HORIGUCHI NAOYA(堀口 直也)
2021-06-09 4:27 ` [PATCH v2 09/10] mm/thp: remap_page() is only needed on anonymous THP Hugh Dickins
2021-06-09 4:30 ` [PATCH v2 10/10] mm: hwpoison_user_mappings() try_to_unmap() with TTU_SYNC Hugh Dickins
2021-06-09 10:27 ` Kirill A. Shutemov
2021-06-10 7:38 ` HORIGUCHI NAOYA(堀口 直也)
[not found] ` <20210610151505.d0124033e55bda07fa3d4408@linux-foundation.org>
[not found] ` <2014832.e7zRqyNrDn@nvdebian>
2021-06-11 0:15 ` [PATCH v2 00/10] mm/thp: fix THP splitting unmap BUGs and related Hugh Dickins
2021-06-11 7:28 ` Alistair Popple
2021-06-11 20:56 ` Hugh Dickins
2021-06-12 7:34 ` Alistair Popple
2021-06-12 8:20 ` Hugh Dickins
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAHbLzkqFba6c7=PM39sPqBdZyc+Yqp=9v2SxdwWM-4Z946ZsRw@mail.gmail.com' \
--to=shy828301@gmail.com \
--cc=akpm@linux-foundation.org \
--cc=apopple@nvidia.com \
--cc=hughd@google.com \
--cc=jack@suse.cz \
--cc=juew@google.com \
--cc=kirill.shutemov@linux.intel.com \
--cc=linmiaohe@huawei.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=minchan@kernel.org \
--cc=naoya.horiguchi@nec.com \
--cc=osalvador@suse.de \
--cc=peterx@redhat.com \
--cc=rcampbell@nvidia.com \
--cc=shakeelb@google.com \
--cc=wangyugui@e16-tech.com \
--cc=willy@infradead.org \
--cc=ziy@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.