From: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
To: Vlastimil Babka <vbabka@suse.cz>
Cc: Borislav Petkov <bp@alien8.de>, Andy Lutomirski <luto@kernel.org>,
Dave Hansen <dave.hansen@intel.com>,
Sean Christopherson <seanjc@google.com>,
Andrew Morton <akpm@linux-foundation.org>,
Joerg Roedel <jroedel@suse.de>, Ard Biesheuvel <ardb@kernel.org>,
Nikolay Borisov <nik.borisov@suse.com>,
Andi Kleen <ak@linux.intel.com>,
Kuppuswamy Sathyanarayanan
<sathyanarayanan.kuppuswamy@linux.intel.com>,
David Rientjes <rientjes@google.com>,
Tom Lendacky <thomas.lendacky@amd.com>,
Thomas Gleixner <tglx@linutronix.de>,
Peter Zijlstra <peterz@infradead.org>,
Paolo Bonzini <pbonzini@redhat.com>,
Ingo Molnar <mingo@redhat.com>,
Dario Faggioli <dfaggioli@suse.com>,
Mike Rapoport <rppt@kernel.org>,
David Hildenbrand <david@redhat.com>,
Mel Gorman <mgorman@techsingularity.net>,
marcelo.cerri@canonical.com, tim.gardner@canonical.com,
khalid.elmously@canonical.com, philip.cox@canonical.com,
aarcange@redhat.com, peterx@redhat.com, x86@kernel.org,
linux-mm@kvack.org, linux-coco@lists.linux.dev,
linux-efi@vger.kernel.org, linux-kernel@vger.kernel.org,
stable@kernel.org
Subject: Re: [PATCH] efi/unaccepted: Fix soft lockups caused by parallel memory acceptance
Date: Mon, 16 Oct 2023 19:11:46 +0300 [thread overview]
Message-ID: <20231016161146.4j24jprkeaanflpf@box.shutemov.name> (raw)
In-Reply-To: <088593ea-e001-fa87-909f-a196b1373ca4@suse.cz>
On Mon, Oct 16, 2023 at 12:58:34PM +0200, Vlastimil Babka wrote:
> On 10/14/23 22:40, Kirill A. Shutemov wrote:
> > Michael reported soft lockups on a system that has unaccepted memory.
> > This occurs when a user attempts to allocate and accept memory on
> > multiple CPUs simultaneously.
> >
> > The root cause of the issue is that memory acceptance is serialized with
> > a spinlock, allowing only one CPU to accept memory at a time. The other
> > CPUs spin and wait for their turn, leading to starvation and soft lockup
> > reports.
> >
> > To address this, the code has been modified to release the spinlock
> > while accepting memory. This allows for parallel memory acceptance on
> > multiple CPUs.
> >
> > A newly introduced "accepting_list" keeps track of which memory is
> > currently being accepted. This is necessary to prevent parallel
> > acceptance of the same memory block. If a collision occurs, the lock is
> > released and the process is retried.
> >
> > Such collisions should rarely occur. The main path for memory acceptance
> > is the page allocator, which accepts memory in MAX_ORDER chunks. As long
> > as MAX_ORDER is equal to or larger than the unit_size, collisions will
> > never occur because the caller fully owns the memory block being
> > accepted.
> >
> > Aside from the page allocator, only memblock and deferered_free_range()
> > accept memory, but this only happens during boot.
> >
> > The code has been tested with unit_size == 128MiB to trigger collisions
> > and validate the retry codepath.
> >
> > Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> > Reported-by: Michael Roth <michael.roth@amd.com
> > Fixes: 2053bc57f367 ("efi: Add unaccepted memory support")
> > Cc: <stable@kernel.org>
> > ---
> > drivers/firmware/efi/unaccepted_memory.c | 55 ++++++++++++++++++++++--
> > 1 file changed, 51 insertions(+), 4 deletions(-)
> >
> > diff --git a/drivers/firmware/efi/unaccepted_memory.c b/drivers/firmware/efi/unaccepted_memory.c
> > index 853f7dc3c21d..8af0306c8e5c 100644
> > --- a/drivers/firmware/efi/unaccepted_memory.c
> > +++ b/drivers/firmware/efi/unaccepted_memory.c
> > @@ -5,9 +5,17 @@
> > #include <linux/spinlock.h>
> > #include <asm/unaccepted_memory.h>
> >
> > -/* Protects unaccepted memory bitmap */
> > +/* Protects unaccepted memory bitmap and accepting_list */
> > static DEFINE_SPINLOCK(unaccepted_memory_lock);
> >
> > +struct accept_range {
> > + struct list_head list;
> > + unsigned long start;
> > + unsigned long end;
> > +};
> > +
> > +static LIST_HEAD(accepting_list);
> > +
> > /*
> > * accept_memory() -- Consult bitmap and accept the memory if needed.
> > *
> > @@ -24,6 +32,7 @@ void accept_memory(phys_addr_t start, phys_addr_t end)
> > {
> > struct efi_unaccepted_memory *unaccepted;
> > unsigned long range_start, range_end;
> > + struct accept_range range, *entry;
> > unsigned long flags;
> > u64 unit_size;
> >
> > @@ -78,20 +87,58 @@ void accept_memory(phys_addr_t start, phys_addr_t end)
> > if (end > unaccepted->size * unit_size * BITS_PER_BYTE)
> > end = unaccepted->size * unit_size * BITS_PER_BYTE;
> >
> > - range_start = start / unit_size;
> > -
> > + range.start = start / unit_size;
> > + range.end = DIV_ROUND_UP(end, unit_size);
> > +retry:
> > spin_lock_irqsave(&unaccepted_memory_lock, flags);
> > +
> > + /*
> > + * Check if anybody works on accepting the same range of the memory.
> > + *
> > + * The check with unit_size granularity. It is crucial to catch all
>
> "The check is done ..." ?
Yep.
> > + * accept requests to the same unit_size block, even if they don't
> > + * overlap on physical address level.
> > + */
> > + list_for_each_entry(entry, &accepting_list, list) {
> > + if (entry->end < range.start)
> > + continue;
> > + if (entry->start >= range.end)
> > + continue;
>
> Hmm we really don't have a macro for ranges_intersect()? Given how easy is
> to make a mistake. I found only zone_intersects().
I don't know any.
> > +
> > + /*
> > + * Somebody else accepting the range. Or at least part of it.
> > + *
> > + * Drop the lock and retry until it is complete.
> > + */
> > + spin_unlock_irqrestore(&unaccepted_memory_lock, flags);
> > + cond_resched();
> > + goto retry;
> > + }
> > +
> > + /*
> > + * Register that the range is about to be accepted.
> > + * Make sure nobody else will accept it.
> > + */
> > + list_add(&range.list, &accepting_list);
> > +
> > + range_start = range.start;
> > for_each_set_bitrange_from(range_start, range_end, unaccepted->bitmap,
> > - DIV_ROUND_UP(end, unit_size)) {
> > + range.end) {
> > unsigned long phys_start, phys_end;
> > unsigned long len = range_end - range_start;
> >
> > phys_start = range_start * unit_size + unaccepted->phys_base;
> > phys_end = range_end * unit_size + unaccepted->phys_base;
> >
> > + spin_unlock_irqrestore(&unaccepted_memory_lock, flags);
>
> Hm so this is bad, AFAICS. We enable IRQs, then an IRQ can come and try to
> accept in the same unit_size block, so it will keep the retrying by the goto
> above and itself have irqs disabled so the cond_resched() will never let us
> finish?
Good catch. Will fix in the next version.
--
Kiryl Shutsemau / Kirill A. Shutemov
prev parent reply other threads:[~2023-10-16 16:12 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-10-14 20:40 [PATCH] efi/unaccepted: Fix soft lockups caused by parallel memory acceptance Kirill A. Shutemov
2023-10-15 17:02 ` Nikolay Borisov
2023-10-15 18:52 ` Kirill A. Shutemov
2023-10-16 9:17 ` Nikolay Borisov
2023-10-16 10:58 ` Vlastimil Babka
2023-10-16 16:11 ` Kirill A. Shutemov [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20231016161146.4j24jprkeaanflpf@box.shutemov.name \
--to=kirill.shutemov@linux.intel.com \
--cc=aarcange@redhat.com \
--cc=ak@linux.intel.com \
--cc=akpm@linux-foundation.org \
--cc=ardb@kernel.org \
--cc=bp@alien8.de \
--cc=dave.hansen@intel.com \
--cc=david@redhat.com \
--cc=dfaggioli@suse.com \
--cc=jroedel@suse.de \
--cc=khalid.elmously@canonical.com \
--cc=linux-coco@lists.linux.dev \
--cc=linux-efi@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=luto@kernel.org \
--cc=marcelo.cerri@canonical.com \
--cc=mgorman@techsingularity.net \
--cc=mingo@redhat.com \
--cc=nik.borisov@suse.com \
--cc=pbonzini@redhat.com \
--cc=peterx@redhat.com \
--cc=peterz@infradead.org \
--cc=philip.cox@canonical.com \
--cc=rientjes@google.com \
--cc=rppt@kernel.org \
--cc=sathyanarayanan.kuppuswamy@linux.intel.com \
--cc=seanjc@google.com \
--cc=stable@kernel.org \
--cc=tglx@linutronix.de \
--cc=thomas.lendacky@amd.com \
--cc=tim.gardner@canonical.com \
--cc=vbabka@suse.cz \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).