From: Fuad Tabba <tabba@google.com>
To: kvmarm@lists.linux.dev
Cc: maz@kernel.org, will@kernel.org, qperret@google.com,
tabba@google.com, seanjc@google.com, alexandru.elisei@arm.com,
catalin.marinas@arm.com, philmd@linaro.org, james.morse@arm.com,
suzuki.poulose@arm.com, oliver.upton@linux.dev,
mark.rutland@arm.com, broonie@kernel.org, joey.gouly@arm.com,
rananta@google.com, smostafa@google.com
Subject: [PATCH v2 30/47] KVM: arm64: Remove locking from EL2 allocation fast-paths
Date: Tue, 16 Apr 2024 10:56:21 +0100 [thread overview]
Message-ID: <20240416095638.3620345-31-tabba@google.com> (raw)
In-Reply-To: <20240416095638.3620345-1-tabba@google.com>
From: Will Deacon <will@kernel.org>
hyp_{get,put}_page() are called extensively from the page-table code
to adjust reference counts on page-table pages. As a small step towards
removing reader serialisation on these paths, drop the 'hyp_pool' lock
in the case where the refcount remains positive, only taking the lock
if the page is to be freed back to the allocator.
Remove a misleading comment at the same time, which implies that a page
with a refcount of zero and which is not attached to a freelist is
unsafe whereas in practice it's the other way around which can lead to
problems.
Signed-off-by: Will Deacon <will@kernel.org>
Signed-off-by: Fuad Tabba <tabba@google.com>
---
arch/arm64/kvm/hyp/include/nvhe/gfp.h | 6 +-----
arch/arm64/kvm/hyp/nvhe/page_alloc.c | 16 ++++------------
2 files changed, 5 insertions(+), 17 deletions(-)
diff --git a/arch/arm64/kvm/hyp/include/nvhe/gfp.h b/arch/arm64/kvm/hyp/include/nvhe/gfp.h
index 97c527ef53c2..24eb7840d98e 100644
--- a/arch/arm64/kvm/hyp/include/nvhe/gfp.h
+++ b/arch/arm64/kvm/hyp/include/nvhe/gfp.h
@@ -10,11 +10,7 @@
#define HYP_NO_ORDER USHRT_MAX
struct hyp_pool {
- /*
- * Spinlock protecting concurrent changes to the memory pool as well as
- * the struct hyp_page of the pool's pages until we have a proper atomic
- * API at EL2.
- */
+ /* lock protecting concurrent changes to the memory pool. */
hyp_spinlock_t lock;
struct list_head free_area[NR_PAGE_ORDERS];
phys_addr_t range_start;
diff --git a/arch/arm64/kvm/hyp/nvhe/page_alloc.c b/arch/arm64/kvm/hyp/nvhe/page_alloc.c
index 169cdb43b4b8..96b52f545af0 100644
--- a/arch/arm64/kvm/hyp/nvhe/page_alloc.c
+++ b/arch/arm64/kvm/hyp/nvhe/page_alloc.c
@@ -155,33 +155,25 @@ static struct hyp_page *__hyp_extract_page(struct hyp_pool *pool,
static void __hyp_put_page(struct hyp_pool *pool, struct hyp_page *p)
{
- if (hyp_page_ref_dec_and_test(p))
+ if (hyp_page_ref_dec_and_test(p)) {
+ hyp_spin_lock(&pool->lock);
__hyp_attach_page(pool, p);
+ hyp_spin_unlock(&pool->lock);
+ }
}
-/*
- * Changes to the buddy tree and page refcounts must be done with the hyp_pool
- * lock held. If a refcount change requires an update to the buddy tree (e.g.
- * hyp_put_page()), both operations must be done within the same critical
- * section to guarantee transient states (e.g. a page with null refcount but
- * not yet attached to a free list) can't be observed by well-behaved readers.
- */
void hyp_put_page(struct hyp_pool *pool, void *addr)
{
struct hyp_page *p = hyp_virt_to_page(addr);
- hyp_spin_lock(&pool->lock);
__hyp_put_page(pool, p);
- hyp_spin_unlock(&pool->lock);
}
void hyp_get_page(struct hyp_pool *pool, void *addr)
{
struct hyp_page *p = hyp_virt_to_page(addr);
- hyp_spin_lock(&pool->lock);
hyp_page_ref_inc(p);
- hyp_spin_unlock(&pool->lock);
}
void hyp_split_page(struct hyp_page *p)
--
2.44.0.683.g7961c838ac-goog
next prev parent reply other threads:[~2024-04-16 9:57 UTC|newest]
Thread overview: 67+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-04-16 9:55 [PATCH v2 00/47] KVM: arm64: Preamble for pKVM Fuad Tabba
2024-04-16 9:55 ` [PATCH v2 01/47] KVM: arm64: Allocate per-cpu memory for the host fpsimd state in pKVM Fuad Tabba
2024-04-16 10:57 ` Mark Brown
2024-04-16 12:54 ` Marc Zyngier
2024-04-16 13:08 ` Marc Zyngier
2024-04-16 13:10 ` Fuad Tabba
2024-04-16 9:55 ` [PATCH v2 02/47] KVM: arm64: Do not re-initialize the KVM lock Fuad Tabba
2024-04-16 9:55 ` [PATCH v2 03/47] KVM: arm64: Issue CMOs when tearing down guest s2 pages Fuad Tabba
2024-04-16 9:55 ` [PATCH v2 04/47] KVM: arm64: Avoid BUG-ing from the host abort path Fuad Tabba
2024-04-16 9:55 ` [PATCH v2 05/47] KVM: arm64: Fix comment for __pkvm_vcpu_init_traps() Fuad Tabba
2024-04-16 9:55 ` [PATCH v2 06/47] KVM: arm64: Change kvm_handle_mmio_return() return polarity Fuad Tabba
2024-04-16 9:55 ` [PATCH v2 07/47] KVM: arm64: Move setting the page as dirty out of the critical section Fuad Tabba
2024-04-16 9:55 ` [PATCH v2 08/47] KVM: arm64: Check for PTE validity when checking for executable/cacheable Fuad Tabba
2024-04-16 9:56 ` [PATCH v2 09/47] KVM: arm64: Avoid BBM when changing only s/w bits in Stage-2 PTE Fuad Tabba
2024-04-16 16:59 ` Oliver Upton
2024-04-16 18:06 ` Fuad Tabba
2024-04-16 18:19 ` Oliver Upton
2024-04-16 9:56 ` [PATCH v2 10/47] KVM: arm64: Support TLB invalidation in guest context Fuad Tabba
2024-04-16 9:56 ` [PATCH v2 11/47] KVM: arm64: Simplify vgic-v3 hypercalls Fuad Tabba
2024-04-16 9:56 ` [PATCH v2 12/47] KVM: arm64: Add is_pkvm_initialized() helper Fuad Tabba
2024-04-16 9:56 ` [PATCH v2 13/47] KVM: arm64: Introduce predicates to check for protected state Fuad Tabba
2024-04-16 17:18 ` Oliver Upton
2024-04-16 18:07 ` Fuad Tabba
2024-04-16 18:10 ` Oliver Upton
2024-04-16 18:12 ` Fuad Tabba
2024-04-16 9:56 ` [PATCH v2 14/47] KVM: arm64: Split up nvhe/fixed_config.h Fuad Tabba
2024-04-16 9:56 ` [PATCH v2 15/47] KVM: arm64: Move pstate reset value definitions to kvm_arm.h Fuad Tabba
2024-04-16 9:56 ` [PATCH v2 16/47] KVM: arm64: Clarify rationale for ZCR_EL1 value restored on guest exit Fuad Tabba
2024-04-16 10:47 ` Mark Brown
2024-04-16 13:38 ` Fuad Tabba
2024-04-16 9:56 ` [PATCH v2 17/47] KVM: arm64: Refactor calculating SVE state size to use helpers Fuad Tabba
2024-04-16 10:48 ` Mark Brown
2024-04-16 9:56 ` [PATCH v2 18/47] KVM: arm64: Do not map the host fpsimd state to hyp in pKVM Fuad Tabba
2024-04-16 9:56 ` [PATCH v2 19/47] KVM: arm64: Move some kvm_psci functions to a shared header Fuad Tabba
2024-04-16 9:56 ` [PATCH v2 20/47] KVM: arm64: Refactor reset_mpidr() to extract its computation Fuad Tabba
2024-04-16 9:56 ` [PATCH v2 21/47] KVM: arm64: Refactor kvm_vcpu_enable_ptrauth() for hyp use Fuad Tabba
2024-04-16 9:56 ` [PATCH v2 22/47] KVM: arm64: Refactor enter_exception64() Fuad Tabba
2024-04-16 9:56 ` [PATCH v2 23/47] KVM: arm64: Add PC_UPDATE_REQ flags covering all PC updates Fuad Tabba
2024-04-16 9:56 ` [PATCH v2 24/47] KVM: arm64: Add vcpu flag copy primitive Fuad Tabba
2024-04-16 9:56 ` [PATCH v2 25/47] KVM: arm64: Introduce gfn_to_memslot_prot() Fuad Tabba
2024-04-16 9:56 ` [PATCH v2 26/47] KVM: arm64: Do not use the hva in kvm_handle_guest_abort() Fuad Tabba
2024-04-16 9:56 ` [PATCH v2 27/47] KVM: arm64: Introduce hyp_rwlock_t Fuad Tabba
2024-04-16 9:56 ` [PATCH v2 28/47] KVM: arm64: Add atomics-based checking refcount implementation at EL2 Fuad Tabba
2024-04-16 9:56 ` [PATCH v2 29/47] KVM: arm64: Use atomic refcount helpers for 'struct hyp_page::refcount' Fuad Tabba
2024-04-16 9:56 ` Fuad Tabba [this message]
2024-04-16 9:56 ` [PATCH v2 31/47] KVM: arm64: Reformat/beautify PTP hypercall documentation Fuad Tabba
2024-04-16 9:56 ` [PATCH v2 32/47] KVM: arm64: Rename firmware pseudo-register documentation file Fuad Tabba
2024-04-16 9:56 ` [PATCH v2 33/47] KVM: arm64: Document the KVM/arm64-specific calls in hypercalls.rst Fuad Tabba
2024-04-16 9:56 ` [PATCH v2 34/47] KVM: arm64: Prevent kmemleak from accessing .hyp.data Fuad Tabba
2024-04-16 9:56 ` [PATCH v2 35/47] KVM: arm64: Do not set the virtual timer offset for protected vCPUs Fuad Tabba
2024-04-16 9:56 ` [PATCH v2 36/47] KVM: arm64: Check directly whether a vcpu is protected Fuad Tabba
2024-04-16 9:56 ` [PATCH v2 37/47] KVM: arm64: Trap debug break and watch from guest Fuad Tabba
2024-04-16 9:56 ` [PATCH v2 38/47] KVM: arm64: Restrict protected VM capabilities Fuad Tabba
2024-04-16 9:56 ` [PATCH v2 39/47] KVM: arm64: Do not support MTE for protected VMs Fuad Tabba
2024-04-16 9:56 ` [PATCH v2 40/47] KVM: arm64: Move pkvm_vcpu_init_traps() to hyp vcpu init Fuad Tabba
2024-04-16 9:56 ` [PATCH v2 41/47] KVM: arm64: Fix initializing traps in protected mode Fuad Tabba
2024-04-16 9:56 ` [PATCH v2 42/47] KVM: arm64: Advertise GICv3 sysreg interface to protected guests Fuad Tabba
2024-04-16 9:56 ` [PATCH v2 43/47] KVM: arm64: Force injection of a data abort on NISV MMIO exit Fuad Tabba
2024-04-16 9:56 ` [PATCH v2 44/47] KVM: arm64: Remove unused vcpu parameter from guest_owns_fp_regs() Fuad Tabba
2024-04-16 10:49 ` Mark Brown
2024-04-16 9:56 ` [PATCH v2 45/47] KVM: arm64: Move guest_owns_fp_regs() to enable use in KVM code Fuad Tabba
2024-04-16 10:50 ` Mark Brown
2024-04-16 9:56 ` [PATCH v2 46/47] KVM: arm64: Add host_owns_fp_regs() Fuad Tabba
2024-04-16 10:51 ` Mark Brown
2024-04-16 9:56 ` [PATCH v2 47/47] KVM: arm64: Refactor checks for FP state ownership Fuad Tabba
2024-04-16 10:55 ` Mark Brown
2024-04-16 13:33 ` Marc Zyngier
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240416095638.3620345-31-tabba@google.com \
--to=tabba@google.com \
--cc=alexandru.elisei@arm.com \
--cc=broonie@kernel.org \
--cc=catalin.marinas@arm.com \
--cc=james.morse@arm.com \
--cc=joey.gouly@arm.com \
--cc=kvmarm@lists.linux.dev \
--cc=mark.rutland@arm.com \
--cc=maz@kernel.org \
--cc=oliver.upton@linux.dev \
--cc=philmd@linaro.org \
--cc=qperret@google.com \
--cc=rananta@google.com \
--cc=seanjc@google.com \
--cc=smostafa@google.com \
--cc=suzuki.poulose@arm.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.