From: Paolo Bonzini <pbonzini@redhat.com>
To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org
Cc: Sean Christopherson <seanjc@google.com>,
David Matlack <dmatlack@google.com>,
Kai Huang <kai.huang@intel.com>
Subject: [PATCH 14/17] KVM: x86/mmu: Handle no-slot faults at the beginning of kvm_faultin_pfn()
Date: Tue, 7 May 2024 11:58:14 -0400 [thread overview]
Message-ID: <20240507155817.3951344-15-pbonzini@redhat.com> (raw)
In-Reply-To: <20240507155817.3951344-1-pbonzini@redhat.com>
From: Sean Christopherson <seanjc@google.com>
Handle the "no memslot" case at the beginning of kvm_faultin_pfn(), just
after the private versus shared check, so that there's no need to
repeatedly query whether or not a slot exists. This also makes it more
obvious that, except for private vs. shared attributes, the process of
faulting in a pfn simply doesn't apply to gfns without a slot.
Opportunistically stuff @fault's metadata in kvm_handle_noslot_fault() so
that it doesn't need to be duplicated in all paths that invoke
kvm_handle_noslot_fault(), and to minimize the probability of not stuffing
the right fields.
Leave the existing handle behind, but convert it to a WARN, to guard
against __kvm_faultin_pfn() unexpectedly nullifying fault->slot.
Cc: David Matlack <dmatlack@google.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Message-ID: <20240228024147.41573-14-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
arch/x86/kvm/mmu/mmu.c | 29 +++++++++++++++++------------
1 file changed, 17 insertions(+), 12 deletions(-)
diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index b09c8034ed15..7630ad8cb022 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -3270,6 +3270,10 @@ static int kvm_handle_noslot_fault(struct kvm_vcpu *vcpu,
vcpu_cache_mmio_info(vcpu, gva, fault->gfn,
access & shadow_mmio_access_mask);
+ fault->slot = NULL;
+ fault->pfn = KVM_PFN_NOSLOT;
+ fault->map_writable = false;
+
/*
* If MMIO caching is disabled, emulate immediately without
* touching the shadow page tables as attempting to install an
@@ -4350,15 +4354,18 @@ static int kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault,
return -EFAULT;
}
+ if (unlikely(!slot))
+ return kvm_handle_noslot_fault(vcpu, fault, access);
+
/*
* Retry the page fault if the gfn hit a memslot that is being deleted
* or moved. This ensures any existing SPTEs for the old memslot will
* be zapped before KVM inserts a new MMIO SPTE for the gfn.
*/
- if (slot && (slot->flags & KVM_MEMSLOT_INVALID))
+ if (slot->flags & KVM_MEMSLOT_INVALID)
return RET_PF_RETRY;
- if (slot && slot->id == APIC_ACCESS_PAGE_PRIVATE_MEMSLOT) {
+ if (slot->id == APIC_ACCESS_PAGE_PRIVATE_MEMSLOT) {
/*
* Don't map L1's APIC access page into L2, KVM doesn't support
* using APICv/AVIC to accelerate L2 accesses to L1's APIC,
@@ -4370,12 +4377,9 @@ static int kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault,
* uses different roots for L1 vs. L2, i.e. there is no danger
* of breaking APICv/AVIC for L1.
*/
- if (is_guest_mode(vcpu)) {
- fault->slot = NULL;
- fault->pfn = KVM_PFN_NOSLOT;
- fault->map_writable = false;
- goto faultin_done;
- }
+ if (is_guest_mode(vcpu))
+ return kvm_handle_noslot_fault(vcpu, fault, access);
+
/*
* If the APIC access page exists but is disabled, go directly
* to emulation without caching the MMIO access or creating a
@@ -4386,6 +4390,9 @@ static int kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault,
return RET_PF_EMULATE;
}
+ fault->mmu_seq = vcpu->kvm->mmu_invalidate_seq;
+ smp_rmb();
+
/*
* Check for a relevant mmu_notifier invalidation event before getting
* the pfn from the primary MMU, and before acquiring mmu_lock.
@@ -4407,19 +4414,17 @@ static int kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault,
* *guaranteed* to need to retry, i.e. waiting until mmu_lock is held
* to detect retry guarantees the worst case latency for the vCPU.
*/
- if (fault->slot &&
- mmu_invalidate_retry_gfn_unsafe(vcpu->kvm, fault->mmu_seq, fault->gfn))
+ if (mmu_invalidate_retry_gfn_unsafe(vcpu->kvm, fault->mmu_seq, fault->gfn))
return RET_PF_RETRY;
ret = __kvm_faultin_pfn(vcpu, fault);
if (ret != RET_PF_CONTINUE)
return ret;
-faultin_done:
if (unlikely(is_error_pfn(fault->pfn)))
return kvm_handle_error_pfn(vcpu, fault);
- if (unlikely(!fault->slot))
+ if (WARN_ON_ONCE(!fault->slot))
return kvm_handle_noslot_fault(vcpu, fault, access);
/*
--
2.43.0
next prev parent reply other threads:[~2024-05-07 15:58 UTC|newest]
Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-05-07 15:58 [PATCH v2 00/17] KVM: x86/mmu: Page fault and MMIO cleanups Paolo Bonzini
2024-05-07 15:58 ` [PATCH 01/17] KVM: x86/mmu: Exit to userspace with -EFAULT if private fault hits emulation Paolo Bonzini
2024-05-13 5:25 ` Xiaoyao Li
2024-05-07 15:58 ` [PATCH 02/17] KVM: x86: Remove separate "bit" defines for page fault error code masks Paolo Bonzini
2024-05-13 5:29 ` Xiaoyao Li
2024-05-07 15:58 ` [PATCH 03/17] KVM: x86: Define more SEV+ page fault error bits/flags for #NPF Paolo Bonzini
2024-05-07 15:58 ` [PATCH 04/17] KVM: x86: Move synthetic PFERR_* sanity checks to SVM's #NPF handler Paolo Bonzini
2024-05-13 5:50 ` Xiaoyao Li
2024-05-13 17:31 ` Sean Christopherson
2024-05-14 4:25 ` Xiaoyao Li
2024-05-14 15:32 ` Sean Christopherson
2024-05-15 1:03 ` Xiaoyao Li
2024-05-07 15:58 ` [PATCH 05/17] KVM: x86/mmu: Pass full 64-bit error code when handling page faults Paolo Bonzini
2024-05-07 15:58 ` [PATCH 06/17] KVM: x86/mmu: WARN if upper 32 bits of legacy #PF error code are non-zero Paolo Bonzini
2024-05-07 15:58 ` [PATCH 07/17] KVM: x86/mmu: Use synthetic page fault error code to indicate private faults Paolo Bonzini
2024-05-13 5:56 ` Xiaoyao Li
2024-05-07 15:58 ` [PATCH 08/17] KVM: x86/mmu: check for invalid async page faults involving private memory Paolo Bonzini
2024-05-07 15:58 ` [PATCH 09/17] KVM: x86/mmu: WARN and skip MMIO cache on private, reserved page faults Paolo Bonzini
2024-05-13 6:15 ` Xiaoyao Li
2024-05-07 15:58 ` [PATCH 10/17] KVM: x86/mmu: Move private vs. shared check above slot validity checks Paolo Bonzini
2024-05-13 6:22 ` Xiaoyao Li
2024-05-07 15:58 ` [PATCH 11/17] KVM: x86/mmu: Don't force emulation of L2 accesses to non-APIC internal slots Paolo Bonzini
2024-05-07 15:58 ` [PATCH 12/17] KVM: x86/mmu: Explicitly disallow private accesses to emulated MMIO Paolo Bonzini
2024-05-13 6:26 ` Xiaoyao Li
2024-05-07 15:58 ` [PATCH 13/17] KVM: x86/mmu: Move slot checks from __kvm_faultin_pfn() to kvm_faultin_pfn() Paolo Bonzini
2024-05-13 6:27 ` Xiaoyao Li
2024-05-07 15:58 ` Paolo Bonzini [this message]
2024-05-13 6:28 ` [PATCH 14/17] KVM: x86/mmu: Handle no-slot faults at the beginning of kvm_faultin_pfn() Xiaoyao Li
2024-05-07 15:58 ` [PATCH 15/17] KVM: x86/mmu: Set kvm_page_fault.hva to KVM_HVA_ERR_BAD for "no slot" faults Paolo Bonzini
2024-05-13 6:28 ` Xiaoyao Li
2024-05-07 15:58 ` [PATCH 16/17] KVM: x86/mmu: Initialize kvm_page_fault's pfn and hva to error values Paolo Bonzini
2024-05-13 6:29 ` Xiaoyao Li
2024-05-07 15:58 ` [PATCH 17/17] KVM: x86/mmu: Sanity check that __kvm_faultin_pfn() doesn't create noslot pfns Paolo Bonzini
2024-05-13 6:40 ` Xiaoyao Li
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240507155817.3951344-15-pbonzini@redhat.com \
--to=pbonzini@redhat.com \
--cc=dmatlack@google.com \
--cc=kai.huang@intel.com \
--cc=kvm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=seanjc@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).