From: Fuad Tabba <tabba@google.com>
To: Sean Christopherson <seanjc@google.com>
Cc: "Paolo Bonzini" <pbonzini@redhat.com>,
"Marc Zyngier" <maz@kernel.org>,
"Oliver Upton" <oliver.upton@linux.dev>,
"Huacai Chen" <chenhuacai@kernel.org>,
"Michael Ellerman" <mpe@ellerman.id.au>,
"Anup Patel" <anup@brainfault.org>,
"Paul Walmsley" <paul.walmsley@sifive.com>,
"Palmer Dabbelt" <palmer@dabbelt.com>,
"Albert Ou" <aou@eecs.berkeley.edu>,
"Alexander Viro" <viro@zeniv.linux.org.uk>,
"Christian Brauner" <brauner@kernel.org>,
"Matthew Wilcox (Oracle)" <willy@infradead.org>,
"Andrew Morton" <akpm@linux-foundation.org>,
kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org,
kvmarm@lists.linux.dev, linux-mips@vger.kernel.org,
linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org,
linux-riscv@lists.infradead.org, linux-fsdevel@vger.kernel.org,
linux-mm@kvack.org, linux-kernel@vger.kernel.org,
"Xiaoyao Li" <xiaoyao.li@intel.com>,
"Xu Yilun" <yilun.xu@intel.com>,
"Chao Peng" <chao.p.peng@linux.intel.com>,
"Jarkko Sakkinen" <jarkko@kernel.org>,
"Anish Moorthy" <amoorthy@google.com>,
"David Matlack" <dmatlack@google.com>,
"Yu Zhang" <yu.c.zhang@linux.intel.com>,
"Isaku Yamahata" <isaku.yamahata@intel.com>,
"Mickaël Salaün" <mic@digikod.net>,
"Vlastimil Babka" <vbabka@suse.cz>,
"Vishal Annapurve" <vannapurve@google.com>,
"Ackerley Tng" <ackerleytng@google.com>,
"Maciej Szmigiero" <mail@maciej.szmigiero.name>,
"David Hildenbrand" <david@redhat.com>,
"Quentin Perret" <qperret@google.com>,
"Michael Roth" <michael.roth@amd.com>,
Wang <wei.w.wang@intel.com>,
"Liam Merwick" <liam.merwick@oracle.com>,
"Isaku Yamahata" <isaku.yamahata@gmail.com>,
"Kirill A . Shutemov" <kirill.shutemov@linux.intel.com>
Subject: Re: [PATCH v13 10/35] KVM: Add a dedicated mmu_notifier flag for reclaiming freed memory
Date: Thu, 2 Nov 2023 13:55:03 +0000 [thread overview]
Message-ID: <CA+EHjTx4dQwU3CWzoib_x1tgw66HCtBEiGW_7E8yZxttem6+vw@mail.gmail.com> (raw)
In-Reply-To: <20231027182217.3615211-11-seanjc@google.com>
On Fri, Oct 27, 2023 at 7:22 PM Sean Christopherson <seanjc@google.com> wrote:
>
> Handle AMD SEV's kvm_arch_guest_memory_reclaimed() hook by having
> __kvm_handle_hva_range() return whether or not an overlapping memslot
> was found, i.e. mmu_lock was acquired. Using the .on_unlock() hook
> works, but kvm_arch_guest_memory_reclaimed() needs to run after dropping
> mmu_lock, which makes .on_lock() and .on_unlock() asymmetrical.
>
> Use a small struct to return the tuple of the notifier-specific return,
> plus whether or not overlap was found. Because the iteration helpers are
> __always_inlined, practically speaking, the struct will never actually be
> returned from a function call (not to mention the size of the struct will
> be two bytes in practice).
>
> Signed-off-by: Sean Christopherson <seanjc@google.com>
> ---
Reviewed-by: Fuad Tabba <tabba@google.com>
Tested-by: Fuad Tabba <tabba@google.com>
Cheers,
/fuad
> virt/kvm/kvm_main.c | 53 +++++++++++++++++++++++++++++++--------------
> 1 file changed, 37 insertions(+), 16 deletions(-)
>
> diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
> index 3f5b7c2c5327..2bc04c8ae1f4 100644
> --- a/virt/kvm/kvm_main.c
> +++ b/virt/kvm/kvm_main.c
> @@ -561,6 +561,19 @@ struct kvm_mmu_notifier_range {
> bool may_block;
> };
>
> +/*
> + * The inner-most helper returns a tuple containing the return value from the
> + * arch- and action-specific handler, plus a flag indicating whether or not at
> + * least one memslot was found, i.e. if the handler found guest memory.
> + *
> + * Note, most notifiers are averse to booleans, so even though KVM tracks the
> + * return from arch code as a bool, outer helpers will cast it to an int. :-(
> + */
> +typedef struct kvm_mmu_notifier_return {
> + bool ret;
> + bool found_memslot;
> +} kvm_mn_ret_t;
> +
> /*
> * Use a dedicated stub instead of NULL to indicate that there is no callback
> * function/handler. The compiler technically can't guarantee that a real
> @@ -582,22 +595,25 @@ static const union kvm_mmu_notifier_arg KVM_MMU_NOTIFIER_NO_ARG;
> node; \
> node = interval_tree_iter_next(node, start, last)) \
>
> -static __always_inline int __kvm_handle_hva_range(struct kvm *kvm,
> - const struct kvm_mmu_notifier_range *range)
> +static __always_inline kvm_mn_ret_t __kvm_handle_hva_range(struct kvm *kvm,
> + const struct kvm_mmu_notifier_range *range)
> {
> - bool ret = false, locked = false;
> + struct kvm_mmu_notifier_return r = {
> + .ret = false,
> + .found_memslot = false,
> + };
> struct kvm_gfn_range gfn_range;
> struct kvm_memory_slot *slot;
> struct kvm_memslots *slots;
> int i, idx;
>
> if (WARN_ON_ONCE(range->end <= range->start))
> - return 0;
> + return r;
>
> /* A null handler is allowed if and only if on_lock() is provided. */
> if (WARN_ON_ONCE(IS_KVM_NULL_FN(range->on_lock) &&
> IS_KVM_NULL_FN(range->handler)))
> - return 0;
> + return r;
>
> idx = srcu_read_lock(&kvm->srcu);
>
> @@ -631,8 +647,8 @@ static __always_inline int __kvm_handle_hva_range(struct kvm *kvm,
> gfn_range.end = hva_to_gfn_memslot(hva_end + PAGE_SIZE - 1, slot);
> gfn_range.slot = slot;
>
> - if (!locked) {
> - locked = true;
> + if (!r.found_memslot) {
> + r.found_memslot = true;
> KVM_MMU_LOCK(kvm);
> if (!IS_KVM_NULL_FN(range->on_lock))
> range->on_lock(kvm);
> @@ -640,14 +656,14 @@ static __always_inline int __kvm_handle_hva_range(struct kvm *kvm,
> if (IS_KVM_NULL_FN(range->handler))
> break;
> }
> - ret |= range->handler(kvm, &gfn_range);
> + r.ret |= range->handler(kvm, &gfn_range);
> }
> }
>
> - if (range->flush_on_ret && ret)
> + if (range->flush_on_ret && r.ret)
> kvm_flush_remote_tlbs(kvm);
>
> - if (locked) {
> + if (r.found_memslot) {
> KVM_MMU_UNLOCK(kvm);
> if (!IS_KVM_NULL_FN(range->on_unlock))
> range->on_unlock(kvm);
> @@ -655,8 +671,7 @@ static __always_inline int __kvm_handle_hva_range(struct kvm *kvm,
>
> srcu_read_unlock(&kvm->srcu, idx);
>
> - /* The notifiers are averse to booleans. :-( */
> - return (int)ret;
> + return r;
> }
>
> static __always_inline int kvm_handle_hva_range(struct mmu_notifier *mn,
> @@ -677,7 +692,7 @@ static __always_inline int kvm_handle_hva_range(struct mmu_notifier *mn,
> .may_block = false,
> };
>
> - return __kvm_handle_hva_range(kvm, &range);
> + return __kvm_handle_hva_range(kvm, &range).ret;
> }
>
> static __always_inline int kvm_handle_hva_range_no_flush(struct mmu_notifier *mn,
> @@ -696,7 +711,7 @@ static __always_inline int kvm_handle_hva_range_no_flush(struct mmu_notifier *mn
> .may_block = false,
> };
>
> - return __kvm_handle_hva_range(kvm, &range);
> + return __kvm_handle_hva_range(kvm, &range).ret;
> }
>
> static bool kvm_change_spte_gfn(struct kvm *kvm, struct kvm_gfn_range *range)
> @@ -798,7 +813,7 @@ static int kvm_mmu_notifier_invalidate_range_start(struct mmu_notifier *mn,
> .end = range->end,
> .handler = kvm_mmu_unmap_gfn_range,
> .on_lock = kvm_mmu_invalidate_begin,
> - .on_unlock = kvm_arch_guest_memory_reclaimed,
> + .on_unlock = (void *)kvm_null_fn,
> .flush_on_ret = true,
> .may_block = mmu_notifier_range_blockable(range),
> };
> @@ -830,7 +845,13 @@ static int kvm_mmu_notifier_invalidate_range_start(struct mmu_notifier *mn,
> gfn_to_pfn_cache_invalidate_start(kvm, range->start, range->end,
> hva_range.may_block);
>
> - __kvm_handle_hva_range(kvm, &hva_range);
> + /*
> + * If one or more memslots were found and thus zapped, notify arch code
> + * that guest memory has been reclaimed. This needs to be done *after*
> + * dropping mmu_lock, as x86's reclaim path is slooooow.
> + */
> + if (__kvm_handle_hva_range(kvm, &hva_range).found_memslot)
> + kvm_arch_guest_memory_reclaimed(kvm);
>
> return 0;
> }
> --
> 2.42.0.820.g83a721a137-goog
>
next prev parent reply other threads:[~2023-11-02 13:55 UTC|newest]
Thread overview: 148+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-10-27 18:21 [PATCH v13 00/35] KVM: guest_memfd() and per-page attributes Sean Christopherson
2023-10-27 18:21 ` [PATCH v13 01/35] KVM: Tweak kvm_hva_range and hva_handler_t to allow reusing for gfn ranges Sean Christopherson
2023-11-01 12:46 ` Fuad Tabba
2023-10-27 18:21 ` [PATCH v13 02/35] KVM: Assert that mmu_invalidate_in_progress *never* goes negative Sean Christopherson
2023-10-30 16:27 ` Paolo Bonzini
2023-11-01 12:46 ` Fuad Tabba
2023-10-27 18:21 ` [PATCH v13 03/35] KVM: Use gfn instead of hva for mmu_notifier_retry Sean Christopherson
2023-10-30 16:30 ` Paolo Bonzini
2023-10-30 16:53 ` David Matlack
2023-10-30 17:00 ` Paolo Bonzini
2023-10-30 18:21 ` David Matlack
2023-10-30 18:19 ` David Matlack
2023-11-01 15:31 ` Xu Yilun
2023-10-27 18:21 ` [PATCH v13 04/35] KVM: WARN if there are dangling MMU invalidations at VM destruction Sean Christopherson
2023-10-30 16:32 ` Paolo Bonzini
2023-11-01 12:50 ` Fuad Tabba
2023-10-27 18:21 ` [PATCH v13 05/35] KVM: PPC: Drop dead code related to KVM_ARCH_WANT_MMU_NOTIFIER Sean Christopherson
2023-10-30 16:34 ` Paolo Bonzini
2023-11-01 12:51 ` Fuad Tabba
2023-10-27 18:21 ` [PATCH v13 06/35] KVM: PPC: Return '1' unconditionally for KVM_CAP_SYNC_MMU Sean Christopherson
2023-10-27 18:21 ` [PATCH v13 07/35] KVM: Convert KVM_ARCH_WANT_MMU_NOTIFIER to CONFIG_KVM_GENERIC_MMU_NOTIFIER Sean Christopherson
2023-10-30 16:37 ` Paolo Bonzini
2023-11-01 12:54 ` Fuad Tabba
2023-10-27 18:21 ` [PATCH v13 08/35] KVM: Introduce KVM_SET_USER_MEMORY_REGION2 Sean Christopherson
2023-10-30 16:41 ` Paolo Bonzini
2023-10-30 20:25 ` Sean Christopherson
2023-10-30 22:12 ` Sean Christopherson
2023-10-30 23:22 ` Paolo Bonzini
2023-10-31 0:18 ` Sean Christopherson
2023-10-31 2:26 ` Xiaoyao Li
2023-10-31 14:04 ` Sean Christopherson
2023-11-01 14:19 ` Fuad Tabba
2023-10-27 18:21 ` [PATCH v13 09/35] KVM: Add KVM_EXIT_MEMORY_FAULT exit to report faults to userspace Sean Christopherson
2023-10-30 17:22 ` Paolo Bonzini
2023-11-01 7:30 ` Binbin Wu
2023-11-01 10:52 ` Huang, Kai
2023-11-01 17:36 ` Sean Christopherson
2023-11-02 2:19 ` Xiaoyao Li
2023-11-02 15:51 ` Sean Christopherson
2023-11-02 3:17 ` Huang, Kai
2023-11-02 9:35 ` Huang, Kai
2023-11-02 11:03 ` Paolo Bonzini
2023-11-02 15:44 ` Sean Christopherson
2023-11-02 18:35 ` Huang, Kai
2023-11-02 15:56 ` Sean Christopherson
2023-11-02 11:01 ` Paolo Bonzini
2023-11-03 4:09 ` Xu Yilun
2023-10-27 18:21 ` [PATCH v13 10/35] KVM: Add a dedicated mmu_notifier flag for reclaiming freed memory Sean Christopherson
2023-10-30 17:11 ` Paolo Bonzini
2023-11-02 13:55 ` Fuad Tabba [this message]
2023-10-27 18:21 ` [PATCH v13 11/35] KVM: Drop .on_unlock() mmu_notifier hook Sean Christopherson
2023-10-30 17:18 ` Paolo Bonzini
2023-11-02 13:55 ` Fuad Tabba
2023-10-27 18:21 ` [PATCH v13 12/35] KVM: Prepare for handling only shared mappings in mmu_notifier events Sean Christopherson
2023-10-30 17:21 ` Paolo Bonzini
2023-10-30 22:07 ` Sean Christopherson
2023-11-02 5:59 ` Binbin Wu
2023-11-02 11:14 ` Paolo Bonzini
2023-11-02 14:01 ` Fuad Tabba
2023-11-02 14:41 ` Sean Christopherson
2023-11-02 14:57 ` Fuad Tabba
2023-10-27 18:21 ` [PATCH v13 13/35] KVM: Introduce per-page memory attributes Sean Christopherson
2023-10-30 8:11 ` Chao Gao
2023-10-30 16:10 ` Sean Christopherson
2023-10-30 22:05 ` Sean Christopherson
2023-10-31 16:43 ` David Matlack
2023-11-02 3:01 ` Huang, Kai
2023-11-02 10:32 ` Paolo Bonzini
2023-11-02 10:55 ` Huang, Kai
2023-10-27 18:21 ` [PATCH v13 14/35] mm: Add AS_UNMOVABLE to mark mapping as completely unmovable Sean Christopherson
2023-10-30 17:24 ` Paolo Bonzini
2023-10-27 18:21 ` [PATCH v13 15/35] fs: Export anon_inode_getfile_secure() for use by KVM Sean Christopherson
2023-10-30 17:30 ` Paolo Bonzini
2023-11-02 16:24 ` Christian Brauner
2023-11-03 10:40 ` Paolo Bonzini
2023-10-27 18:21 ` [PATCH v13 16/35] KVM: Add KVM_CREATE_GUEST_MEMFD ioctl() for guest-specific backing memory Sean Christopherson
2023-10-31 2:27 ` Xiaoyao Li
2023-10-31 6:30 ` Chao Gao
2023-10-31 14:10 ` Sean Christopherson
2023-10-31 15:05 ` Fuad Tabba
2023-10-31 22:13 ` Sean Christopherson
2023-10-31 22:18 ` Paolo Bonzini
2023-11-01 10:51 ` Fuad Tabba
2023-11-01 21:55 ` Sean Christopherson
2023-11-02 13:52 ` Fuad Tabba
2023-11-03 23:17 ` Sean Christopherson
2023-10-31 18:24 ` David Matlack
2023-10-31 21:36 ` Sean Christopherson
2023-10-31 22:39 ` David Matlack
2023-11-02 15:48 ` Paolo Bonzini
2023-11-02 16:03 ` Sean Christopherson
2023-11-02 16:28 ` David Matlack
2023-11-02 17:37 ` Sean Christopherson
2023-11-03 9:42 ` Fuad Tabba
2023-11-04 10:26 ` Xu Yilun
2023-11-06 15:43 ` Sean Christopherson
2023-10-27 18:21 ` [PATCH v13 17/35] KVM: Add transparent hugepage support for dedicated guest memory Sean Christopherson
2023-10-31 8:35 ` Xiaoyao Li
2023-10-31 14:16 ` Sean Christopherson
2023-11-01 7:25 ` Xiaoyao Li
2023-11-01 13:41 ` Sean Christopherson
2023-11-01 13:49 ` Paolo Bonzini
2023-11-01 16:36 ` Sean Christopherson
2023-11-01 22:28 ` Paolo Bonzini
2023-11-01 22:34 ` Sean Christopherson
2023-11-01 23:17 ` Paolo Bonzini
2023-11-02 15:38 ` Sean Christopherson
2023-11-02 15:46 ` Paolo Bonzini
2023-11-27 11:13 ` Vlastimil Babka
2023-11-29 22:40 ` Sean Christopherson
2023-10-27 18:22 ` [PATCH v13 18/35] KVM: x86: "Reset" vcpu->run->exit_reason early in KVM_RUN Sean Christopherson
2023-10-30 17:31 ` Paolo Bonzini
2023-11-02 14:16 ` Fuad Tabba
2023-10-27 18:22 ` [PATCH v13 19/35] KVM: x86: Disallow hugepages when memory attributes are mixed Sean Christopherson
2023-10-27 18:22 ` [PATCH v13 20/35] KVM: x86/mmu: Handle page fault for private memory Sean Christopherson
2023-11-02 14:34 ` Fuad Tabba
2023-11-05 13:02 ` Xu Yilun
2023-11-05 16:19 ` Paolo Bonzini
2023-11-06 13:29 ` Xu Yilun
2023-11-06 15:56 ` Sean Christopherson
2023-10-27 18:22 ` [PATCH v13 21/35] KVM: Drop superfluous __KVM_VCPU_MULTIPLE_ADDRESS_SPACE macro Sean Christopherson
2023-11-02 14:35 ` Fuad Tabba
2023-10-27 18:22 ` [PATCH v13 22/35] KVM: Allow arch code to track number of memslot address spaces per VM Sean Christopherson
2023-10-30 17:34 ` Paolo Bonzini
2023-11-02 14:52 ` Fuad Tabba
2023-10-27 18:22 ` [PATCH v13 23/35] KVM: x86: Add support for "protected VMs" that can utilize private memory Sean Christopherson
2023-10-30 17:36 ` Paolo Bonzini
2023-11-06 11:00 ` Fuad Tabba
2023-11-06 11:03 ` Paolo Bonzini
2023-10-27 18:22 ` [PATCH v13 24/35] KVM: selftests: Drop unused kvm_userspace_memory_region_find() helper Sean Christopherson
2023-10-27 18:22 ` [PATCH v13 25/35] KVM: selftests: Convert lib's mem regions to KVM_SET_USER_MEMORY_REGION2 Sean Christopherson
2024-04-25 14:12 ` Dan Carpenter
2024-04-25 14:45 ` Shuah Khan
2024-04-25 15:09 ` Sean Christopherson
2024-04-25 16:22 ` Shuah Khan
2024-04-26 7:33 ` Jarkko Sakkinen
2023-10-27 18:22 ` [PATCH v13 26/35] KVM: selftests: Add support for creating private memslots Sean Christopherson
2023-10-27 18:22 ` [PATCH v13 27/35] KVM: selftests: Add helpers to convert guest memory b/w private and shared Sean Christopherson
2023-11-06 11:26 ` Fuad Tabba
2023-10-27 18:22 ` [PATCH v13 28/35] KVM: selftests: Add helpers to do KVM_HC_MAP_GPA_RANGE hypercalls (x86) Sean Christopherson
2023-10-27 18:22 ` [PATCH v13 29/35] KVM: selftests: Introduce VM "shape" to allow tests to specify the VM type Sean Christopherson
2023-10-27 18:22 ` [PATCH v13 30/35] KVM: selftests: Add GUEST_SYNC[1-6] macros for synchronizing more data Sean Christopherson
2023-10-27 18:22 ` [PATCH v13 31/35] KVM: selftests: Add x86-only selftest for private memory conversions Sean Christopherson
2023-10-27 18:22 ` [PATCH v13 32/35] KVM: selftests: Add KVM_SET_USER_MEMORY_REGION2 helper Sean Christopherson
2023-10-27 18:22 ` [PATCH v13 33/35] KVM: selftests: Expand set_memory_region_test to validate guest_memfd() Sean Christopherson
2023-10-27 18:22 ` [PATCH v13 34/35] KVM: selftests: Add basic selftest for guest_memfd() Sean Christopherson
2023-10-27 18:22 ` [PATCH v13 35/35] KVM: selftests: Test KVM exit behavior for private memory/access Sean Christopherson
2023-10-30 17:39 ` [PATCH v13 00/35] KVM: guest_memfd() and per-page attributes Paolo Bonzini
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CA+EHjTx4dQwU3CWzoib_x1tgw66HCtBEiGW_7E8yZxttem6+vw@mail.gmail.com \
--to=tabba@google.com \
--cc=ackerleytng@google.com \
--cc=akpm@linux-foundation.org \
--cc=amoorthy@google.com \
--cc=anup@brainfault.org \
--cc=aou@eecs.berkeley.edu \
--cc=brauner@kernel.org \
--cc=chao.p.peng@linux.intel.com \
--cc=chenhuacai@kernel.org \
--cc=david@redhat.com \
--cc=dmatlack@google.com \
--cc=isaku.yamahata@gmail.com \
--cc=isaku.yamahata@intel.com \
--cc=jarkko@kernel.org \
--cc=kirill.shutemov@linux.intel.com \
--cc=kvm-riscv@lists.infradead.org \
--cc=kvm@vger.kernel.org \
--cc=kvmarm@lists.linux.dev \
--cc=liam.merwick@oracle.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mips@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-riscv@lists.infradead.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=mail@maciej.szmigiero.name \
--cc=maz@kernel.org \
--cc=mic@digikod.net \
--cc=michael.roth@amd.com \
--cc=mpe@ellerman.id.au \
--cc=oliver.upton@linux.dev \
--cc=palmer@dabbelt.com \
--cc=paul.walmsley@sifive.com \
--cc=pbonzini@redhat.com \
--cc=qperret@google.com \
--cc=seanjc@google.com \
--cc=vannapurve@google.com \
--cc=vbabka@suse.cz \
--cc=viro@zeniv.linux.org.uk \
--cc=wei.w.wang@intel.com \
--cc=willy@infradead.org \
--cc=xiaoyao.li@intel.com \
--cc=yilun.xu@intel.com \
--cc=yu.c.zhang@linux.intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).