From: Xu Yilun <yilun.xu@intel.com>
To: isaku.yamahata@intel.com
Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org,
isaku.yamahata@gmail.com, Michael Roth <michael.roth@amd.com>,
Paolo Bonzini <pbonzini@redhat.com>,
Sean Christopherson <seanjc@google.com>,
erdemaktas@google.com, Sagi Shahar <sagis@google.com>,
David Matlack <dmatlack@google.com>,
Kai Huang <kai.huang@intel.com>,
Zhi Wang <zhi.wang.linux@gmail.com>,
chen.bo@intel.com, linux-coco@lists.linux.dev,
Chao Peng <chao.p.peng@linux.intel.com>,
Ackerley Tng <ackerleytng@google.com>,
Vishal Annapurve <vannapurve@google.com>,
Yuan Yao <yuan.yao@linux.intel.com>,
Jarkko Sakkinen <jarkko@kernel.org>,
Quentin Perret <qperret@google.com>,
wei.w.wang@intel.com, Fuad Tabba <tabba@google.com>
Subject: Re: [PATCH 6/8] KVM: gmem, x86: Add gmem hook for invalidating private memory
Date: Tue, 10 Oct 2023 17:17:36 +0800 [thread overview]
Message-ID: <ZSUWsK8dGPjlrCR1@yilunxu-OptiPlex-7050> (raw)
In-Reply-To: <8c9f0470ba6e5dc122f3f4e37c4dcfb6fb97b184.1692119201.git.isaku.yamahata@intel.com>
On 2023-08-15 at 10:18:53 -0700, isaku.yamahata@intel.com wrote:
> From: Michael Roth <michael.roth@amd.com>
>
> TODO: add a CONFIG option that can be to completely skip arch
> invalidation loop and avoid __weak references for arch/platforms that
> don't need an additional invalidation hook.
>
> In some cases, like with SEV-SNP, guest memory needs to be updated in a
> platform-specific manner before it can be safely freed back to the host.
> Add hooks to wire up handling of this sort when freeing memory in
> response to FALLOC_FL_PUNCH_HOLE operations.
>
> Also issue invalidations of all allocated pages when releasing the gmem
> file so that the pages are not left in an unusable state when they get
> freed back to the host.
>
> Signed-off-by: Michael Roth <michael.roth@amd.com>
> Link: https://lore.kernel.org/r/20230612042559.375660-3-michael.roth@amd.com
>
[...]
> +/* Handle arch-specific hooks needed before releasing guarded pages. */
> +static void kvm_gmem_issue_arch_invalidate(struct kvm *kvm, struct file *file,
> + pgoff_t start, pgoff_t end)
> +{
> + pgoff_t file_end = i_size_read(file_inode(file)) >> PAGE_SHIFT;
> + pgoff_t index = start;
> +
> + end = min(end, file_end);
> +
> + while (index < end) {
> + struct folio *folio;
> + unsigned int order;
> + struct page *page;
> + kvm_pfn_t pfn;
> +
> + folio = __filemap_get_folio(file->f_mapping, index,
> + FGP_LOCK, 0);
> + if (!folio) {
> + index++;
> + continue;
> + }
> +
> + page = folio_file_page(folio, index);
> + pfn = page_to_pfn(page);
> + order = folio_order(folio);
> +
> + kvm_arch_gmem_invalidate(kvm, pfn, pfn + min((1ul << order), end - index));
Observed an issue there.
The valid page may not point to the first page in the folio, then the
range [pfn, pfn + (1ul << order)) expands to the next folio. This makes
a part of the pages be invalidated again when loop to the next folio.
On TDX, it causes TDH_PHYMEM_PAGE_WBINVD failed.
> +
> + index = folio_next_index(folio);
> + folio_unlock(folio);
> + folio_put(folio);
> +
> + cond_resched();
> + }
> +}
My fix would be:
diff --git a/virt/kvm/guest_mem.c b/virt/kvm/guest_mem.c
index e629782d73d5..3665003c3746 100644
--- a/virt/kvm/guest_mem.c
+++ b/virt/kvm/guest_mem.c
@@ -155,7 +155,7 @@ static void kvm_gmem_issue_arch_invalidate(struct kvm *kvm, struct inode *inode,
while (index < end) {
struct folio *folio;
- unsigned int order;
+ pgoff_t ntails;
struct page *page;
kvm_pfn_t pfn;
@@ -168,9 +168,9 @@ static void kvm_gmem_issue_arch_invalidate(struct kvm *kvm, struct inode *inode,
page = folio_file_page(folio, index);
pfn = page_to_pfn(page);
- order = folio_order(folio);
+ ntails = folio_nr_pages(folio) - folio_page_idx(folio, page);
- kvm_arch_gmem_invalidate(kvm, pfn, pfn + min((1ul << order), end - index));
+ kvm_arch_gmem_invalidate(kvm, pfn, pfn + min(ntails, end - index));
index = folio_next_index(folio);
folio_unlock(folio);
Thanks,
Yilun
next prev parent reply other threads:[~2023-10-10 9:18 UTC|newest]
Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-08-15 17:18 [PATCH 0/8] KVM: gmem: Adding hooks for SEV and TDX isaku.yamahata
2023-08-15 17:18 ` [PATCH 1/8] KVM: gmem: Make kvm_gmem_bind return EBADF on wrong fd isaku.yamahata
2023-08-15 17:18 ` [PATCH 2/8] KVM: gmem: removed duplicated kvm_gmem_init() isaku.yamahata
2023-08-15 17:18 ` [PATCH 3/8] KVM: gmem: Fix kvm_gmem_issue_arch_invalidate() isaku.yamahata
2023-08-18 22:33 ` Sean Christopherson
2023-08-15 17:18 ` [PATCH 4/8] KVM: gmem: protect kvm_mmu_invalidate_end() isaku.yamahata
2023-08-16 20:28 ` Jarkko Sakkinen
2023-08-18 17:55 ` Sean Christopherson
2023-08-18 20:32 ` Kalra, Ashish
2023-08-18 22:44 ` Sean Christopherson
2023-08-19 2:08 ` Mingwei Zhang
2023-08-21 14:42 ` Sean Christopherson
2023-08-21 21:44 ` Kalra, Ashish
2023-08-22 22:30 ` Kalra, Ashish
2023-08-22 23:17 ` Sean Christopherson
2023-08-31 16:50 ` Kalra, Ashish
2023-08-15 17:18 ` [PATCH 5/8] KVM: gmem, x86: Add gmem hook for initializing private memory isaku.yamahata
2023-08-16 20:30 ` Jarkko Sakkinen
2023-08-15 17:18 ` [PATCH 6/8] KVM: gmem, x86: Add gmem hook for invalidating " isaku.yamahata
2023-08-16 0:42 ` kernel test robot
2023-08-16 20:37 ` Isaku Yamahata
2023-10-10 9:17 ` Xu Yilun [this message]
2023-08-15 17:18 ` [PATCH 7/8] KVM: gmem: Avoid race with kvm_gmem_release and mmu notifier isaku.yamahata
2023-08-18 18:15 ` Sean Christopherson
2023-08-15 17:18 ` [PATCH 8/8] RFC: KVM: gmem: Guarantee the order of destruction isaku.yamahata
2023-08-18 23:14 ` [PATCH 0/8] KVM: gmem: Adding hooks for SEV and TDX Sean Christopherson
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ZSUWsK8dGPjlrCR1@yilunxu-OptiPlex-7050 \
--to=yilun.xu@intel.com \
--cc=ackerleytng@google.com \
--cc=chao.p.peng@linux.intel.com \
--cc=chen.bo@intel.com \
--cc=dmatlack@google.com \
--cc=erdemaktas@google.com \
--cc=isaku.yamahata@gmail.com \
--cc=isaku.yamahata@intel.com \
--cc=jarkko@kernel.org \
--cc=kai.huang@intel.com \
--cc=kvm@vger.kernel.org \
--cc=linux-coco@lists.linux.dev \
--cc=linux-kernel@vger.kernel.org \
--cc=michael.roth@amd.com \
--cc=pbonzini@redhat.com \
--cc=qperret@google.com \
--cc=sagis@google.com \
--cc=seanjc@google.com \
--cc=tabba@google.com \
--cc=vannapurve@google.com \
--cc=wei.w.wang@intel.com \
--cc=yuan.yao@linux.intel.com \
--cc=zhi.wang.linux@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).