From: Baoquan He <bhe@redhat.com>
To: Joerg Roedel <joro@8bytes.org>, Tom Lendacky <thomas.lendacky@amd.com>
Cc: x86@kernel.org, Joerg Roedel <jroedel@suse.de>,
Eric Biederman <ebiederm@xmission.com>,
kexec@lists.infradead.org, hpa@zytor.com,
Andy Lutomirski <luto@kernel.org>,
Dave Hansen <dave.hansen@linux.intel.com>,
Peter Zijlstra <peterz@infradead.org>,
Jiri Slaby <jslaby@suse.cz>,
Dan Williams <dan.j.williams@intel.com>,
Juergen Gross <jgross@suse.com>,
Kees Cook <keescook@chromium.org>,
David Rientjes <rientjes@google.com>,
Cfir Cohen <cfir@google.com>, Erdem Aktas <erdemaktas@google.com>,
Masami Hiramatsu <mhiramat@kernel.org>,
Mike Stunes <mstunes@vmware.com>,
Sean Christopherson <seanjc@google.com>,
Martin Radev <martin.b.radev@gmail.com>,
Arvind Sankar <nivedita@alum.mit.edu>,
linux-coco@lists.linux.dev, linux-kernel@vger.kernel.org,
kvm@vger.kernel.org, virtualization@lists.linux-foundation.org
Subject: Re: [PATCH v3 00/10] x86/sev: KEXEC/KDUMP support for SEV-ES guests
Date: Sun, 4 Jun 2023 21:07:39 +0800 [thread overview]
Message-ID: <ZHyMm0iVyB2EFIsJ@MiWiFi-R3L-srv> (raw)
In-Reply-To: <20220127101044.13803-1-joro@8bytes.org>
Hi Joerg and Tom,
On 01/27/22 at 11:10am, Joerg Roedel wrote:
> From: Joerg Roedel <jroedel@suse.de>
>
> Hi,
>
> here are changes to enable kexec/kdump in SEV-ES guests. The biggest
> problem for supporting kexec/kdump under SEV-ES is to find a way to
> hand the non-boot CPUs (APs) from one kernel to another.
>
> Without SEV-ES the first kernel parks the CPUs in a HLT loop until
> they get reset by the kexec'ed kernel via an INIT-SIPI-SIPI sequence.
> For virtual machines the CPU reset is emulated by the hypervisor,
> which sets the vCPU registers back to reset state.
Is there any plan for this patchset to proceed? Without this patchset,
it does fail kexec and kdump with multiple cpus.
Thanks
Baoquan
>
> This does not work under SEV-ES, because the hypervisor has no access
> to the vCPU registers and can't make modifications to them. So an
> SEV-ES guest needs to reset the vCPU itself and park it using the
> AP-reset-hold protocol. Upon wakeup the guest needs to jump to
> real-mode and to the reset-vector configured in the AP-Jump-Table.
>
> The code to do this is the main part of this patch-set. It works by
> placing code on the AP Jump-Table page itself to park the vCPU and for
> jumping to the reset vector upon wakeup. The code on the AP Jump Table
> runs in 16-bit protected mode with segment base set to the beginning
> of the page. The AP Jump-Table is usually not within the first 1MB of
> memory, so the code can't run in real-mode.
>
> The AP Jump-Table is the best place to put the parking code, because
> the memory is owned, but read-only by the firmware and writeable by
> the OS. Only the first 4 bytes are used for the reset-vector, leaving
> the rest of the page for code/data/stack to park a vCPU. The code
> can't be in kernel memory because by the time the vCPU wakes up the
> memory will be owned by the new kernel, which might have overwritten it
> already.
>
> The other patches add initial GHCB Version 2 protocol support, because
> kexec/kdump need the MSR-based (without a GHCB) AP-reset-hold VMGEXIT,
> which is a GHCB protocol version 2 feature.
>
> The kexec'ed kernel is also entered via the decompressor and needs
> MMIO support there, so this patch-set also adds MMIO #VC support to
> the decompressor and support for handling CLFLUSH instructions.
>
> Finally there is also code to disable kexec/kdump support at runtime
> when the environment does not support it (e.g. no GHCB protocol
> version 2 support or AP Jump Table over 4GB).
>
> The diffstat looks big, but most of it is moving code for MMIO #VC
> support around to make it available to the decompressor.
>
> The previous version of this patch-set can be found here:
>
> https://lore.kernel.org/lkml/20210913155603.28383-1-joro@8bytes.org/
>
> Please review.
>
> Thanks,
>
> Joerg
>
> Changes v2->v3:
>
> - Rebased to v5.17-rc1
> - Applied most review comments by Boris
> - Use the name 'AP jump table' consistently
> - Make kexec-disabling for unsupported guests x86-specific
> - Cleanup and consolidate patches to detect GHCB v2 protocol
> support
>
> Joerg Roedel (10):
> x86/kexec/64: Disable kexec when SEV-ES is active
> x86/sev: Save and print negotiated GHCB protocol version
> x86/sev: Set GHCB data structure version
> x86/sev: Cache AP Jump Table Address
> x86/sev: Setup code to park APs in the AP Jump Table
> x86/sev: Park APs on AP Jump Table with GHCB protocol version 2
> x86/sev: Use AP Jump Table blob to stop CPU
> x86/sev: Add MMIO handling support to boot/compressed/ code
> x86/sev: Handle CLFLUSH MMIO events
> x86/kexec/64: Support kexec under SEV-ES with AP Jump Table Blob
>
> arch/x86/boot/compressed/sev.c | 45 +-
> arch/x86/include/asm/insn-eval.h | 1 +
> arch/x86/include/asm/realmode.h | 5 +
> arch/x86/include/asm/sev-ap-jumptable.h | 29 +
> arch/x86/include/asm/sev.h | 11 +-
> arch/x86/kernel/machine_kexec_64.c | 12 +
> arch/x86/kernel/process.c | 8 +
> arch/x86/kernel/sev-shared.c | 233 +++++-
> arch/x86/kernel/sev.c | 404 +++++------
> arch/x86/lib/insn-eval-shared.c | 913 ++++++++++++++++++++++++
> arch/x86/lib/insn-eval.c | 909 +----------------------
> arch/x86/realmode/Makefile | 9 +-
> arch/x86/realmode/rm/Makefile | 11 +-
> arch/x86/realmode/rm/header.S | 3 +
> arch/x86/realmode/rm/sev.S | 85 +++
> arch/x86/realmode/rmpiggy.S | 6 +
> arch/x86/realmode/sev/Makefile | 33 +
> arch/x86/realmode/sev/ap_jump_table.S | 131 ++++
> arch/x86/realmode/sev/ap_jump_table.lds | 24 +
> 19 files changed, 1730 insertions(+), 1142 deletions(-)
> create mode 100644 arch/x86/include/asm/sev-ap-jumptable.h
> create mode 100644 arch/x86/lib/insn-eval-shared.c
> create mode 100644 arch/x86/realmode/rm/sev.S
> create mode 100644 arch/x86/realmode/sev/Makefile
> create mode 100644 arch/x86/realmode/sev/ap_jump_table.S
> create mode 100644 arch/x86/realmode/sev/ap_jump_table.lds
>
>
> base-commit: e783362eb54cd99b2cac8b3a9aeac942e6f6ac07
> --
> 2.34.1
>
prev parent reply other threads:[~2023-06-04 13:07 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-01-27 10:10 [PATCH v3 00/10] x86/sev: KEXEC/KDUMP support for SEV-ES guests Joerg Roedel
2022-01-27 10:10 ` [PATCH v3 01/10] x86/kexec/64: Disable kexec when SEV-ES is active Joerg Roedel
2022-01-27 10:10 ` [PATCH v3 02/10] x86/sev: Save and print negotiated GHCB protocol version Joerg Roedel
2022-01-27 10:10 ` [PATCH v3 03/10] x86/sev: Set GHCB data structure version Joerg Roedel
2022-01-27 10:10 ` [PATCH v3 04/10] x86/sev: Cache AP Jump Table Address Joerg Roedel
2022-02-07 22:03 ` Sean Christopherson
2022-01-27 10:10 ` [PATCH v3 05/10] x86/sev: Setup code to park APs in the AP Jump Table Joerg Roedel
2022-02-07 22:11 ` Sean Christopherson
2022-01-27 10:10 ` [PATCH v3 06/10] x86/sev: Park APs on AP Jump Table with GHCB protocol version 2 Joerg Roedel
2022-01-27 10:10 ` [PATCH v3 07/10] x86/sev: Use AP Jump Table blob to stop CPU Joerg Roedel
2022-01-27 10:10 ` [PATCH v3 08/10] x86/sev: Add MMIO handling support to boot/compressed/ code Joerg Roedel
2022-01-27 10:10 ` [PATCH v3 09/10] x86/sev: Handle CLFLUSH MMIO events Joerg Roedel
2022-01-27 10:10 ` [PATCH v3 10/10] x86/kexec/64: Support kexec under SEV-ES with AP Jump Table Blob Joerg Roedel
2022-04-29 9:06 ` [PATCH v3 00/10] x86/sev: KEXEC/KDUMP support for SEV-ES guests Tao Liu
2022-04-29 13:08 ` Tom Lendacky
2022-07-29 10:28 ` Tao Liu
2023-06-04 13:07 ` Baoquan He [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ZHyMm0iVyB2EFIsJ@MiWiFi-R3L-srv \
--to=bhe@redhat.com \
--cc=cfir@google.com \
--cc=dan.j.williams@intel.com \
--cc=dave.hansen@linux.intel.com \
--cc=ebiederm@xmission.com \
--cc=erdemaktas@google.com \
--cc=hpa@zytor.com \
--cc=jgross@suse.com \
--cc=joro@8bytes.org \
--cc=jroedel@suse.de \
--cc=jslaby@suse.cz \
--cc=keescook@chromium.org \
--cc=kexec@lists.infradead.org \
--cc=kvm@vger.kernel.org \
--cc=linux-coco@lists.linux.dev \
--cc=linux-kernel@vger.kernel.org \
--cc=luto@kernel.org \
--cc=martin.b.radev@gmail.com \
--cc=mhiramat@kernel.org \
--cc=mstunes@vmware.com \
--cc=nivedita@alum.mit.edu \
--cc=peterz@infradead.org \
--cc=rientjes@google.com \
--cc=seanjc@google.com \
--cc=thomas.lendacky@amd.com \
--cc=virtualization@lists.linux-foundation.org \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).