From: Elias El Yandouzi <eliasely@amazon.com>
To: <xen-devel@lists.xenproject.org>
Cc: <julien@xen.org>, <pdurrant@amazon.com>, <dwmw@amazon.com>,
"Elias El Yandouzi" <eliasely@amazon.com>
Subject: [PATCH V3 00/19] Remove the directmap
Date: Mon, 13 May 2024 11:10:58 +0000 [thread overview]
Message-ID: <20240513111117.68828-1-eliasely@amazon.com> (raw)
Hi all,
A few years ago, Wei Liu implemented a PoC to remove the directmap
from Xen. The last version was sent by Hongyan Xia [1].
I will start with thanking both Wei and Hongyan for the initial work
to upstream the feature. A lot of patches already went in and this is
the last few patches missing to effectively enable the feature.
=== What is the directmap? ===
At the moment, on both arm64 and x86, most of the RAM is mapped
in Xen address space. This means that domain memory is easily
accessible in Xen.
=== Why do we want to remove the directmap? ===
(Summarizing my understanding of the previous discussion)
Speculation attacks (like Spectre SP1) rely on loading piece of memory
in the cache. If the region is not mapped then it can't be loaded.
So removing reducing the amount of memory mapped in Xen will also
reduce the surface attack.
=== What's the performance impact? ===
As the guest memory is not always mapped, then the cost of mapping
will increase. I haven't done the numbers with this new version, but
some measurement were provided in the previous version for x86.
=== Improvement possible ===
The known area to improve on x86 are:
* Mapcache: There was a patch sent by Hongyan:
https://lore.kernel.org/xen-devel/4058e92ce21627731c49b588a95809dc0affd83a.1581015491.git.hongyxia@amazon.com/
* EPT: At the moment an guest page-tabel walk requires about 20 map/unmap.
This will have an very high impact on the performance. We need to decide
whether keep the EPT always mapped is a problem
The original series didn't have support for Arm64. But as there were
some interest, I have provided a PoC.
There are more extra work for Arm64:
* The mapcache is quite simple. We would investigate the performance
* The mapcache should be made compliant to the Arm Arm (this is now
more critical).
* We will likely have the same problem as for the EPT.
* We have no support for merging table to a superpage, neither
free empty page-tables. (See more below)
=== Implementation ===
The subject is probably a misnomer. The directmap is still present but
the RAM is not mapped by default. Instead, the region will still be used
to map pages allocate via alloc_xenheap_pages().
The advantage is the solution is simple (so IHMO good enough for been
merged as a tech preview). The disadvantage is the page allocator is not
trying to keep all the xenheap pages together. So we may end up to have
an increase of page table usage.
In the longer term, we should consider to remove the direct map
completely and switch to vmap(). The main problem with this approach
is it is frequent to use mfn_to_virt() in the code. So we would need
to cache the mapping (maybe in the struct page_info).
=== Why arm32 is not covered? ===
On Arm32, the domheap and xenheap is always separated. So by design
the guest memory is not mapped by default.
At this stage, it seems unnecessary to have to map/unmap xenheap pages
every time they are allocated.
=== Why not using a separate domheap and xenheap? ===
While a separate xenheap/domheap reduce the page-table usage (all
xenheap pages are contiguous and could be always mapped), it is also
currently less scalable because the split is fixed at boot time (XXX:
Can this be dynamic?).
=== Future of secret-free hypervisor ===
There are some information in an e-mail from Andrew a few years ago:
https://lore.kernel.org/xen-devel/e3219697-0759-39fc-2486-715cdec1ca9e@citrix.com/
Cheers,
[1] https://lore.kernel.org/xen-devel/cover.1588278317.git.hongyxia@amazon.com/
*** BLURB HERE ***
Elias El Yandouzi (3):
xen/x86: Add build assertion for fixmap entries
Rename mfn_to_virt() calls
Rename maddr_to_virt() calls
Hongyan Xia (9):
x86: Create per-domain mapping of guest_root_pt
x86/pv: Rewrite how building PV dom0 handles domheap mappings
x86/mapcache: Initialise the mapcache for the idle domain
x86: Add a boot option to enable and disable the direct map
x86/domain_page: Remove the fast paths when mfn is not in the
directmap
xen/page_alloc: Add a path for xenheap when there is no direct map
x86/setup: Leave early boot slightly earlier
x86/setup: vmap heap nodes when they are outside the direct map
x86/setup: Do not create valid mappings when directmap=no
Julien Grall (5):
xen/x86: Add support for the PMAP
xen/arm32: mm: Rename 'first' to 'root' in init_secondary_pagetables()
xen/arm64: mm: Use per-pCPU page-tables
xen/arm64: Implement a mapcache for arm64
xen/arm64: Allow the admin to enable/disable the directmap
Wei Liu (2):
x86/pv: Domheap pages should be mapped while relocating initrd
x86: Lift mapcache variable to the arch level
docs/misc/xen-command-line.pandoc | 12 +++
xen/arch/arm/Kconfig | 2 +-
xen/arch/arm/arm64/mmu/mm.c | 45 ++++++++-
xen/arch/arm/domain_page.c | 50 +++++++++-
xen/arch/arm/include/asm/arm32/mm.h | 8 --
xen/arch/arm/include/asm/arm64/mm.h | 7 +-
xen/arch/arm/include/asm/domain_page.h | 13 +++
xen/arch/arm/include/asm/mm.h | 9 ++
xen/arch/arm/include/asm/mmu/layout.h | 13 ++-
xen/arch/arm/include/asm/mmu/mm.h | 2 +
xen/arch/arm/mm.c | 1 +
xen/arch/arm/mmu/pt.c | 12 +--
xen/arch/arm/mmu/setup.c | 27 ++---
xen/arch/arm/mmu/smpboot.c | 30 ++----
xen/arch/arm/setup.c | 2 +
xen/arch/x86/Kconfig | 2 +
xen/arch/x86/dmi_scan.c | 4 +-
xen/arch/x86/domain.c | 12 ++-
xen/arch/x86/domain_page.c | 74 +++++++++-----
xen/arch/x86/hvm/dom0_build.c | 4 +-
xen/arch/x86/include/asm/config.h | 10 +-
xen/arch/x86/include/asm/domain.h | 13 +--
xen/arch/x86/include/asm/fixmap.h | 9 ++
.../x86/include/asm/mach-default/bios_ebda.h | 2 +-
xen/arch/x86/include/asm/mm.h | 6 ++
xen/arch/x86/include/asm/page.h | 8 +-
xen/arch/x86/include/asm/pmap.h | 25 +++++
xen/arch/x86/include/asm/x86_64/page.h | 2 +-
xen/arch/x86/mm.c | 18 +++-
xen/arch/x86/mpparse.c | 2 +-
xen/arch/x86/pv/dom0_build.c | 70 +++++++++----
xen/arch/x86/pv/domain.c | 36 +++++++
xen/arch/x86/setup.c | 98 ++++++++++++++++---
xen/arch/x86/tboot.c | 2 +-
xen/arch/x86/x86_64/asm-offsets.c | 1 +
xen/arch/x86/x86_64/entry.S | 8 ++
xen/arch/x86/x86_64/mm.c | 26 +++--
xen/common/Kconfig | 17 ++++
xen/common/efi/boot.c | 23 +++--
xen/common/page_alloc.c | 89 ++++++++++++++---
xen/common/trace.c | 8 +-
xen/include/xen/mm.h | 7 ++
42 files changed, 630 insertions(+), 179 deletions(-)
create mode 100644 xen/arch/arm/include/asm/domain_page.h
create mode 100644 xen/arch/x86/include/asm/pmap.h
--
2.40.1
next reply other threads:[~2024-05-13 11:11 UTC|newest]
Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-05-13 11:10 Elias El Yandouzi [this message]
2024-05-13 11:10 ` [PATCH V3 01/19] x86: Create per-domain mapping of guest_root_pt Elias El Yandouzi
2024-05-13 15:27 ` Roger Pau Monné
2024-05-14 8:03 ` Jan Beulich
2024-05-14 15:46 ` Alejandro Vallejo
2024-05-14 17:15 ` Elias El Yandouzi
2024-05-15 9:05 ` Roger Pau Monné
2024-05-13 11:11 ` [PATCH V3 02/19] x86/pv: Domheap pages should be mapped while relocating initrd Elias El Yandouzi
2024-05-13 11:11 ` [PATCH V3 03/19] x86/pv: Rewrite how building PV dom0 handles domheap mappings Elias El Yandouzi
2024-05-13 11:11 ` [PATCH V3 04/19] x86: Lift mapcache variable to the arch level Elias El Yandouzi
2024-05-13 11:11 ` [PATCH V3 05/19] x86/mapcache: Initialise the mapcache for the idle domain Elias El Yandouzi
2024-05-13 11:11 ` [PATCH V3 06/19] x86: Add a boot option to enable and disable the direct map Elias El Yandouzi
2024-05-13 11:11 ` [PATCH V3 07/19] xen/x86: Add support for the PMAP Elias El Yandouzi
2024-05-13 11:11 ` [PATCH V3 08/19] xen/x86: Add build assertion for fixmap entries Elias El Yandouzi
2024-05-13 11:11 ` [PATCH V3 09/19] x86/domain_page: Remove the fast paths when mfn is not in the directmap Elias El Yandouzi
2024-05-13 11:11 ` [PATCH V3 10/19] xen/page_alloc: Add a path for xenheap when there is no direct map Elias El Yandouzi
2024-05-13 11:11 ` [PATCH V3 11/19] x86/setup: Leave early boot slightly earlier Elias El Yandouzi
2024-05-13 11:11 ` [PATCH V3 12/19] x86/setup: vmap heap nodes when they are outside the direct map Elias El Yandouzi
2024-05-13 11:11 ` [PATCH V3 13/19] x86/setup: Do not create valid mappings when directmap=no Elias El Yandouzi
2024-05-13 11:11 ` [PATCH V3 14/19] Rename mfn_to_virt() calls Elias El Yandouzi
2024-05-13 11:11 ` [PATCH V3 15/19] Rename maddr_to_virt() calls Elias El Yandouzi
2024-05-13 11:11 ` [PATCH V3 16/19] xen/arm32: mm: Rename 'first' to 'root' in init_secondary_pagetables() Elias El Yandouzi
2024-05-13 11:11 ` [PATCH V3 17/19] xen/arm64: mm: Use per-pCPU page-tables Elias El Yandouzi
2024-05-13 11:11 ` [PATCH V3 18/19] xen/arm64: Implement a mapcache for arm64 Elias El Yandouzi
2024-05-13 11:11 ` [PATCH V3 19/19] xen/arm64: Allow the admin to enable/disable the directmap Elias El Yandouzi
2024-05-13 12:52 ` [PATCH V3 00/19] Remove " Roger Pau Monné
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240513111117.68828-1-eliasely@amazon.com \
--to=eliasely@amazon.com \
--cc=dwmw@amazon.com \
--cc=julien@xen.org \
--cc=pdurrant@amazon.com \
--cc=xen-devel@lists.xenproject.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).