From: Kristen Carlson Accardi <kristen@linux.intel.com>
To: jarkko@kernel.org, dave.hansen@linux.kernel.org, tj@kernel.org,
linux-kernel@vger.kernel.org, linux-sgx@vger.kernel.org,
cgroups@vger.kernel.org
Cc: zhiquan1.li@intel.com
Subject: [PATCH 00/26] Add Cgroup support for SGX EPC memory
Date: Fri, 11 Nov 2022 10:35:05 -0800 [thread overview]
Message-ID: <20221111183532.3676646-1-kristen@linux.intel.com> (raw)
Utilize the Miscellaneous cgroup controller to regulate the distribution
of SGX EPC memory, which is a subset of system RAM that is used to provide
SGX-enabled applications with protected memory, and is otherwise inaccessible.
SGX EPC memory allocations are separate from normal RAM allocations,
and is managed solely by the SGX subsystem. The existing cgroup memory
controller cannot be used to limit or account for SGX EPC memory.
This patchset implements the support for sgx_epc memory within the
misc cgroup controller, and then utilizes the misc cgroup controller
to provide support for setting the total system capacity, max limit
per cgroup, and events.
This work was originally authored by Sean Christopherson a few years ago,
and was modified to work with more recent kernels, and to utilize the
misc cgroup controller rather than a custom controller. It is currently
based on top of the MCA patches.
Here's the MCA patchset for reference.
https://lore.kernel.org/linux-sgx/2d52c8c4-8ed0-6df2-2911-da5b9fcc9ae4@intel.com/T/#t
The patchset adds support for multiple LRUs to track both reclaimable
EPC pages (i.e. pages the reclaimer knows about), as well as unreclaimable
EPC pages (i.e. pages which the reclaimer isn't aware of, such as va pages).
These pages are assigned to an LRU, as well as an enclave, so that an
enclave's full EPC usage can be tracked, and limited to a max value. During
OOM events, an enclave can be have its memory zapped, and all the EPC pages
not tracked by the reclaimer can be freed.
I appreciate your comments and feedback.
Kristen Carlson Accardi (13):
x86/sgx: Add 'struct sgx_epc_lru' to encapsulate lru list(s)
x86/sgx: Use sgx_epc_lru for existing active page list
x86/sgx: Track epc pages on reclaimable or unreclaimable lists
cgroup/misc: Add notifier block list support for css events
cgroup/misc: Expose root_misc
cgroup/misc: Expose parent_misc()
cgroup/misc: allow users of misc cgroup to read specific cgroup usage
cgroup/misc: allow misc cgroup consumers to read the max value
cgroup/misc: Add private per cgroup data to struct misc_cg
cgroup/misc: Add tryget functionality for misc controller
cgroup/misc: Add SGX EPC resource type
x86/sgx: Add support for misc cgroup controller
Docs/x86/sgx: Add description for cgroup support
Sean Christopherson (13):
x86/sgx: Call cond_resched() at the end of sgx_reclaim_pages()
x86/sgx: Store struct sgx_encl when allocating new va pages
x86/sgx: Introduce RECLAIM_IN_PROGRESS flag for EPC pages
x86/sgx: Use a list to track to-be-reclaimed pages during reclaim
x86/sgx: Add EPC page flags to identify type of page
x86/sgx: Allow reclaiming up to 32 pages, but scan 16 by default
x86/sgx: Return the number of EPC pages that were successfully
reclaimed
x86/sgx: Add option to ignore age of page during EPC reclaim
x86/sgx: Add helper to retrieve SGX EPC LRU given an EPC page
x86/sgx: Prepare for multiple LRUs
x86/sgx: Expose sgx_reclaim_pages() for use by EPC cgroup
x86/sgx: Add helper to grab pages from an arbitrary EPC LRU
x86/sgx: Add EPC OOM path to forcefully reclaim EPC
Documentation/x86/sgx.rst | 77 ++++
arch/x86/Kconfig | 13 +
arch/x86/kernel/cpu/sgx/Makefile | 1 +
arch/x86/kernel/cpu/sgx/encl.c | 89 ++++-
arch/x86/kernel/cpu/sgx/encl.h | 4 +-
arch/x86/kernel/cpu/sgx/epc_cgroup.c | 561 +++++++++++++++++++++++++++
arch/x86/kernel/cpu/sgx/epc_cgroup.h | 59 +++
arch/x86/kernel/cpu/sgx/ioctl.c | 13 +-
arch/x86/kernel/cpu/sgx/main.c | 405 +++++++++++++++----
arch/x86/kernel/cpu/sgx/sgx.h | 96 ++++-
arch/x86/kernel/cpu/sgx/virt.c | 28 +-
include/linux/misc_cgroup.h | 71 ++++
kernel/cgroup/misc.c | 145 ++++++-
13 files changed, 1446 insertions(+), 116 deletions(-)
create mode 100644 arch/x86/kernel/cpu/sgx/epc_cgroup.c
create mode 100644 arch/x86/kernel/cpu/sgx/epc_cgroup.h
--
2.37.3
next reply other threads:[~2022-11-11 18:36 UTC|newest]
Thread overview: 43+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-11-11 18:35 Kristen Carlson Accardi [this message]
2022-11-11 18:35 ` [PATCH 01/26] x86/sgx: Call cond_resched() at the end of sgx_reclaim_pages() Kristen Carlson Accardi
2022-11-15 23:27 ` Jarkko Sakkinen
2022-11-16 1:00 ` Reinette Chatre
2022-11-11 18:35 ` [PATCH 02/26] x86/sgx: Store struct sgx_encl when allocating new va pages Kristen Carlson Accardi
2022-11-15 23:31 ` Jarkko Sakkinen
2022-11-11 18:35 ` [PATCH 03/26] x86/sgx: Add 'struct sgx_epc_lru' to encapsulate lru list(s) Kristen Carlson Accardi
2022-11-15 23:35 ` Jarkko Sakkinen
2022-11-11 18:35 ` [PATCH 04/26] x86/sgx: Use sgx_epc_lru for existing active page list Kristen Carlson Accardi
2022-11-11 18:35 ` [PATCH 05/26] x86/sgx: Track epc pages on reclaimable or unreclaimable lists Kristen Carlson Accardi
2022-11-11 18:35 ` [PATCH 06/26] x86/sgx: Introduce RECLAIM_IN_PROGRESS flag for EPC pages Kristen Carlson Accardi
2022-11-15 23:42 ` Jarkko Sakkinen
2022-11-11 18:35 ` [PATCH 07/26] x86/sgx: Use a list to track to-be-reclaimed pages during reclaim Kristen Carlson Accardi
2022-11-11 18:35 ` [PATCH 08/26] x86/sgx: Add EPC page flags to identify type of page Kristen Carlson Accardi
2022-11-11 18:35 ` [PATCH 09/26] x86/sgx: Allow reclaiming up to 32 pages, but scan 16 by default Kristen Carlson Accardi
2022-11-11 18:35 ` [PATCH 10/26] x86/sgx: Return the number of EPC pages that were successfully reclaimed Kristen Carlson Accardi
2022-11-11 18:35 ` [PATCH 11/26] x86/sgx: Add option to ignore age of page during EPC reclaim Kristen Carlson Accardi
2022-11-11 18:35 ` [PATCH 12/26] x86/sgx: Add helper to retrieve SGX EPC LRU given an EPC page Kristen Carlson Accardi
2022-11-11 18:35 ` [PATCH 13/26] x86/sgx: Prepare for multiple LRUs Kristen Carlson Accardi
2022-11-11 18:35 ` [PATCH 14/26] x86/sgx: Expose sgx_reclaim_pages() for use by EPC cgroup Kristen Carlson Accardi
2022-11-11 18:35 ` [PATCH 15/26] x86/sgx: Add helper to grab pages from an arbitrary EPC LRU Kristen Carlson Accardi
2022-11-11 18:35 ` [PATCH 16/26] x86/sgx: Add EPC OOM path to forcefully reclaim EPC Kristen Carlson Accardi
2022-11-11 18:35 ` [PATCH 17/26] cgroup/misc: Add notifier block list support for css events Kristen Carlson Accardi
2022-11-14 22:42 ` Tejun Heo
2022-11-14 23:10 ` Kristen Carlson Accardi
2022-11-14 23:11 ` Tejun Heo
2022-11-14 23:17 ` Kristen Carlson Accardi
2022-11-11 18:35 ` [PATCH 18/26] cgroup/misc: Expose root_misc Kristen Carlson Accardi
2022-11-14 22:19 ` Tejun Heo
2022-11-11 18:35 ` [PATCH 19/26] cgroup/misc: Expose parent_misc() Kristen Carlson Accardi
2022-11-14 22:30 ` Tejun Heo
2022-11-11 18:35 ` [PATCH 20/26] cgroup/misc: allow users of misc cgroup to read specific cgroup usage Kristen Carlson Accardi
2022-11-14 22:31 ` Tejun Heo
2022-11-11 18:35 ` [PATCH 21/26] cgroup/misc: allow misc cgroup consumers to read the max value Kristen Carlson Accardi
2022-11-14 22:33 ` Tejun Heo
2022-11-11 18:35 ` [PATCH 22/26] cgroup/misc: Add private per cgroup data to struct misc_cg Kristen Carlson Accardi
2022-11-14 22:34 ` Tejun Heo
2022-11-11 18:35 ` [PATCH 23/26] cgroup/misc: Add tryget functionality for misc controller Kristen Carlson Accardi
2022-11-11 18:35 ` [PATCH 24/26] cgroup/misc: Add SGX EPC resource type Kristen Carlson Accardi
2022-11-11 18:35 ` [PATCH 25/26] x86/sgx: Add support for misc cgroup controller Kristen Carlson Accardi
2022-11-14 22:38 ` Tejun Heo
2022-11-11 18:35 ` [PATCH 26/26] Docs/x86/sgx: Add description for cgroup support Kristen Carlson Accardi
2022-11-12 9:28 ` Bagas Sanjaya
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20221111183532.3676646-1-kristen@linux.intel.com \
--to=kristen@linux.intel.com \
--cc=cgroups@vger.kernel.org \
--cc=dave.hansen@linux.kernel.org \
--cc=jarkko@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-sgx@vger.kernel.org \
--cc=tj@kernel.org \
--cc=zhiquan1.li@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).