From: "Haitao Huang" <haitao.huang@linux.intel.com>
To: jarkko@kernel.org, dave.hansen@linux.intel.com, tj@kernel.org,
mkoutny@suse.com, linux-kernel@vger.kernel.org,
linux-sgx@vger.kernel.org, x86@kernel.org,
cgroups@vger.kernel.org, tglx@linutronix.de, mingo@redhat.com,
bp@alien8.de, hpa@zytor.com, sohil.mehta@intel.com,
tim.c.chen@linux.intel.com, "Huang, Kai" <kai.huang@intel.com>
Cc: zhiquan1.li@intel.com, kristen@linux.intel.com,
seanjc@google.com, zhanb@microsoft.com, anakrish@microsoft.com,
mikko.ylinen@linux.intel.com, yangjie@microsoft.com,
chrisyan@microsoft.com
Subject: Re: [PATCH v13 12/14] x86/sgx: Turn on per-cgroup EPC reclamation
Date: Mon, 06 May 2024 22:21:39 -0500 [thread overview]
Message-ID: <op.2ndsydgywjvjmi@hhuan26-mobl.amr.corp.intel.com> (raw)
In-Reply-To: <966e4afe-b177-4527-80c9-1146d2503c93@intel.com>
On Mon, 06 May 2024 19:10:42 -0500, Huang, Kai <kai.huang@intel.com> wrote:
>
>
> On 1/05/2024 7:51 am, Haitao Huang wrote:
>> static void sgx_reclaim_pages_global(struct mm_struct *charge_mm)
>> {
>> - sgx_reclaim_pages(&sgx_global_lru, charge_mm);
>> + if (IS_ENABLED(CONFIG_CGROUP_MISC))
>> + sgx_cgroup_reclaim_pages(misc_cg_root(), charge_mm);
>> + else
>> + sgx_reclaim_pages(&sgx_global_lru, charge_mm);
>> }
>>
>
> I think we have a problem here when we do global reclaim starting from
> the ROOT cgroup:
>
> This function will mostly just only try to reclaim from the ROOT cgroup,
> but won't reclaim from the descendants.
>
> The reason is the sgx_cgroup_reclaim_pages() will simply return after
> "scanning" SGX_NR_TO_SCAN (16) pages w/o going to the descendants, and
> the "scanning" here simply means "removing the EPC page from the
> cgroup's LRU list".
>
> So as long as the ROOT cgroup LRU contains more than SGX_NR_TO_SCAN (16)
> pages, effectively sgx_cgroup_reclaim_pages() will just scan and return
> w/o going into the descendants. Having 16 EPC pages should be a "almost
> always true" case I suppose.
>
> When the sgx_reclaim_pages_global() is called again, we will start from
> the ROOT again.
>
> That means the this doesn't truly reclaim "from global" at all.
>
> IMHO the behaviour of sgx_cgroup_reclaim_pages() is OK for per-cgroup
> reclaim because I think in this case our intention is we should try best
> to reclaim from the cgroup, i.e., whether we can reclaim from
> descendants doesn't matter.
>
> But for global reclaim this doesn't work.
>
> Am I missing anything?
>
Good catch. This is indeed a problem if pages in a higher level cgroup are
always busy (being 'young').The reclamation loop starting from this group
may be stuck in only shifting the pages from front to tail in this group
and never tries to scan & reclaim pages in its descendants.
Though this may not happen often, I think it does require a fix. Will do
it in v14 :-)
Thanks
Haitao
next prev parent reply other threads:[~2024-05-07 3:21 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-04-30 19:50 [PATCH v13 00/14] Add Cgroup support for SGX EPC memory Haitao Huang
2024-04-30 19:50 ` [PATCH v13 01/14] x86/sgx: Replace boolean parameters with enums Haitao Huang
2024-04-30 19:50 ` [PATCH v13 02/14] cgroup/misc: Add per resource callbacks for CSS events Haitao Huang
2024-04-30 19:50 ` [PATCH v13 03/14] cgroup/misc: Export APIs for SGX driver Haitao Huang
2024-04-30 19:50 ` [PATCH v13 04/14] cgroup/misc: Add SGX EPC resource type Haitao Huang
2024-04-30 19:50 ` [PATCH v13 05/14] x86/sgx: Implement basic EPC misc cgroup functionality Haitao Huang
2024-04-30 19:51 ` [PATCH v13 06/14] x86/sgx: Add sgx_epc_lru_list to encapsulate LRU list Haitao Huang
2024-04-30 19:51 ` [PATCH v13 07/14] x86/sgx: Abstract tracking reclaimable pages in LRU Haitao Huang
2024-04-30 19:51 ` [PATCH v13 08/14] x86/sgx: Add basic EPC reclamation flow for cgroup Haitao Huang
2024-04-30 19:51 ` [PATCH v13 09/14] x86/sgx: Implement async reclamation " Haitao Huang
2024-04-30 19:51 ` [PATCH v13 10/14] x86/sgx: Charge mem_cgroup for per-cgroup reclamation Haitao Huang
2024-04-30 19:51 ` [PATCH v13 11/14] x86/sgx: Abstract check for global reclaimable pages Haitao Huang
2024-05-02 23:23 ` Huang, Kai
2024-05-06 18:00 ` Haitao Huang
2024-04-30 19:51 ` [PATCH v13 12/14] x86/sgx: Turn on per-cgroup EPC reclamation Haitao Huang
2024-05-07 0:10 ` Huang, Kai
2024-05-07 3:21 ` Haitao Huang [this message]
2024-04-30 19:51 ` [PATCH v13 13/14] Docs/x86/sgx: Add description for cgroup support Haitao Huang
2024-04-30 19:51 ` [PATCH v13 14/14] selftests/sgx: Add scripts for EPC cgroup testing Haitao Huang
2024-05-03 1:44 ` Jarkko Sakkinen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=op.2ndsydgywjvjmi@hhuan26-mobl.amr.corp.intel.com \
--to=haitao.huang@linux.intel.com \
--cc=anakrish@microsoft.com \
--cc=bp@alien8.de \
--cc=cgroups@vger.kernel.org \
--cc=chrisyan@microsoft.com \
--cc=dave.hansen@linux.intel.com \
--cc=hpa@zytor.com \
--cc=jarkko@kernel.org \
--cc=kai.huang@intel.com \
--cc=kristen@linux.intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-sgx@vger.kernel.org \
--cc=mikko.ylinen@linux.intel.com \
--cc=mingo@redhat.com \
--cc=mkoutny@suse.com \
--cc=seanjc@google.com \
--cc=sohil.mehta@intel.com \
--cc=tglx@linutronix.de \
--cc=tim.c.chen@linux.intel.com \
--cc=tj@kernel.org \
--cc=x86@kernel.org \
--cc=yangjie@microsoft.com \
--cc=zhanb@microsoft.com \
--cc=zhiquan1.li@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).