From: "Christoph Lameter (Ampere)" <cl@linux.com>
To: Khalid Aziz <khalid.aziz@oracle.com>
Cc: lsf-pc@lists.linux-foundation.org,
"linux-mm@kvack.org" <linux-mm@kvack.org>
Subject: Re: [LSF/MM/BPF TOPIC] Sharing page tables across processes (mshare)
Date: Tue, 14 May 2024 11:21:37 -0700 (PDT) [thread overview]
Message-ID: <1ee0e6e0-f82a-5c90-0bca-5c5b93aeadc9@linux.com> (raw)
In-Reply-To: <ac4411f4-5faa-4df7-8d19-b2f62e10e0c9@oracle.com>
> 1. Amount of memory required for PTEs to map physical pages stays low
> even when large number of threads share the same pages since PTEs are
> shared across threads.
>
> 2. Page protection attributes are shared across threads and a change
> of attributes applies immediately to every thread without any overhead
> of coordinating protection bit changes across threads.
>
> These advantages no longer apply when unrelated processes share pages.
> Large database applications can easily comprise of 1000s of processes
> that share 100s of GB of pages. In cases like this, amount of memory
> consumed by page tables can exceed the size of actual shared data.
> On a database server with 300GB SGA, a system crash was seen with
> out-of-memory condition when 1500+ clients tried to share this SGA even
> though the system had 512GB of memory. On this server, in the worst case
> scenario of all 1500 processes mapping every page from SGA would have
> required 878GB+ for just the PTEs.
Ok then use 1Gig pages or higher for a shared mapping of huge pages. I am
not sure why there is a need for sharing page tables here. I just listened
to your talk at the LSF/MM and noted some things.
It may be best to follow established shared memory approaches like for
example implemented already in shmem.
If you want to do it with actually sharing page table semantics then the
proper implementation using shmem would be maybe to add an additional
flag. Lets call this O_SHARED_PAGE_TABLE for now.
Then you would do
fd = shmem_open("shared_pagetable_segment", O_CREATE|O_RDWR|O_SHARED_PAGE_TABLE, 0666);
The remaining handling is straightforward and the shmem subsystem already
provides consistent handling of shared memory segments.
What you would have to do is to sort out the kernel internal problems
created by sharing page table sections when using SHM vmas. But with that
there are only limited changes required to special types of vma and the
shmem subsystem. So the impact on the kernel overall is limited and you
are following an established method of managing shared memory.
I actually need something like shared page tables also for another in
kernel page table use case in order to define sections in kernel virtual
memory that are special for cpus or nodes. Some abstracted functions to
manage page tables that share pgd,pud,pmd would be good to have in the
kernel if you dont mind.
But for this use case I'd suggest to use gigabyte shmem mappings and
be done with it.
https://lwn.net/Articles/375098/
next prev parent reply other threads:[~2024-05-14 18:21 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-02-28 22:56 [LSF/MM/BPF TOPIC] Sharing page tables across processes (mshare) Khalid Aziz
2024-02-29 9:21 ` David Hildenbrand
2024-02-29 14:12 ` Matthew Wilcox
2024-02-29 15:15 ` David Hildenbrand
2024-03-04 16:45 ` Khalid Aziz
2024-03-25 17:57 ` David Hildenbrand
2024-05-14 18:21 ` Christoph Lameter (Ampere) [this message]
2024-05-17 21:23 ` Khalid Aziz
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1ee0e6e0-f82a-5c90-0bca-5c5b93aeadc9@linux.com \
--to=cl@linux.com \
--cc=khalid.aziz@oracle.com \
--cc=linux-mm@kvack.org \
--cc=lsf-pc@lists.linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).