From: Anshuman Khandual <anshuman.khandual@arm.com>
To: Nadav Amit <namit@vmware.com>
Cc: "wangkefeng.wang@huawei.com" <wangkefeng.wang@huawei.com>,
"prime.zeng@hisilicon.com" <prime.zeng@hisilicon.com>,
"linux-doc@vger.kernel.org" <linux-doc@vger.kernel.org>,
"peterz@infradead.org" <peterz@infradead.org>,
"catalin.marinas@arm.com" <catalin.marinas@arm.com>,
"yangyicong@hisilicon.com" <yangyicong@hisilicon.com>,
Linux MM <linux-mm@kvack.org>,
"guojian@oppo.com" <guojian@oppo.com>,
"linux-riscv@lists.infradead.org"
<linux-riscv@lists.infradead.org>, Will Deacon <will@kernel.org>,
"linux-s390@vger.kernel.org" <linux-s390@vger.kernel.org>,
"zhangshiming@oppo.com" <zhangshiming@oppo.com>,
"lipeifeng@oppo.com" <lipeifeng@oppo.com>,
"corbet@lwn.net" <corbet@lwn.net>,
"x86@kernel.org" <x86@kernel.org>, Barry Song <21cnbao@gmail.com>,
Mel Gorman <mgorman@suse.de>,
"linux-mips@vger.kernel.org" <linux-mips@vger.kernel.org>,
"arnd@arndb.de" <arnd@arndb.de>,
"realmz6@gmail.com" <realmz6@gmail.com>,
Barry Song <v-songbaohua@oppo.com>,
"openrisc@lists.librecores.org" <openrisc@lists.librecores.org>,
"darren@os.amperecomputing.com" <darren@os.amperecomputing.com>,
"linux-arm-kernel@lists.infradead.org"
<linux-arm-kernel@lists.infradead.org>,
"xhao@linux.alibaba.com" <xhao@linux.alibaba.com>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"huzhanyuan@oppo.com" <huzhanyuan@oppo.com>,
Yicong Yang <yangyicong@huawei.com>,
Andrew Morton <akpm@linux-foundation.org>,
"linuxppc-dev@lists.ozlabs.org" <linuxppc-dev@lists.ozlabs.org>
Subject: Re: [PATCH v3 4/4] arm64: support batched/deferred tlb shootdown during page reclamation
Date: Thu, 22 Sep 2022 08:45:22 +0530 [thread overview]
Message-ID: <0236922f-841e-c6d8-c9ee-599d72c458d3@arm.com> (raw)
In-Reply-To: <36B9DE22-E3BC-4CB2-8E3F-B21B61434CD3@vmware.com>
On 9/21/22 12:47, Nadav Amit wrote:
> On Sep 20, 2022, at 11:53 PM, Anshuman Khandual <anshuman.khandual@arm.com> wrote:
>
>> ⚠ External Email
>>
>> On 8/22/22 13:51, Yicong Yang wrote:
>>> +static inline void arch_tlbbatch_add_mm(struct arch_tlbflush_unmap_batch *batch,
>>> + struct mm_struct *mm,
>>> + unsigned long uaddr)
>>> +{
>>> + __flush_tlb_page_nosync(mm, uaddr);
>>> +}
>>> +
>>> +static inline void arch_tlbbatch_flush(struct arch_tlbflush_unmap_batch *batch)
>>> +{
>>> + dsb(ish);
>>> +}
>>
>> Just wondering if arch_tlbbatch_add_mm() could also detect continuous mapping
>> TLB invalidation requests on a given mm and try to generate a range based TLB
>> invalidation such as flush_tlb_range().
>>
>> struct arch_tlbflush_unmap_batch via task->tlb_ubc->arch can track continuous
>> ranges while being queued up via arch_tlbbatch_add_mm(), any range formed can
>> later be flushed in subsequent arch_tlbbatch_flush() ?
>>
>> OR
>>
>> It might not be worth the effort and complexity, in comparison to performance
>> improvement, TLB range flush brings in ?
>
> So here are my 2 cents, based on my experience with Intel-x86. It is likely
> different on arm64, but perhaps it can provide you some insight into what
> parameters you should measure and consider.
>
> In general there is a tradeoff between full TLB flushes and entry-specific
> ones. Flushing specific entries takes more time than flushing the entire
> TLB, but sade TLB refills.
Right.
>
> Dave Hansen made some calculations in the past and came up with 33 as a
> magic cutoff number, i.e., if you need to flush more than 33 entries, just
> flush the entire TLB. I am not sure that this exact number is very
> meaningful, since one might argue that it should’ve taken PTI into account
> (which might require twice as many TLB invalidations).
Okay.
>
> Anyhow, back to arch_tlbbatch_add_mm(). It may be possible to track ranges,
> but the question is whether you would actually succeed in forming continuous
> ranges that are eventually (on x86) smaller than the full TLB flush cutoff
> (=33). Questionable (perhaps better with MGLRU?).
This proposal here for arm64 does not cause a full TLB flush ever. It creates
individual TLB flushes all the time. Hence the choice here is not between full
TLB flush and possible range flushes. Choice is actually between individual
TLB flushes and range/full TLB flushes.
>
> Then, you should remember that tracking should be very efficient, since even
> few cache misses might have greater cost than what you save by
> selective-flushing. Finally, on x86 you would need to invoke the smp/IPI
> layer multiple times to send different cores the relevant range they need to
> flush.
Agreed, these reasons make it much difficult to gain any more performance.
>
> IOW: It is somewhat complicated to implement efficeintly. On x86, and
> probably other IPI-based TLB shootdown systems, does not have clear
> performance benefit (IMHO).
Agreed, thanks for such a detailed explanation, appreciate it.
next prev parent reply other threads:[~2022-09-22 3:15 UTC|newest]
Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-08-22 8:21 [PATCH v3 0/4] mm: arm64: bring up BATCHED_UNMAP_TLB_FLUSH Yicong Yang
2022-08-22 8:21 ` [PATCH v3 1/4] Revert "Documentation/features: mark BATCHED_UNMAP_TLB_FLUSH doesn't apply to ARM64" Yicong Yang
2022-09-09 4:26 ` Anshuman Khandual
2022-09-09 4:40 ` Barry Song
2022-08-22 8:21 ` [PATCH v3 2/4] mm/tlbbatch: Introduce arch_tlbbatch_should_defer() Yicong Yang
2022-08-24 9:40 ` Kefeng Wang
2022-09-09 4:16 ` Anshuman Khandual
2022-08-22 8:21 ` [PATCH v3 3/4] mm: rmap: Extend tlbbatch APIs to fit new platforms Yicong Yang
2022-08-24 9:43 ` Kefeng Wang
2022-09-09 4:51 ` Anshuman Khandual
2022-09-09 5:25 ` Barry Song
2022-08-22 8:21 ` [PATCH v3 4/4] arm64: support batched/deferred tlb shootdown during page reclamation Yicong Yang
2022-08-24 9:46 ` Kefeng Wang
2022-09-09 5:24 ` Anshuman Khandual
2022-09-09 5:35 ` Barry Song
2022-09-09 6:32 ` Yicong Yang
2022-09-15 6:07 ` Anshuman Khandual
2022-09-15 6:42 ` Barry Song
2022-09-15 14:31 ` Nadav Amit
2022-09-19 2:46 ` Anshuman Khandual
2022-09-19 4:24 ` Anshuman Khandual
2022-09-19 4:53 ` Barry Song
2022-09-19 5:08 ` Barry Song
2022-09-20 3:00 ` Anshuman Khandual
2022-09-20 3:39 ` Barry Song
2022-09-20 8:45 ` Anshuman Khandual
2022-09-21 1:50 ` Barry Song
2022-09-21 1:51 ` Barry Song
2022-09-21 3:33 ` Anshuman Khandual
2022-09-21 6:53 ` Anshuman Khandual
2022-09-21 7:15 ` Barry Song
2022-09-21 7:17 ` Nadav Amit
2022-09-22 3:15 ` Anshuman Khandual [this message]
2022-09-06 8:53 ` [PATCH v3 0/4] mm: arm64: bring up BATCHED_UNMAP_TLB_FLUSH Yicong Yang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=0236922f-841e-c6d8-c9ee-599d72c458d3@arm.com \
--to=anshuman.khandual@arm.com \
--cc=21cnbao@gmail.com \
--cc=akpm@linux-foundation.org \
--cc=arnd@arndb.de \
--cc=catalin.marinas@arm.com \
--cc=corbet@lwn.net \
--cc=darren@os.amperecomputing.com \
--cc=guojian@oppo.com \
--cc=huzhanyuan@oppo.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mips@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-riscv@lists.infradead.org \
--cc=linux-s390@vger.kernel.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=lipeifeng@oppo.com \
--cc=mgorman@suse.de \
--cc=namit@vmware.com \
--cc=openrisc@lists.librecores.org \
--cc=peterz@infradead.org \
--cc=prime.zeng@hisilicon.com \
--cc=realmz6@gmail.com \
--cc=v-songbaohua@oppo.com \
--cc=wangkefeng.wang@huawei.com \
--cc=will@kernel.org \
--cc=x86@kernel.org \
--cc=xhao@linux.alibaba.com \
--cc=yangyicong@hisilicon.com \
--cc=yangyicong@huawei.com \
--cc=zhangshiming@oppo.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).