From: Frederic Weisbecker <frederic@kernel.org>
To: LKML <linux-kernel@vger.kernel.org>
Cc: Frederic Weisbecker <frederic@kernel.org>,
Valentin Schneider <vschneid@redhat.com>,
"Paul E . McKenney" <paulmck@kernel.org>,
Boqun Feng <boqun.feng@gmail.com>,
Joel Fernandes <joel@joelfernandes.org>,
Neeraj Upadhyay <neeraj.upadhyay@amd.com>,
Uladzislau Rezki <urezki@gmail.com>,
Zqiang <qiang.zhang1211@gmail.com>, rcu <rcu@vger.kernel.org>
Subject: [PATCH 1/6] rcu: Remove full ordering on second EQS snapshot
Date: Wed, 15 May 2024 14:53:27 +0200 [thread overview]
Message-ID: <20240515125332.9306-2-frederic@kernel.org> (raw)
In-Reply-To: <20240515125332.9306-1-frederic@kernel.org>
When the grace period kthread checks the extended quiescent state
counter of a CPU, full ordering is necessary to ensure that either:
* If the GP kthread observes the remote target in an extended quiescent
state, then that target must observe all accesses prior to the current
grace period, including the current grace period sequence number, once
it exits that extended quiescent state. Also the GP kthread must
observe all accesses performed by the target prior it entering in
EQS.
or:
* If the GP kthread observes the remote target NOT in an extended
quiescent state, then the target further entering in an extended
quiescent state must observe all accesses prior to the current
grace period, including the current grace period sequence number, once
it enters that extended quiescent state. Also the GP kthread later
observing that EQS must also observe all accesses performed by the
target prior it entering in EQS.
This ordering is explicitly performed both on the first EQS snapshot
and on the second one as well through the combination of a preceding
full barrier followed by an acquire read. However the second snapshot's
full memory barrier is redundant and not needed to enforce the above
guarantees:
GP kthread Remote target
---- -----
// Access prior GP
WRITE_ONCE(A, 1)
// first snapshot
smp_mb()
x = smp_load_acquire(EQS)
// Access prior GP
WRITE_ONCE(B, 1)
// EQS enter
// implied full barrier by atomic_add_return()
atomic_add_return(RCU_DYNTICKS_IDX, EQS)
// implied full barrier by atomic_add_return()
READ_ONCE(A)
// second snapshot
y = smp_load_acquire(EQS)
z = READ_ONCE(B)
If the GP kthread above fails to observe the remote target in EQS
(x not in EQS), the remote target will observe A == 1 after further
entering in EQS. Then the second snapshot taken by the GP kthread only
need to be an acquire read in order to observe z == 1.
Therefore remove the needless full memory barrier on second snapshot.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
---
kernel/rcu/tree.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index 5e6828132007..58415cdc54f8 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -325,7 +325,7 @@ static bool rcu_dynticks_in_eqs(int snap)
*/
static bool rcu_dynticks_in_eqs_since(struct rcu_data *rdp, int snap)
{
- return snap != rcu_dynticks_snap(rdp->cpu);
+ return snap != ct_dynticks_cpu_acquire(rdp->cpu);
}
/*
--
2.44.0
next prev parent reply other threads:[~2024-05-15 12:53 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-05-15 12:53 [PATCH 0/6] rcu: Remove several redundant memory barriers Frederic Weisbecker
2024-05-15 12:53 ` Frederic Weisbecker [this message]
2024-05-15 17:32 ` [PATCH 1/6] rcu: Remove full ordering on second EQS snapshot Valentin Schneider
2024-05-15 12:53 ` [PATCH 2/6] rcu: Remove superfluous full memory barrier upon first " Frederic Weisbecker
2024-05-16 15:31 ` Valentin Schneider
2024-05-16 16:08 ` Frederic Weisbecker
2024-05-16 17:08 ` Valentin Schneider
2024-05-17 7:29 ` Andrea Parri
2024-05-17 11:40 ` Frederic Weisbecker
2024-05-17 16:27 ` Andrea Parri
2024-05-15 12:53 ` [PATCH 3/6] rcu/exp: " Frederic Weisbecker
2024-05-15 12:53 ` [PATCH 4/6] rcu: Remove full memory barrier on boot time eqs sanity check Frederic Weisbecker
2024-05-16 17:09 ` Valentin Schneider
2024-05-15 12:53 ` [PATCH 5/6] rcu: Remove full memory barrier on RCU stall printout Frederic Weisbecker
2024-05-16 17:09 ` Valentin Schneider
2024-05-15 12:53 ` [PATCH 6/6] rcu/exp: Remove redundant full memory barrier at the end of GP Frederic Weisbecker
2024-05-15 17:32 ` [PATCH 0/6] rcu: Remove several redundant memory barriers Valentin Schneider
2024-05-15 23:13 ` Frederic Weisbecker
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240515125332.9306-2-frederic@kernel.org \
--to=frederic@kernel.org \
--cc=boqun.feng@gmail.com \
--cc=joel@joelfernandes.org \
--cc=linux-kernel@vger.kernel.org \
--cc=neeraj.upadhyay@amd.com \
--cc=paulmck@kernel.org \
--cc=qiang.zhang1211@gmail.com \
--cc=rcu@vger.kernel.org \
--cc=urezki@gmail.com \
--cc=vschneid@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).