From: Frederic Weisbecker <frederic@kernel.org>
To: LKML <linux-kernel@vger.kernel.org>
Cc: Frederic Weisbecker <frederic@kernel.org>,
Valentin Schneider <vschneid@redhat.com>,
"Paul E . McKenney" <paulmck@kernel.org>,
Boqun Feng <boqun.feng@gmail.com>,
Joel Fernandes <joel@joelfernandes.org>,
Neeraj Upadhyay <neeraj.upadhyay@amd.com>,
Uladzislau Rezki <urezki@gmail.com>,
Zqiang <qiang.zhang1211@gmail.com>, rcu <rcu@vger.kernel.org>
Subject: [PATCH 2/6] rcu: Remove superfluous full memory barrier upon first EQS snapshot
Date: Wed, 15 May 2024 14:53:28 +0200 [thread overview]
Message-ID: <20240515125332.9306-3-frederic@kernel.org> (raw)
In-Reply-To: <20240515125332.9306-1-frederic@kernel.org>
When the grace period kthread checks the extended quiescent state
counter of a CPU, full ordering is necessary to ensure that either:
* If the GP kthread observes the remote target in an extended quiescent
state, then that target must observe all accesses prior to the current
grace period, including the current grace period sequence number, once
it exits that extended quiescent state.
or:
* If the GP kthread observes the remote target NOT in an extended
quiescent state, then the target further entering in an extended
quiescent state must observe all accesses prior to the current
grace period, including the current grace period sequence number, once
it enters that extended quiescent state.
This ordering is enforced through a full memory barrier placed right
before taking the first EQS snapshot. However this is superfluous
because the snapshot is taken while holding the target's rnp lock which
provides the necessary ordering through its chain of
smp_mb__after_unlock_lock().
Remove the needless explicit barrier before the snapshot and put a
comment about the implicit barrier newly relied upon here.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
---
.../Design/Memory-Ordering/Tree-RCU-Memory-Ordering.rst | 6 +++---
kernel/rcu/tree.c | 7 ++++++-
2 files changed, 9 insertions(+), 4 deletions(-)
diff --git a/Documentation/RCU/Design/Memory-Ordering/Tree-RCU-Memory-Ordering.rst b/Documentation/RCU/Design/Memory-Ordering/Tree-RCU-Memory-Ordering.rst
index 5750f125361b..728b1e690c64 100644
--- a/Documentation/RCU/Design/Memory-Ordering/Tree-RCU-Memory-Ordering.rst
+++ b/Documentation/RCU/Design/Memory-Ordering/Tree-RCU-Memory-Ordering.rst
@@ -149,9 +149,9 @@ This case is handled by calls to the strongly ordered
``atomic_add_return()`` read-modify-write atomic operation that
is invoked within ``rcu_dynticks_eqs_enter()`` at idle-entry
time and within ``rcu_dynticks_eqs_exit()`` at idle-exit time.
-The grace-period kthread invokes ``rcu_dynticks_snap()`` and
-``rcu_dynticks_in_eqs_since()`` (both of which invoke
-an ``atomic_add_return()`` of zero) to detect idle CPUs.
+The grace-period kthread invokes first ``ct_dynticks_cpu_acquire()``
+(preceded by a full memory barrier) and ``rcu_dynticks_in_eqs_since()``
+(both of which rely on acquire semantics) to detect idle CPUs.
+-----------------------------------------------------------------------+
| **Quick Quiz**: |
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index 58415cdc54f8..f5354de5644b 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -773,7 +773,12 @@ static void rcu_gpnum_ovf(struct rcu_node *rnp, struct rcu_data *rdp)
*/
static int dyntick_save_progress_counter(struct rcu_data *rdp)
{
- rdp->dynticks_snap = rcu_dynticks_snap(rdp->cpu);
+ /*
+ * Full ordering against accesses prior current GP and also against
+ * current GP sequence number is enforced by current rnp locking
+ * with chained smp_mb__after_unlock_lock().
+ */
+ rdp->dynticks_snap = ct_dynticks_cpu_acquire(rdp->cpu);
if (rcu_dynticks_in_eqs(rdp->dynticks_snap)) {
trace_rcu_fqs(rcu_state.name, rdp->gp_seq, rdp->cpu, TPS("dti"));
rcu_gpnum_ovf(rdp->mynode, rdp);
--
2.44.0
next prev parent reply other threads:[~2024-05-15 12:53 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-05-15 12:53 [PATCH 0/6] rcu: Remove several redundant memory barriers Frederic Weisbecker
2024-05-15 12:53 ` [PATCH 1/6] rcu: Remove full ordering on second EQS snapshot Frederic Weisbecker
2024-05-15 17:32 ` Valentin Schneider
2024-05-15 12:53 ` Frederic Weisbecker [this message]
2024-05-16 15:31 ` [PATCH 2/6] rcu: Remove superfluous full memory barrier upon first " Valentin Schneider
2024-05-16 16:08 ` Frederic Weisbecker
2024-05-16 17:08 ` Valentin Schneider
2024-05-17 7:29 ` Andrea Parri
2024-05-17 11:40 ` Frederic Weisbecker
2024-05-17 16:27 ` Andrea Parri
2024-05-15 12:53 ` [PATCH 3/6] rcu/exp: " Frederic Weisbecker
2024-05-15 12:53 ` [PATCH 4/6] rcu: Remove full memory barrier on boot time eqs sanity check Frederic Weisbecker
2024-05-16 17:09 ` Valentin Schneider
2024-05-15 12:53 ` [PATCH 5/6] rcu: Remove full memory barrier on RCU stall printout Frederic Weisbecker
2024-05-16 17:09 ` Valentin Schneider
2024-05-15 12:53 ` [PATCH 6/6] rcu/exp: Remove redundant full memory barrier at the end of GP Frederic Weisbecker
2024-05-15 17:32 ` [PATCH 0/6] rcu: Remove several redundant memory barriers Valentin Schneider
2024-05-15 23:13 ` Frederic Weisbecker
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240515125332.9306-3-frederic@kernel.org \
--to=frederic@kernel.org \
--cc=boqun.feng@gmail.com \
--cc=joel@joelfernandes.org \
--cc=linux-kernel@vger.kernel.org \
--cc=neeraj.upadhyay@amd.com \
--cc=paulmck@kernel.org \
--cc=qiang.zhang1211@gmail.com \
--cc=rcu@vger.kernel.org \
--cc=urezki@gmail.com \
--cc=vschneid@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).