From: Leonardo Bras <leobras@redhat.com>
To: Paolo Bonzini <pbonzini@redhat.com>,
"Paul E. McKenney" <paulmck@kernel.org>,
Frederic Weisbecker <frederic@kernel.org>,
Neeraj Upadhyay <quic_neeraju@quicinc.com>,
Joel Fernandes <joel@joelfernandes.org>,
Josh Triplett <josh@joshtriplett.org>,
Boqun Feng <boqun.feng@gmail.com>,
Steven Rostedt <rostedt@goodmis.org>,
Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
Lai Jiangshan <jiangshanlai@gmail.com>,
Zqiang <qiang.zhang1211@gmail.com>,
Marcelo Tosatti <mtosatti@redhat.com>
Cc: Leonardo Bras <leobras@redhat.com>,
kvm@vger.kernel.org, linux-kernel@vger.kernel.org,
rcu@vger.kernel.org
Subject: [RFC PATCH v1 0/2] Avoid rcu_core() if CPU just left guest vcpu
Date: Thu, 28 Mar 2024 14:19:45 -0300 [thread overview]
Message-ID: <20240328171949.743211-1-leobras@redhat.com> (raw)
I am dealing with a latency issue inside a KVM guest, which is caused by
a sched_switch to rcuc[1].
During guest entry, kernel code will signal to RCU that current CPU was on
a quiescent state, making sure no other CPU is waiting for this one.
If a vcpu just stopped running (guest_exit), and a syncronize_rcu() was
issued somewhere since guest entry, there is a chance a timer interrupt
will happen in that CPU, which will cause rcu_sched_clock_irq() to run.
rcu_sched_clock_irq() will check rcu_pending() which will return true,
and cause invoke_rcu_core() to be called, which will (in current config)
cause rcuc/N to be scheduled into the current cpu.
On rcu_pending(), I noticed we can avoid returning true (and thus invoking
rcu_core()) if the current cpu is nohz_full, and the cpu came from either
idle or userspace, since both are considered quiescent states.
Since this is also true to guest context, my idea to solve this latency
issue by avoiding rcu_core() invocation if it was running a guest vcpu.
On the other hand, I could not find a way of reliably saying the current
cpu was running a guest vcpu, so patch #1 implements a per-cpu variable
for keeping the time (jiffies) of the last guest exit.
In patch #2 I compare current time to that time, and if less than a second
has past, we just skip rcu_core() invocation, since there is a high chance
it will just go back to the guest in a moment.
What I know it's weird with this patch:
1 - Not sure if this is the best way of finding out if the cpu was
running a guest recently.
2 - This per-cpu variable needs to get set at each guest_exit(), so it's
overhead, even though it's supposed to be in local cache. If that's
an issue, I would suggest having this part compiled out on
!CONFIG_NO_HZ_FULL, but further checking each cpu for being nohz_full
enabled seems more expensive than just setting this out.
3 - It checks if the guest exit happened over than 1 second ago. This 1
second value was copied from rcu_nohz_full_cpu() which checks if the
grace period started over than a second ago. If this value is bad,
I have no issue changing it.
4 - Even though I could detect no issue, I included linux/kvm_host.h into
rcu/tree_plugin.h, which is the first time it's getting included
outside of kvm or arch code, and can be weird. An alternative would
be to create a new header for providing data for non-kvm code.
Please provide feedback.
Thanks!
Leo
...
[1]: It uses a PREEMPT_RT kernel, with the guest cpus running on isolated,
rcu_nocbs, nohz_full cpus.
Leonardo Bras (2):
kvm: Implement guest_exit_last_time()
rcu: Ignore RCU in nohz_full cpus if it was running a guest recently
include/linux/kvm_host.h | 13 +++++++++++++
kernel/rcu/tree_plugin.h | 14 ++++++++++++++
kernel/rcu/tree.c | 4 +++-
virt/kvm/kvm_main.c | 3 +++
4 files changed, 33 insertions(+), 1 deletion(-)
base-commit: 8d025e2092e29bfd13e56c78e22af25fac83c8ec
--
2.44.0
next reply other threads:[~2024-03-28 17:20 UTC|newest]
Thread overview: 60+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-03-28 17:19 Leonardo Bras [this message]
2024-03-28 17:19 ` [RFC PATCH v1 1/2] kvm: Implement guest_exit_last_time() Leonardo Bras
2024-03-28 17:19 ` [RFC PATCH v1 2/2] rcu: Ignore RCU in nohz_full cpus if it was running a guest recently Leonardo Bras
2024-04-01 15:52 ` Paul E. McKenney
2024-04-01 20:21 ` [RFC PATCH v1 0/2] Avoid rcu_core() if CPU just left guest vcpu Sean Christopherson
2024-04-05 13:45 ` Marcelo Tosatti
2024-04-05 14:42 ` Sean Christopherson
2024-04-06 0:03 ` Paul E. McKenney
2024-04-08 17:16 ` Sean Christopherson
2024-04-08 18:42 ` Paul E. McKenney
2024-04-08 20:06 ` Sean Christopherson
2024-04-08 21:02 ` Paul E. McKenney
2024-04-08 21:56 ` Sean Christopherson
2024-04-08 22:35 ` Paul E. McKenney
2024-04-08 23:06 ` Sean Christopherson
2024-04-08 23:20 ` Paul E. McKenney
2024-04-10 2:39 ` Marcelo Tosatti
2024-04-15 19:47 ` Marcelo Tosatti
2024-04-15 21:29 ` Sean Christopherson
2024-04-16 12:36 ` Marcelo Tosatti
2024-04-16 14:07 ` Sean Christopherson
2024-04-17 16:14 ` Marcelo Tosatti
2024-04-17 17:22 ` Sean Christopherson
2024-05-03 20:44 ` Leonardo Bras
2024-05-06 18:47 ` Marcelo Tosatti
2024-05-07 18:05 ` Sean Christopherson
2024-05-07 22:36 ` Leonardo Bras
2024-05-03 18:42 ` Leonardo Bras
2024-05-03 19:09 ` Leonardo Bras
2024-05-03 21:29 ` Sean Christopherson
2024-05-03 22:00 ` Leonardo Bras
2024-05-03 22:00 ` Paul E. McKenney
2024-05-07 17:55 ` Sean Christopherson
2024-05-07 19:15 ` Paul E. McKenney
2024-05-07 21:00 ` Sean Christopherson
2024-05-07 21:37 ` Paul E. McKenney
2024-05-07 23:47 ` Sean Christopherson
2024-05-08 0:08 ` Sean Christopherson
2024-05-08 2:51 ` Leonardo Bras
2024-05-08 3:22 ` Paul E. McKenney
2024-05-08 6:19 ` Leonardo Bras
2024-05-08 14:01 ` Sean Christopherson
2024-05-09 3:32 ` Paul E. McKenney
2024-05-09 8:16 ` Leonardo Bras
2024-05-09 10:14 ` Leonardo Bras
2024-05-09 23:45 ` Paul E. McKenney
2024-05-10 16:06 ` Leonardo Bras
2024-05-10 16:21 ` Paul E. McKenney
2024-05-10 17:12 ` Leonardo Bras
2024-05-10 17:41 ` Paul E. McKenney
2024-05-10 19:50 ` Leonardo Bras
2024-05-10 21:15 ` Leonardo Bras
2024-05-10 21:38 ` Paul E. McKenney
2024-05-09 22:41 ` Paul E. McKenney
2024-05-09 23:07 ` Leonardo Bras Soares Passos
2024-05-11 2:08 ` Leonardo Bras
2024-05-08 3:20 ` Paul E. McKenney
2024-05-08 4:04 ` Paul E. McKenney
2024-05-08 14:36 ` Paul E. McKenney
2024-05-08 15:35 ` Sean Christopherson
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240328171949.743211-1-leobras@redhat.com \
--to=leobras@redhat.com \
--cc=boqun.feng@gmail.com \
--cc=frederic@kernel.org \
--cc=jiangshanlai@gmail.com \
--cc=joel@joelfernandes.org \
--cc=josh@joshtriplett.org \
--cc=kvm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mathieu.desnoyers@efficios.com \
--cc=mtosatti@redhat.com \
--cc=paulmck@kernel.org \
--cc=pbonzini@redhat.com \
--cc=qiang.zhang1211@gmail.com \
--cc=quic_neeraju@quicinc.com \
--cc=rcu@vger.kernel.org \
--cc=rostedt@goodmis.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).