From: Julian Anastasov <ja@ssi.bg>
To: Jiri Wiesner <jwiesner@suse.de>
Cc: Simon Horman <horms@verge.net.au>,
lvs-devel@vger.kernel.org,
yunhong-cgl jiang <xintian1976@gmail.com>,
yunhjiang@ebay.com, dust.li@linux.alibaba.com,
tangyang@zhihu.com
Subject: [RFC PATCH 0/4] Use kthreads for stats
Date: Sat, 27 Aug 2022 20:41:50 +0300 [thread overview]
Message-ID: <20220827174154.220651-1-ja@ssi.bg> (raw)
Hello,
This patchset implements stats estimation in
kthread context. Simple tests do not show any problem.
Please review, comment, test, etc.
Overview of the basic concepts. More in the
commit messages...
RCU Locking:
- when RCU preemption is enabled the kthreads use just RCU
lock for walking the chains and we do not need to reschedule.
May be this is the common case for distribution kernels.
In this case ip_vs_stop_estimator() is completely lockless.
- when RCU preemption is not enabled, we reschedule by using
refcnt for every estimator to track if the currently removed
estimator is used at the same time by kthread for estimation.
As RCU lock is unlocked during rescheduling, the deletion
should wait kd->mutex, so that a new RCU lock is applied
before the estimator is freed with RCU callback.
- As stats are now RCU-locked, tot_stats, svc and dest which
hold estimator structures are now always freed from RCU
callback. This ensures RCU grace period after the
ip_vs_stop_estimator() call.
Kthread data:
- every kthread works over its own data structure and all
such structures are attached to array
- even while there can be a kthread structure, its task
may not be running, eg. before first service is added or
while the sysctl var is set to an empty cpulist or
when run_estimation is 0.
- a task and its structure may be released if all
estimators are unlinked from its chains, leaving the
slot in the array empty
- to add new estimators we use the last added kthread
context (est_add_ktid). The new estimators are linked to
the chain just before the estimated one, based on add_row.
This ensures their estimation will start after 2 seconds.
If estimators are added in bursts, common case if all
services and dests are initially configured, we may
spread the estimators to more chains. This will reduce
the chain imbalance.
- the chain imbalance is not so fatal when we use
kthreads. We design each kthread for part of the
possible CPU usage, so even if some chain exceeds its
time slot it would happen all the time or sporadic
depending on the scheduling but still keeping the
2-second interval. The cpulist isolation can make
the things more stable as a 2-second time interval
per estimator.
Julian Anastasov (4):
ipvs: add rcu protection to stats
ipvs: use kthreads for stats estimation
ipvs: add est_cpulist and est_nice sysctl vars
ipvs: run_estimation should control the kthread tasks
Documentation/networking/ipvs-sysctl.rst | 24 +-
include/net/ip_vs.h | 144 +++++++-
net/netfilter/ipvs/ip_vs_core.c | 10 +-
net/netfilter/ipvs/ip_vs_ctl.c | 287 ++++++++++++++--
net/netfilter/ipvs/ip_vs_est.c | 408 +++++++++++++++++++----
5 files changed, 771 insertions(+), 102 deletions(-)
--
2.37.2
next reply other threads:[~2022-08-27 17:41 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-08-27 17:41 Julian Anastasov [this message]
2022-08-27 17:41 ` [RFC PATCH 1/4] ipvs: add rcu protection to stats Julian Anastasov
2022-09-05 10:43 ` Jiri Wiesner
2022-08-27 17:41 ` [RFC PATCH 2/4] ipvs: use kthreads for stats estimation Julian Anastasov
2022-09-05 6:47 ` dust.li
2022-09-07 18:07 ` Julian Anastasov
2022-09-05 13:19 ` Jiri Wiesner
2022-09-07 19:01 ` Julian Anastasov
2022-09-08 16:00 ` Jiri Wiesner
2022-08-27 17:41 ` [RFC PATCH 3/4] ipvs: add est_cpulist and est_nice sysctl vars Julian Anastasov
2022-09-05 14:53 ` Jiri Wiesner
2022-08-27 17:41 ` [RFC PATCH 4/4] ipvs: run_estimation should control the kthread tasks Julian Anastasov
2022-09-05 14:57 ` Jiri Wiesner
2022-09-05 6:34 ` [RFC PATCH 0/4] Use kthreads for stats dust.li
2022-09-05 8:26 ` Jiri Wiesner
2022-09-07 18:33 ` Julian Anastasov
2022-09-08 15:35 ` Jiri Wiesner
2022-09-08 18:32 ` Jiri Wiesner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220827174154.220651-1-ja@ssi.bg \
--to=ja@ssi.bg \
--cc=dust.li@linux.alibaba.com \
--cc=horms@verge.net.au \
--cc=jwiesner@suse.de \
--cc=lvs-devel@vger.kernel.org \
--cc=tangyang@zhihu.com \
--cc=xintian1976@gmail.com \
--cc=yunhjiang@ebay.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).