From: Vineeth Remanan Pillai <vineeth@bitbyteword.org>
To: Ben Segall <bsegall@google.com>, Borislav Petkov <bp@alien8.de>,
Daniel Bristot de Oliveira <bristot@redhat.com>,
Dave Hansen <dave.hansen@linux.intel.com>,
Dietmar Eggemann <dietmar.eggemann@arm.com>,
"H . Peter Anvin" <hpa@zytor.com>, Ingo Molnar <mingo@redhat.com>,
Juri Lelli <juri.lelli@redhat.com>, Mel Gorman <mgorman@suse.de>,
Paolo Bonzini <pbonzini@redhat.com>,
Andy Lutomirski <luto@kernel.org>,
Peter Zijlstra <peterz@infradead.org>,
Sean Christopherson <seanjc@google.com>,
Thomas Gleixner <tglx@linutronix.de>,
Valentin Schneider <vschneid@redhat.com>,
Vincent Guittot <vincent.guittot@linaro.org>,
Vitaly Kuznetsov <vkuznets@redhat.com>,
Wanpeng Li <wanpengli@tencent.com>
Cc: Steven Rostedt <rostedt@goodmis.org>,
Joel Fernandes <joel@joelfernandes.org>,
Suleiman Souhlal <suleiman@google.com>,
Masami Hiramatsu <mhiramat@kernel.org>,
himadrics@inria.fr, kvm@vger.kernel.org,
linux-kernel@vger.kernel.org, x86@kernel.org,
Tejun Heo <tj@kernel.org>, Josh Don <joshdon@google.com>,
Barret Rhoden <brho@google.com>, David Vernet <dvernet@meta.com>
Subject: Re: [RFC PATCH v2 0/5] Paravirt Scheduling (Dynamic vcpu priority management)
Date: Mon, 8 Apr 2024 09:54:03 -0400 [thread overview]
Message-ID: <CAO7JXPiiN+w+Liuov3rXAbr1QLwt+eUq=4Weoy8gB0fXaC7D3Q@mail.gmail.com> (raw)
In-Reply-To: <20240403140116.3002809-1-vineeth@bitbyteword.org>
Sorry I missed sched_ext folks, adding them as well.
Thanks,
Vineeth
On Wed, Apr 3, 2024 at 10:01 AM Vineeth Pillai (Google)
<vineeth@bitbyteword.org> wrote:
>
> Double scheduling is a concern with virtualization hosts where the host
> schedules vcpus without knowing whats run by the vcpu and guest schedules
> tasks without knowing where the vcpu is physically running. This causes
> issues related to latencies, power consumption, resource utilization
> etc. An ideal solution would be to have a cooperative scheduling
> framework where the guest and host shares scheduling related information
> and makes an educated scheduling decision to optimally handle the
> workloads. As a first step, we are taking a stab at reducing latencies
> for latency sensitive workloads in the guest.
>
> v1 RFC[1] was posted in December 2023. The main disagreement was in the
> implementation where the patch was making scheduling policy decisions
> in kvm and kvm is not the right place to do it. The suggestion was to
> move the polcy decisions outside of kvm and let kvm only handle the
> notifications needed to make the policy decisions. This patch series is
> an iterative step towards implementing the feature as a layered
> design where the policy could be implemented outside of kvm as a
> kernel built-in, a kernel module or a bpf program.
>
> This design comprises mainly of 4 components:
>
> - pvsched driver: Implements the scheduling policies. Register with
> host with a set of callbacks that hypervisor(kvm) can use to notify
> vcpu events that the driver is interested in. The callback will be
> passed in the address of shared memory so that the driver can get
> scheduling information shared by the guest and also update the
> scheduling policies set by the driver.
> - kvm component: Selects the pvsched driver for a guest and notifies
> the driver via callbacks for events that the driver is interested
> in. Also interface with the guest in retreiving the shared memory
> region for sharing the scheduling information.
> - host kernel component: Implements the APIs for:
> - pvsched driver for register/unregister to the host kernel, and
> - hypervisor for assingning/unassigning driver for guests.
> - guest component: Implements a framework for sharing the scheduling
> information with the pvsched driver through kvm.
>
> There is another component that we refer to as pvsched protocol. This
> defines the details about shared memory layout, information sharing and
> sheduling policy decisions. The protocol need not be part of the kernel
> and can be defined separately based on the use case and requirements.
> Both guest and the selected pvsched driver need to match the protocol
> for the feature to work. Protocol shall be identified by a name and a
> possible versioning scheme. Guest will advertise the protocol and then
> the hypervisor can assign the driver implementing the protocol if it is
> registered in the host kernel.
>
> This patch series only implements the first 3 components. Guest side
> implementation and the protocol framework shall come as a separate
> series once we finalize rest of the design.
>
> This series also implements a sample bpf program and a kernel-builtin
> pvsched drivers. They do not do any real stuff now, but just skeletons
> to demonstrate the feature.
>
> Rebased on 6.8.2.
>
> [1]: https://lwn.net/Articles/955145/
>
> Vineeth Pillai (Google) (5):
> pvsched: paravirt scheduling framework
> kvm: Implement the paravirt sched framework for kvm
> kvm: interface for managing pvsched driver for guest VMs
> pvsched: bpf support for pvsched
> selftests/bpf: sample implementation of a bpf pvsched driver.
>
> Kconfig | 2 +
> arch/x86/kvm/Kconfig | 13 +
> arch/x86/kvm/x86.c | 3 +
> include/linux/kvm_host.h | 32 +++
> include/linux/pvsched.h | 102 +++++++
> include/uapi/linux/kvm.h | 6 +
> kernel/bpf/bpf_struct_ops_types.h | 4 +
> kernel/sysctl.c | 27 ++
> .../testing/selftests/bpf/progs/bpf_pvsched.c | 37 +++
> virt/Makefile | 2 +-
> virt/kvm/kvm_main.c | 265 ++++++++++++++++++
> virt/pvsched/Kconfig | 12 +
> virt/pvsched/Makefile | 2 +
> virt/pvsched/pvsched.c | 215 ++++++++++++++
> virt/pvsched/pvsched_bpf.c | 141 ++++++++++
> 15 files changed, 862 insertions(+), 1 deletion(-)
> create mode 100644 include/linux/pvsched.h
> create mode 100644 tools/testing/selftests/bpf/progs/bpf_pvsched.c
> create mode 100644 virt/pvsched/Kconfig
> create mode 100644 virt/pvsched/Makefile
> create mode 100644 virt/pvsched/pvsched.c
> create mode 100644 virt/pvsched/pvsched_bpf.c
>
> --
> 2.40.1
>
next prev parent reply other threads:[~2024-04-08 13:54 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-04-03 14:01 [RFC PATCH v2 0/5] Paravirt Scheduling (Dynamic vcpu priority management) Vineeth Pillai (Google)
2024-04-03 14:01 ` [RFC PATCH v2 1/5] pvsched: paravirt scheduling framework Vineeth Pillai (Google)
2024-04-08 13:57 ` Vineeth Remanan Pillai
2024-04-03 14:01 ` [RFC PATCH v2 2/5] kvm: Implement the paravirt sched framework for kvm Vineeth Pillai (Google)
2024-04-08 13:58 ` Vineeth Remanan Pillai
2024-04-03 14:01 ` [RFC PATCH v2 3/5] kvm: interface for managing pvsched driver for guest VMs Vineeth Pillai (Google)
2024-04-08 13:59 ` Vineeth Remanan Pillai
2024-04-03 14:01 ` [RFC PATCH v2 4/5] pvsched: bpf support for pvsched Vineeth Pillai (Google)
2024-04-08 14:00 ` Vineeth Remanan Pillai
2024-04-03 14:01 ` [RFC PATCH v2 5/5] selftests/bpf: sample implementation of a bpf pvsched driver Vineeth Pillai (Google)
2024-04-08 14:01 ` Vineeth Remanan Pillai
2024-04-08 13:54 ` Vineeth Remanan Pillai [this message]
2024-05-01 15:29 ` [RFC PATCH v2 0/5] Paravirt Scheduling (Dynamic vcpu priority management) Sean Christopherson
2024-05-02 13:42 ` Vineeth Remanan Pillai
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAO7JXPiiN+w+Liuov3rXAbr1QLwt+eUq=4Weoy8gB0fXaC7D3Q@mail.gmail.com' \
--to=vineeth@bitbyteword.org \
--cc=bp@alien8.de \
--cc=brho@google.com \
--cc=bristot@redhat.com \
--cc=bsegall@google.com \
--cc=dave.hansen@linux.intel.com \
--cc=dietmar.eggemann@arm.com \
--cc=dvernet@meta.com \
--cc=himadrics@inria.fr \
--cc=hpa@zytor.com \
--cc=joel@joelfernandes.org \
--cc=joshdon@google.com \
--cc=juri.lelli@redhat.com \
--cc=kvm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=luto@kernel.org \
--cc=mgorman@suse.de \
--cc=mhiramat@kernel.org \
--cc=mingo@redhat.com \
--cc=pbonzini@redhat.com \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=seanjc@google.com \
--cc=suleiman@google.com \
--cc=tglx@linutronix.de \
--cc=tj@kernel.org \
--cc=vincent.guittot@linaro.org \
--cc=vkuznets@redhat.com \
--cc=vschneid@redhat.com \
--cc=wanpengli@tencent.com \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).