From: Anup Patel <anup@brainfault.org>
To: Palmer Dabbelt <palmer@dabbelt.com>,
Palmer Dabbelt <palmer@rivosinc.com>
Cc: linux-kernel@vger.kernel.org,
Ajay Kaher <ajay.kaher@broadcom.com>,
Albert Ou <aou@eecs.berkeley.edu>,
Alexandre Ghiti <alexghiti@rivosinc.com>,
Andrew Jones <ajones@ventanamicro.com>,
samuel.holland@sifive.com,
Conor Dooley <conor.dooley@microchip.com>,
Juergen Gross <jgross@suse.com>,
kvm-riscv@lists.infradead.org, kvm@vger.kernel.org,
linux-kselftest@vger.kernel.org,
linux-riscv@lists.infradead.org,
Mark Rutland <mark.rutland@arm.com>,
Paolo Bonzini <pbonzini@redhat.com>,
Paul Walmsley <paul.walmsley@sifive.com>,
Atish Patra <atishp@rivosinc.com>, Shuah Khan <shuah@kernel.org>,
virtualization@lists.linux.dev, Will Deacon <will@kernel.org>,
x86@kernel.org
Subject: Re: [PATCH v8 00/24] RISC-V SBI v2.0 PMU improvements and Perf sampling in KVM guest
Date: Tue, 30 Apr 2024 11:24:18 +0530 [thread overview]
Message-ID: <CAAhSdy0k=-wtge-iA-rckQrd_ziNtWDnf4J-yCKP0OooZCaPzA@mail.gmail.com> (raw)
In-Reply-To: <CAAhSdy2tw1vy-vS3u8-Eq92d=TS_wFENTTrAFG7ked1aEfpkGA@mail.gmail.com>
Hi Palmer,
On Mon, Apr 22, 2024 at 3:29 PM Anup Patel <anup@brainfault.org> wrote:
>
> On Sat, Apr 20, 2024 at 5:17 AM Atish Patra <atishp@rivosinc.com> wrote:
> >
> > This series implements SBI PMU improvements done in SBI v2.0[1] i.e. PMU snapshot
> > and fw_read_hi() functions.
> >
> > SBI v2.0 introduced PMU snapshot feature which allows the SBI implementation
> > to provide counter information (i.e. values/overflow status) via a shared
> > memory between the SBI implementation and supervisor OS. This allows to minimize
> > the number of traps in when perf being used inside a kvm guest as it relies on
> > SBI PMU + trap/emulation of the counters.
> >
> > The current set of ratified RISC-V specification also doesn't allow scountovf
> > to be trap/emulated by the hypervisor. The SBI PMU snapshot bridges the gap
> > in ISA as well and enables perf sampling in the guest. However, LCOFI in the
> > guest only works via IRQ filtering in AIA specification. That's why, AIA
> > has to be enabled in the hardware (at least the Ssaia extension) in order to
> > use the sampling support in the perf.
> >
> > Here are the patch wise implementation details.
> >
> > PATCH 1,4,7,8,9,10,11,15 : Generic cleanups/improvements.
> > PATCH 2,3,14 : FW_READ_HI function implementation
> > PATCH 5-6: Add PMU snapshot feature in sbi pmu driver
> > PATCH 12-13: KVM implementation for snapshot and sampling in kvm guests
> > PATCH 16-17: Generic improvements for kvm selftests
> > PATCH 18-22: KVM selftests for SBI PMU extension
> >
> > The series is based on v6.9-rc4 and is available at:
> >
> > https://github.com/atishp04/linux/tree/kvm_pmu_snapshot_v8
> >
> > The kvmtool patch is also available at:
> > https://github.com/atishp04/kvmtool/tree/sscofpmf
> >
> > It also requires Ssaia ISA extension to be present in the hardware in order to
> > get perf sampling support in the guest. In Qemu virt machine, it can be done
> > by the following config.
> >
> > ```
> > -cpu rv64,sscofpmf=true,x-ssaia=true
> > ```
> >
> > There is no other dependencies on AIA apart from that. Thus, Ssaia must be disabled
> > for the guest if AIA patches are not available. Here is the example command.
> >
> > ```
> > ./lkvm-static run -m 256 -c2 --console serial -p "console=ttyS0 earlycon" --disable-ssaia -k ./Image --debug
> > ```
> >
> > The series has been tested only in Qemu.
> > Here is the snippet of the perf running inside a kvm guest.
> >
> > ===================================================
> > $ perf record -e cycles -e instructions perf bench sched messaging -g 5
> > ...
> > $ Running 'sched/messaging' benchmark:
> > ...
> > [ 45.928723] perf_duration_warn: 2 callbacks suppressed
> > [ 45.929000] perf: interrupt took too long (484426 > 483186), lowering kernel.perf_event_max_sample_rate to 250
> > $ 20 sender and receiver processes per group
> > $ 5 groups == 200 processes run
> >
> > Total time: 14.220 [sec]
> > [ perf record: Woken up 1 times to write data ]
> > [ perf record: Captured and wrote 0.117 MB perf.data (1942 samples) ]
> > $ perf report --stdio
> > $ To display the perf.data header info, please use --header/--header-only optio>
> > $
> > $
> > $ Total Lost Samples: 0
> > $
> > $ Samples: 943 of event 'cycles'
> > $ Event count (approx.): 5128976844
> > $
> > $ Overhead Command Shared Object Symbol >
> > $ ........ ............... ........................... .....................>
> > $
> > 7.59% sched-messaging [kernel.kallsyms] [k] memcpy
> > 5.48% sched-messaging [kernel.kallsyms] [k] percpu_counter_ad>
> > 5.24% sched-messaging [kernel.kallsyms] [k] __sbi_rfence_v02_>
> > 4.00% sched-messaging [kernel.kallsyms] [k] _raw_spin_unlock_>
> > 3.79% sched-messaging [kernel.kallsyms] [k] set_pte_range
> > 3.72% sched-messaging [kernel.kallsyms] [k] next_uptodate_fol>
> > 3.46% sched-messaging [kernel.kallsyms] [k] filemap_map_pages
> > 3.31% sched-messaging [kernel.kallsyms] [k] handle_mm_fault
> > 3.20% sched-messaging [kernel.kallsyms] [k] finish_task_switc>
> > 3.16% sched-messaging [kernel.kallsyms] [k] clear_page
> > 3.03% sched-messaging [kernel.kallsyms] [k] mtree_range_walk
> > 2.42% sched-messaging [kernel.kallsyms] [k] flush_icache_pte
> >
> > ===================================================
> >
> > [1] https://github.com/riscv-non-isa/riscv-sbi-doc
> >
> > Changes from v7->v8:
> > 1. Updated event states so that shared memory is updated only during stop
> > operations.
> > 2. Avoid clobbering lower XLEN counter/overflow values in shared memory
> > by maintaining a temporary copy for RV32.
> > 3. Improved overflow handling in snapshot case by supporting all 64 values.
> > 4. Minor cleanups based on suggestions on v7.
> >
> > Changes from v6->v7:
> > 1. Used SBI_SHMEM_DISABLE in the driver.
> > 2. Added RB Tags.
> > 3. Improved the sbi_pmu_test commandline to allow disabling multiple
> > tests.
> >
> > Changes from v5->v6:
> > 1. Added a patch for command line option for the sbi pmu tests.
> > 2. Removed redundant prints and restructure the code little bit.
> > 3. Added a patch for computing the sbi minor version correctly.
> > 4. Addressed all other comments on v5.
> >
> > Changes from v4->v5:
> > 1. Moved sbi related definitions to its own header file from processor.h
> > 2. Added few helper functions for selftests.
> > 3. Improved firmware counter read and RV32 start/stop functions.
> > 4. Converted all the shifting operations to use BIT macro
> > 5. Addressed all other comments on v4.
> >
> > Changes from v3->v4:
> > 1. Added selftests.
> > 2. Fixed an issue to clear the interrupt pending bits.
> > 3. Fixed the counter index in snapshot memory start function.
> >
> > Changes from v2->v3:
> > 1. Fixed a patchwork warning on patch6.
> > 2. Fixed a comment formatting & nit fix in PATCH 3 & 5.
> > 3. Moved the hvien update and sscofpmf enabling to PATCH 9 from PATCH 8.
> >
> > Changes from v1->v2:
> > 1. Fixed warning/errors from patchwork CI.
> > 2. Rebased on top of kvm-next.
> > 3. Added Acked-by tags.
> >
> > Changes from RFC->v1:
> > 1. Addressed all the comments on RFC series.
> > 2. Removed PATCH2 and merged into later patches.
> > 3. Added 2 more patches for minor fixes.
> > 4. Fixed KVM boot issue without Ssaia and made sscofpmf in guest dependent on
> > Ssaia in the host.
> >
> > Atish Patra (24):
> > RISC-V: Fix the typo in Scountovf CSR name
> > RISC-V: Add FIRMWARE_READ_HI definition
> > drivers/perf: riscv: Read upper bits of a firmware counter
> > drivers/perf: riscv: Use BIT macro for shifting operations
> > RISC-V: Add SBI PMU snapshot definitions
> > RISC-V: KVM: Rename the SBI_STA_SHMEM_DISABLE to a generic name
> > RISC-V: Use the minor version mask while computing sbi version
> > drivers/perf: riscv: Fix counter mask iteration for RV32
> > drivers/perf: riscv: Implement SBI PMU snapshot function
> > RISC-V: KVM: Fix the initial sample period value
> > RISC-V: KVM: No need to update the counter value during reset
> > RISC-V: KVM: No need to exit to the user space if perf event failed
> > RISC-V: KVM: Implement SBI PMU Snapshot feature
> > RISC-V: KVM: Add perf sampling support for guests
> > RISC-V: KVM: Support 64 bit firmware counters on RV32
> > RISC-V: KVM: Improve firmware counter read function
> > KVM: riscv: selftests: Move sbi definitions to its own header file
> > KVM: riscv: selftests: Add helper functions for extension checks
> > KVM: riscv: selftests: Add Sscofpmf to get-reg-list test
> > KVM: riscv: selftests: Add SBI PMU extension definitions
> > KVM: riscv: selftests: Add SBI PMU selftest
> > KVM: riscv: selftests: Add a test for PMU snapshot functionality
> > KVM: riscv: selftests: Add a test for counter overflow
> > KVM: riscv: selftests: Add commandline option for SBI PMU test
>
> Queued this series for Linux-6.10
>
> If new issues are discovered then send patches based on
> the KVM riscv queue.
Please use the kvm-riscv-for-palmer-6.10 tag in the KVM RISC-V
repo (https://github.com/kvm-riscv/linux.git) as a shared tag for
the upcoming Linux-6.10 merge window.
This tag is based on Linux-6.9-rc3.
Regards,
Anup
>
> Thanks,
> Anup
>
> >
> > arch/riscv/include/asm/csr.h | 5 +-
> > arch/riscv/include/asm/kvm_vcpu_pmu.h | 16 +-
> > arch/riscv/include/asm/sbi.h | 38 +-
> > arch/riscv/include/uapi/asm/kvm.h | 1 +
> > arch/riscv/kernel/paravirt.c | 6 +-
> > arch/riscv/kvm/aia.c | 5 +
> > arch/riscv/kvm/vcpu.c | 15 +-
> > arch/riscv/kvm/vcpu_onereg.c | 6 +
> > arch/riscv/kvm/vcpu_pmu.c | 260 ++++++-
> > arch/riscv/kvm/vcpu_sbi_pmu.c | 17 +-
> > arch/riscv/kvm/vcpu_sbi_sta.c | 4 +-
> > drivers/perf/riscv_pmu.c | 1 +
> > drivers/perf/riscv_pmu_sbi.c | 309 +++++++-
> > include/linux/perf/riscv_pmu.h | 6 +
> > tools/testing/selftests/kvm/Makefile | 1 +
> > .../selftests/kvm/include/riscv/processor.h | 49 +-
> > .../testing/selftests/kvm/include/riscv/sbi.h | 141 ++++
> > .../selftests/kvm/include/riscv/ucall.h | 1 +
> > .../selftests/kvm/lib/riscv/processor.c | 12 +
> > .../testing/selftests/kvm/riscv/arch_timer.c | 2 +-
> > .../selftests/kvm/riscv/get-reg-list.c | 4 +
> > .../selftests/kvm/riscv/sbi_pmu_test.c | 681 ++++++++++++++++++
> > tools/testing/selftests/kvm/steal_time.c | 4 +-
> > 23 files changed, 1467 insertions(+), 117 deletions(-)
> > create mode 100644 tools/testing/selftests/kvm/include/riscv/sbi.h
> > create mode 100644 tools/testing/selftests/kvm/riscv/sbi_pmu_test.c
> >
> > --
> > 2.34.1
> >
prev parent reply other threads:[~2024-04-30 5:54 UTC|newest]
Thread overview: 45+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-04-20 15:17 [PATCH v8 00/24] RISC-V SBI v2.0 PMU improvements and Perf sampling in KVM guest Atish Patra
2024-04-20 15:17 ` [PATCH v8 01/24] RISC-V: Fix the typo in Scountovf CSR name Atish Patra
2024-04-20 15:17 ` [PATCH v8 02/24] RISC-V: Add FIRMWARE_READ_HI definition Atish Patra
2024-04-20 15:17 ` [PATCH v8 03/24] drivers/perf: riscv: Read upper bits of a firmware counter Atish Patra
2024-04-20 15:17 ` [PATCH v8 04/24] drivers/perf: riscv: Use BIT macro for shifting operations Atish Patra
2024-04-20 15:17 ` [PATCH v8 05/24] RISC-V: Add SBI PMU snapshot definitions Atish Patra
2024-04-20 15:17 ` [PATCH v8 06/24] RISC-V: KVM: Rename the SBI_STA_SHMEM_DISABLE to a generic name Atish Patra
2024-04-20 15:17 ` [PATCH v8 07/24] RISC-V: Use the minor version mask while computing sbi version Atish Patra
2024-04-20 15:17 ` [PATCH v8 08/24] drivers/perf: riscv: Fix counter mask iteration for RV32 Atish Patra
2024-04-20 0:37 ` Samuel Holland
2024-04-20 1:08 ` Atish Kumar Patra
2024-04-20 15:17 ` [PATCH v8 09/24] drivers/perf: riscv: Implement SBI PMU snapshot function Atish Patra
2024-04-22 16:06 ` Samuel Holland
2024-04-23 20:34 ` Atish Kumar Patra
2024-04-20 15:17 ` [PATCH v8 10/24] RISC-V: KVM: Fix the initial sample period value Atish Patra
2024-04-22 5:24 ` Anup Patel
2024-04-20 15:17 ` [PATCH v8 11/24] RISC-V: KVM: No need to update the counter value during reset Atish Patra
2024-04-20 15:17 ` [PATCH v8 12/24] RISC-V: KVM: No need to exit to the user space if perf event failed Atish Patra
2024-04-20 15:17 ` [PATCH v8 13/24] RISC-V: KVM: Implement SBI PMU Snapshot feature Atish Patra
2024-04-20 15:17 ` [PATCH v8 14/24] RISC-V: KVM: Add perf sampling support for guests Atish Patra
2024-04-20 15:17 ` [PATCH v8 15/24] RISC-V: KVM: Support 64 bit firmware counters on RV32 Atish Patra
2024-04-20 15:17 ` [PATCH v8 16/24] RISC-V: KVM: Improve firmware counter read function Atish Patra
2024-04-22 5:26 ` Anup Patel
2024-04-20 15:17 ` [PATCH v8 17/24] KVM: riscv: selftests: Move sbi definitions to its own header file Atish Patra
2024-04-22 5:27 ` Anup Patel
2024-04-23 8:57 ` Muhammad Usama Anjum
2024-04-20 15:17 ` [PATCH v8 18/24] KVM: riscv: selftests: Add helper functions for extension checks Atish Patra
2024-04-22 5:29 ` Anup Patel
2024-04-23 8:57 ` Muhammad Usama Anjum
2024-04-20 15:17 ` [PATCH v8 19/24] KVM: riscv: selftests: Add Sscofpmf to get-reg-list test Atish Patra
2024-04-23 8:58 ` Muhammad Usama Anjum
2024-04-20 15:17 ` [PATCH v8 20/24] KVM: riscv: selftests: Add SBI PMU extension definitions Atish Patra
2024-04-23 8:58 ` Muhammad Usama Anjum
2024-04-20 15:17 ` [PATCH v8 21/24] KVM: riscv: selftests: Add SBI PMU selftest Atish Patra
2024-04-23 8:59 ` Muhammad Usama Anjum
2024-04-20 15:17 ` [PATCH v8 22/24] KVM: riscv: selftests: Add a test for PMU snapshot functionality Atish Patra
2024-04-23 9:00 ` Muhammad Usama Anjum
2024-04-20 15:17 ` [PATCH v8 23/24] KVM: riscv: selftests: Add a test for counter overflow Atish Patra
2024-04-23 9:00 ` Muhammad Usama Anjum
2024-04-20 15:17 ` [PATCH v8 24/24] KVM: riscv: selftests: Add commandline option for SBI PMU test Atish Patra
2024-04-22 5:32 ` Anup Patel
2024-04-23 9:03 ` Muhammad Usama Anjum
2024-04-23 10:56 ` Andrew Jones
2024-04-22 9:59 ` [PATCH v8 00/24] RISC-V SBI v2.0 PMU improvements and Perf sampling in KVM guest Anup Patel
2024-04-30 5:54 ` Anup Patel [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAAhSdy0k=-wtge-iA-rckQrd_ziNtWDnf4J-yCKP0OooZCaPzA@mail.gmail.com' \
--to=anup@brainfault.org \
--cc=ajay.kaher@broadcom.com \
--cc=ajones@ventanamicro.com \
--cc=alexghiti@rivosinc.com \
--cc=aou@eecs.berkeley.edu \
--cc=atishp@rivosinc.com \
--cc=conor.dooley@microchip.com \
--cc=jgross@suse.com \
--cc=kvm-riscv@lists.infradead.org \
--cc=kvm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=linux-riscv@lists.infradead.org \
--cc=mark.rutland@arm.com \
--cc=palmer@dabbelt.com \
--cc=palmer@rivosinc.com \
--cc=paul.walmsley@sifive.com \
--cc=pbonzini@redhat.com \
--cc=samuel.holland@sifive.com \
--cc=shuah@kernel.org \
--cc=virtualization@lists.linux.dev \
--cc=will@kernel.org \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).