From: Kalle Valo <kvalo@kernel.org>
To: Baochen Qiang <quic_bqiang@quicinc.com>
Cc: James Prestwood <prestwoj@gmail.com>,
<linux-wireless@vger.kernel.org>, <ath11k@lists.infradead.org>,
David Woodhouse <dwmw2@infradead.org>,
iommu@lists.linux.dev
Subject: Re: ath11k and vfio-pci support
Date: Sun, 14 Jan 2024 16:36:02 +0200 [thread overview]
Message-ID: <87il3w7zjh.fsf@kernel.org> (raw)
In-Reply-To: <3d22a730-aee5-4f2a-9ddc-b4b5bd4d62fe@quicinc.com> (Baochen Qiang's message of "Sun, 14 Jan 2024 20:37:03 +0800")
Baochen Qiang <quic_bqiang@quicinc.com> writes:
>>> Strange that still fails. Are you now seeing this error in your
>>> host or your Qemu? or both?
>>> Could you share your test steps? And if you can share please be as
>>> detailed as possible since I'm not familiar with passing WLAN
>>> hardware to a VM using vfio-pci.
>>
>> Just in Qemu, the hardware works fine on my host machine.
>> I basically follow this guide to set it up, its written in the
>> context of GPUs/libvirt but the host setup is exactly the same. By
>> no means do you need to read it all, once you set the vfio-pci.ids
>> and see your unclaimed adapter you can stop:
>> https://wiki.archlinux.org/title/PCI_passthrough_via_OVMF
>> In short you should be able to set the following host kernel options
>> and reboot (assuming your motherboard/hardware is compatible):
>> intel_iommu=on iommu=pt vfio-pci.ids=17cb:1103
>> Obviously change the device/vendor IDs to whatever ath11k hw you
>> have. Once the host is rebooted you should see your wlan adapter as
>> UNCLAIMED, showing the driver in use as vfio-pci. If not, its likely
>> your motherboard just isn't compatible, the device has to be in its
>> own IOMMU group (you could try switching PCI ports if this is the
>> case).
>> I then build a "kvm_guest.config" kernel with the driver/firmware
>> for ath11k and boot into that with the following Qemu options:
>> -enable-kvm -device -vfio-pci,host=<PCI address>
>> If it seems easier you could also utilize IWD's test-runner which
>> handles launching the Qemu kernel automatically, detecting any
>> vfio-devices and passes them through and mounts some useful host
>> folders into the VM. Its actually a very good general purpose tool
>> for kernel testing, not just for IWD:
>> https://git.kernel.org/pub/scm/network/wireless/iwd.git/tree/doc/test-runner.txt
>> Once set up you can just run test-runner with a few flags and you'll
>> boot into a shell:
>> ./tools/test-runner -k <kernel-image> --hw --start /bin/bash
>> Please reach out if you have questions, thanks for looking into
>> this.
>
> Thanks for these details. I reproduced this issue by following your guide.
>
> Seems the root cause is that the MSI vector assigned to WCN6855 in
> qemu is different with that in host. In my case the MSI vector in qemu
> is [Address: fee00000 Data: 0020] while in host it is [Address:
> fee00578 Data: 0000]. So in qemu ath11k configures MSI vector
> [Address: fee00000 Data: 0020] to WCN6855 hardware/firmware, and
> firmware uses that vector to fire interrupts to host/qemu. However
> host IOMMU doesn't know that vector because the real vector is
> [Address: fee00578 Data: 0000], as a result host blocks that
> interrupt and reports an error, see below log:
>
> [ 1414.206069] DMAR: DRHD: handling fault status reg 2
> [ 1414.206081] DMAR: [INTR-REMAP] Request device [02:00.0] fault index
> 0x0 [fault reason 0x25] Blocked a compatibility format interrupt
> request
> [ 1414.210334] DMAR: DRHD: handling fault status reg 2
> [ 1414.210342] DMAR: [INTR-REMAP] Request device [02:00.0] fault index
> 0x0 [fault reason 0x25] Blocked a compatibility format interrupt
> request
> [ 1414.212496] DMAR: DRHD: handling fault status reg 2
> [ 1414.212503] DMAR: [INTR-REMAP] Request device [02:00.0] fault index
> 0x0 [fault reason 0x25] Blocked a compatibility format interrupt
> request
> [ 1414.214600] DMAR: DRHD: handling fault status reg 2
>
> While I don't think there is a way for qemu/ath11k to get the real MSI
> vector from host, I will try to read the vfio code to check further.
> Before that, to unblock you, a possible hack is to hard code the MSI
> vector in qemu to the same as in host, on condition that the MSI
> vector doesn't change.
Baochen, awesome that you were able to debug this further. Now we at
least know what's the problem.
I'll add David Woodhouse and the iommu list in hopes that they could
give us ideas how to solve this. Full thread here:
https://lore.kernel.org/all/adcb785e-4dc7-4c4a-b341-d53b72e13467@gmail.com/
--
https://patchwork.kernel.org/project/linux-wireless/list/
https://wireless.wiki.kernel.org/en/developers/documentation/submittingpatches
next parent reply other threads:[~2024-01-14 14:36 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <adcb785e-4dc7-4c4a-b341-d53b72e13467@gmail.com>
[not found] ` <8734v5zhol.fsf@kernel.org>
[not found] ` <87fa5220-6fd9-433d-879b-c55ac67a0748@gmail.com>
[not found] ` <87r0ipcn7j.fsf@kernel.org>
[not found] ` <356e0b05-f396-4ad7-9b29-c492b54af834@gmail.com>
[not found] ` <26119c3f-9012-47bb-948e-7e976d4773a7@quicinc.com>
[not found] ` <87mstccmk6.fsf@kernel.org>
[not found] ` <df9fd970-5af3-468c-b1f1-18f91215cf44@gmail.com>
[not found] ` <8734v4auc4.fsf@kernel.org>
[not found] ` <e8878979-1f3f-4635-a716-9ac381c617d9@gmail.com>
[not found] ` <285b84d0-229c-4c83-a7d6-4c3c23139597@quicinc.com>
[not found] ` <4607fb37-8227-49a3-9e8c-10c9b117ec7b@gmail.com>
[not found] ` <3d22a730-aee5-4f2a-9ddc-b4b5bd4d62fe@quicinc.com>
2024-01-14 14:36 ` Kalle Valo [this message]
2024-01-15 17:46 ` ath11k and vfio-pci support Alex Williamson
2024-01-16 10:08 ` Baochen Qiang
2024-01-16 10:41 ` David Woodhouse
2024-01-16 15:29 ` Jason Gunthorpe
2024-01-16 18:28 ` Alex Williamson
2024-01-16 21:10 ` Jeff Johnson
2024-01-17 5:47 ` Baochen Qiang
2024-03-21 19:14 ` Johannes Berg
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87il3w7zjh.fsf@kernel.org \
--to=kvalo@kernel.org \
--cc=ath11k@lists.infradead.org \
--cc=dwmw2@infradead.org \
--cc=iommu@lists.linux.dev \
--cc=linux-wireless@vger.kernel.org \
--cc=prestwoj@gmail.com \
--cc=quic_bqiang@quicinc.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).