From: Alex Williamson <alex.williamson@redhat.com>
To: Yi Liu <yi.l.liu@intel.com>
Cc: "Tian, Kevin" <kevin.tian@intel.com>,
Jason Gunthorpe <jgg@nvidia.com>,
"joro@8bytes.org" <joro@8bytes.org>,
"robin.murphy@arm.com" <robin.murphy@arm.com>,
"eric.auger@redhat.com" <eric.auger@redhat.com>,
"nicolinc@nvidia.com" <nicolinc@nvidia.com>,
"kvm@vger.kernel.org" <kvm@vger.kernel.org>,
"chao.p.peng@linux.intel.com" <chao.p.peng@linux.intel.com>,
"iommu@lists.linux.dev" <iommu@lists.linux.dev>,
"baolu.lu@linux.intel.com" <baolu.lu@linux.intel.com>,
"Duan, Zhenzhong" <zhenzhong.duan@intel.com>,
"Pan, Jacob jun" <jacob.jun.pan@intel.com>
Subject: Re: [PATCH v2 0/4] vfio-pci support pasid attach/detach
Date: Thu, 18 Apr 2024 14:37:47 -0600 [thread overview]
Message-ID: <20240418143747.28b36750.alex.williamson@redhat.com> (raw)
In-Reply-To: <4037d5f4-ae6b-4c17-97d8-e0f7812d5a6d@intel.com>
On Thu, 18 Apr 2024 17:03:15 +0800
Yi Liu <yi.l.liu@intel.com> wrote:
> On 2024/4/18 08:06, Tian, Kevin wrote:
> >> From: Alex Williamson <alex.williamson@redhat.com>
> >> Sent: Thursday, April 18, 2024 7:02 AM
> >>
> >> On Wed, 17 Apr 2024 09:20:51 -0300
> >> Jason Gunthorpe <jgg@nvidia.com> wrote:
> >>
> >>> On Wed, Apr 17, 2024 at 07:16:05AM +0000, Tian, Kevin wrote:
> >>>>> From: Jason Gunthorpe <jgg@nvidia.com>
> >>>>> Sent: Wednesday, April 17, 2024 1:50 AM
> >>>>>
> >>>>> On Tue, Apr 16, 2024 at 08:38:50AM +0000, Tian, Kevin wrote:
> >>>>>>> From: Liu, Yi L <yi.l.liu@intel.com>
> >>>>>>> Sent: Friday, April 12, 2024 4:21 PM
> >>>>>>>
> >>>>>>> A userspace VMM is supposed to get the details of the device's
> >> PASID
> >>>>>>> capability
> >>>>>>> and assemble a virtual PASID capability in a proper offset in the
> >> virtual
> >>>>> PCI
> >>>>>>> configuration space. While it is still an open on how to get the
> >> available
> >>>>>>> offsets. Devices may have hidden bits that are not in the PCI cap
> >> chain.
> >>>>> For
> >>>>>>> now, there are two options to get the available offsets.[2]
> >>>>>>>
> >>>>>>> - Report the available offsets via ioctl. This requires device-specific
> >> logic
> >>>>>>> to provide available offsets. e.g., vfio-pci variant driver. Or may the
> >>>>> device
> >>>>>>> provide the available offset by DVSEC.
> >>>>>>> - Store the available offsets in a static table in userspace VMM.
> >> VMM gets
> >>>>> the
> >>>>>>> empty offsets from this table.
> >>>>>>>
> >>>>>>
> >>>>>> I'm not a fan of requesting a variant driver for every PASID-capable
> >>>>>> VF just for the purpose of reporting a free range in the PCI config
> >> space.
> >>>>>>
> >>>>>> It's easier to do that quirk in userspace.
> >>>>>>
> >>>>>> But I like Alex's original comment that at least for PF there is no
> >> reason
> >>>>>> to hide the offset. there could be a flag+field to communicate it. or
> >>>>>> if there will be a new variant VF driver for other purposes e.g.
> >> migration
> >>>>>> it can certainly fill the field too.
> >>>>>
> >>>>> Yes, since this has been such a sticking point can we get a clean
> >>>>> series that just enables it for PF and then come with a solution for
> >>>>> VF?
> >>>>>
> >>>>
> >>>> sure but we at least need to reach consensus on a minimal required
> >>>> uapi covering both PF/VF to move forward so the user doesn't need
> >>>> to touch different contracts for PF vs. VF.
> >>>
> >>> Do we? The situation where the VMM needs to wholly make a up a PASID
> >>> capability seems completely new and seperate from just using an
> >>> existing PASID capability as in the PF case.
> >>
> >> But we don't actually expose the PASID capability on the PF and as
> >> argued in path 4/ we can't because it would break existing userspace.
> > > Come back to this statement.
> >
> > Does 'break' means that legacy Qemu will crash due to a guest write
> > to the read-only PASID capability, or just a conceptually functional
> > break i.e. non-faithful emulation due to writes being dropped?
I expect more the latter.
> > If the latter it's probably not a bad idea to allow exposing the PASID
> > capability on the PF as a sane guest shouldn't enable the PASID
> > capability w/o seeing vIOMMU supporting PASID. And there is no
> > status bit defined in the PASID capability to check back so even
> > if an insane guest wants to blindly enable PASID it will naturally
> > write and done. The only niche case is that the enable bits are
> > defined as RW so ideally reading back those bits should get the
> > latest written value. But probably this can be tolerated?
Some degree of inconsistency is likely tolerated, the guest is unlikely
to check that a RW bit was set or cleared. How would we virtualize the
control registers for a VF and are they similarly virtualized for a PF
or would we allow the guest to manipulate the physical PASID control
registers?
> > With that then should we consider exposing the PASID capability
> > in PCI config space as the first option? For PF it's simple as how
> > other caps are exposed. For VF a variant driver can also fake the
> > PASID capability or emulate a DVSEC capability for unused space
> > (to motivate the physical implementation so no variant driver is
> > required in the future)
>
> If kernel exposes pasid cap for PF same as other caps, and in the meantime
> the variant driver chooses to emulate a DVSEC cap, then userspace follows
> the below steps to expose pasid cap to VM.
If we have a variant driver, why wouldn't it expose an emulated PASID
capability rather than a DVSEC if we're choosing to expose PASID for
PFs?
>
> 1) Check if a pasid cap is already present in the virtual config space
> read from kernel. If no, but user wants pasid, then goto step 2).
> 2) Userspace invokes VFIO_DEVICE_FETURE to check if the device support
> pasid cap. If yes, goto step 3).
Why do we need the vfio feature interface if a physical or virtual PASID
capability on the device exposes the same info?
> 3) Userspace gets an available offset via reading the DVSEC cap.
What's the scenario where we'd have a VF wanting to expose PASID
support which doesn't also have a variant driver that could implement a
virtual PASID?
> 4) Userspace assembles a pasid cap and inserts it to the vconfig space.
>
> For PF, step 1) is enough. For VF, it needs to go through all the 4 steps.
> This is a bit different from what we planned at the beginning. But sounds
> doable if we want to pursue the staging direction.
Seems like if we decide that we can just expose the PASID capability
for a PF then we should just have any VF variant drivers also implement
a virtual PASID capability. In this case DVSEC would only be used to
provide information for a purely userspace emulation of PASID (in which
case it also wouldn't necessarily need the vfio feature because it
might implicitly know the PASID capabilities of the device). Thanks,
Alex
next prev parent reply other threads:[~2024-04-18 20:37 UTC|newest]
Thread overview: 72+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-04-12 8:21 [PATCH v2 0/4] vfio-pci support pasid attach/detach Yi Liu
2024-04-12 8:21 ` [PATCH v2 1/4] ida: Add ida_get_lowest() Yi Liu
2024-04-16 16:03 ` Alex Williamson
2024-04-18 7:02 ` Yi Liu
2024-04-18 16:23 ` Alex Williamson
2024-04-18 17:12 ` Jason Gunthorpe
2024-04-19 13:43 ` Yi Liu
2024-04-19 13:55 ` Alex Williamson
2024-04-19 14:00 ` Jason Gunthorpe
2024-04-23 7:19 ` Yi Liu
2024-04-19 13:40 ` Yi Liu
2024-04-12 8:21 ` [PATCH v2 2/4] vfio-iommufd: Support pasid [at|de]tach for physical VFIO devices Yi Liu
2024-04-16 9:01 ` Tian, Kevin
2024-04-16 9:24 ` Yi Liu
2024-04-16 9:47 ` Tian, Kevin
2024-04-18 7:04 ` Yi Liu
2024-04-23 12:43 ` Jason Gunthorpe
2024-04-24 0:33 ` Tian, Kevin
2024-04-24 4:48 ` Yi Liu
2024-04-12 8:21 ` [PATCH v2 3/4] vfio: Add VFIO_DEVICE_PASID_[AT|DE]TACH_IOMMUFD_PT Yi Liu
2024-04-16 9:13 ` Tian, Kevin
2024-04-16 9:36 ` Yi Liu
2024-04-23 12:45 ` Jason Gunthorpe
2024-04-12 8:21 ` [PATCH v2 4/4] vfio: Report PASID capability via VFIO_DEVICE_FEATURE ioctl Yi Liu
2024-04-16 9:40 ` Tian, Kevin
2024-04-16 17:57 ` Alex Williamson
2024-04-17 7:09 ` Tian, Kevin
2024-04-17 20:25 ` Alex Williamson
2024-04-18 0:21 ` Tian, Kevin
2024-04-18 8:23 ` Yi Liu
2024-04-18 16:34 ` Alex Williamson
2024-04-23 12:39 ` Jason Gunthorpe
2024-04-24 0:24 ` Tian, Kevin
2024-04-24 13:59 ` Jason Gunthorpe
2024-04-16 8:38 ` [PATCH v2 0/4] vfio-pci support pasid attach/detach Tian, Kevin
2024-04-16 17:50 ` Jason Gunthorpe
2024-04-17 7:16 ` Tian, Kevin
2024-04-17 12:20 ` Jason Gunthorpe
2024-04-17 23:02 ` Alex Williamson
2024-04-18 0:06 ` Tian, Kevin
2024-04-18 9:03 ` Yi Liu
2024-04-18 20:37 ` Alex Williamson [this message]
2024-04-19 5:52 ` Tian, Kevin
2024-04-19 16:35 ` Alex Williamson
2024-04-23 7:43 ` Tian, Kevin
2024-04-23 12:01 ` Jason Gunthorpe
2024-04-23 23:47 ` Tian, Kevin
2024-04-24 0:12 ` Jason Gunthorpe
2024-04-24 2:57 ` Tian, Kevin
2024-04-24 12:29 ` Baolu Lu
2024-04-24 14:04 ` Jason Gunthorpe
2024-04-24 5:19 ` Tian, Kevin
2024-04-24 14:15 ` Jason Gunthorpe
2024-04-24 18:38 ` Alex Williamson
2024-04-24 18:45 ` Jason Gunthorpe
2024-04-24 18:24 ` Alex Williamson
2024-04-24 18:36 ` Jason Gunthorpe
2024-04-24 20:13 ` Alex Williamson
2024-04-26 14:11 ` Jason Gunthorpe
2024-04-26 20:13 ` Alex Williamson
2024-04-28 6:19 ` Tian, Kevin
2024-04-29 7:43 ` Yi Liu
2024-04-29 17:15 ` Jason Gunthorpe
2024-04-29 17:44 ` Jason Gunthorpe
2024-04-27 5:05 ` Christoph Hellwig
2024-04-25 9:26 ` Yi Liu
2024-04-25 12:58 ` Alex Williamson
2024-04-26 9:01 ` Yi Liu
2024-04-19 13:59 ` Jason Gunthorpe
2024-04-23 7:58 ` Yi Liu
2024-04-23 12:05 ` Jason Gunthorpe
2024-04-19 13:34 ` Jason Gunthorpe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240418143747.28b36750.alex.williamson@redhat.com \
--to=alex.williamson@redhat.com \
--cc=baolu.lu@linux.intel.com \
--cc=chao.p.peng@linux.intel.com \
--cc=eric.auger@redhat.com \
--cc=iommu@lists.linux.dev \
--cc=jacob.jun.pan@intel.com \
--cc=jgg@nvidia.com \
--cc=joro@8bytes.org \
--cc=kevin.tian@intel.com \
--cc=kvm@vger.kernel.org \
--cc=nicolinc@nvidia.com \
--cc=robin.murphy@arm.com \
--cc=yi.l.liu@intel.com \
--cc=zhenzhong.duan@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).