From: "Zeng, Xin" <xin.zeng@intel.com>
To: liulongfang <liulongfang@huawei.com>,
"herbert@gondor.apana.org.au" <herbert@gondor.apana.org.au>,
"alex.williamson@redhat.com" <alex.williamson@redhat.com>,
"jgg@nvidia.com" <jgg@nvidia.com>,
"yishaih@nvidia.com" <yishaih@nvidia.com>,
"shameerali.kolothum.thodi@huawei.com"
<shameerali.kolothum.thodi@huawei.com>,
"Tian, Kevin" <kevin.tian@intel.com>
Cc: "linux-crypto@vger.kernel.org" <linux-crypto@vger.kernel.org>,
"kvm@vger.kernel.org" <kvm@vger.kernel.org>,
qat-linux <qat-linux@intel.com>
Subject: RE: [PATCH v5 09/10] crypto: qat - implement interface for live migration
Date: Tue, 30 Apr 2024 09:49:40 +0000 [thread overview]
Message-ID: <DM4PR11MB55026F0FF713BCB3317F5AC7881A2@DM4PR11MB5502.namprd11.prod.outlook.com> (raw)
In-Reply-To: <7cedf6ec-3c8a-b6d8-d5fc-778554c011c2@huawei.com>
On Tuesday, April 30, 2024 11:10 AM, liulongfang <liulongfang@huawei.com> wrote:
> To: Zeng, Xin <xin.zeng@intel.com>; herbert@gondor.apana.org.au;
> alex.williamson@redhat.com; jgg@nvidia.com; yishaih@nvidia.com;
> shameerali.kolothum.thodi@huawei.com; Tian, Kevin <kevin.tian@intel.com>
> Cc: linux-crypto@vger.kernel.org; kvm@vger.kernel.org; qat-linux <qat-
> linux@intel.com>
> Subject: Re: [PATCH v5 09/10] crypto: qat - implement interface for live
> migration
>
> On 2024/3/6 21:58, Xin Zeng wrote:
> > Add logic to implement the interface for live migration defined in
> > qat/qat_mig_dev.h. This is specific for QAT GEN4 Virtual Functions
> > (VFs).
> >
> > This introduces a migration data manager which is used to handle the
> > device state during migration. The manager ensures that the device state
> > is stored in a format that can be restored in the destination node.
> >
> > The VF state is organized into a hierarchical structure that includes a
> > preamble, a general state section, a MISC bar section and an ETR bar
> > section. The latter contains the state of the 4 ring pairs contained on
> > a VF. Here is a graphical representation of the state:
> >
> > preamble | general state section | leaf state
> > | MISC bar state section| leaf state
> > | ETR bar state section | bank0 state section | leaf state
> > | bank1 state section | leaf state
> > | bank2 state section | leaf state
> > | bank3 state section | leaf state
> >
> > In addition to the implementation of the qat_migdev_ops interface and
> > the state manager framework, add a mutex in pfvf to avoid pf2vf messages
> > during migration.
> >
> > Signed-off-by: Xin Zeng <xin.zeng@intel.com>
> > Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
> > ---
> > .../intel/qat/qat_420xx/adf_420xx_hw_data.c | 2 +
> > .../intel/qat/qat_4xxx/adf_4xxx_hw_data.c | 2 +
> > drivers/crypto/intel/qat/qat_common/Makefile | 2 +
> > .../intel/qat/qat_common/adf_accel_devices.h | 6 +
> > .../intel/qat/qat_common/adf_gen4_hw_data.h | 10 +
> > .../intel/qat/qat_common/adf_gen4_vf_mig.c | 1010 +++++++++++++++++
> > .../intel/qat/qat_common/adf_mstate_mgr.c | 318 ++++++
> > .../intel/qat/qat_common/adf_mstate_mgr.h | 89 ++
> > .../crypto/intel/qat/qat_common/adf_sriov.c | 7 +-
> > 9 files changed, 1445 insertions(+), 1 deletion(-)
> > create mode 100644
> drivers/crypto/intel/qat/qat_common/adf_gen4_vf_mig.c
> > create mode 100644
> drivers/crypto/intel/qat/qat_common/adf_mstate_mgr.c
> > create mode 100644
> drivers/crypto/intel/qat/qat_common/adf_mstate_mgr.h
> >
> > diff --git a/drivers/crypto/intel/qat/qat_420xx/adf_420xx_hw_data.c
> b/drivers/crypto/intel/qat/qat_420xx/adf_420xx_hw_data.c
> > index 9ccbf5998d5c..d255cb3ebd9c 100644
> > --- a/drivers/crypto/intel/qat/qat_420xx/adf_420xx_hw_data.c
> > +++ b/drivers/crypto/intel/qat/qat_420xx/adf_420xx_hw_data.c
> > @@ -17,6 +17,7 @@
> > +void adf_gen4_init_vf_mig_ops(struct qat_migdev_ops *vfmig_ops)
> > +{
> > + vfmig_ops->init = adf_gen4_vfmig_init_device;
> > + vfmig_ops->cleanup = adf_gen4_vfmig_cleanup_device;
> > + vfmig_ops->reset = adf_gen4_vfmig_reset_device;
> > + vfmig_ops->open = adf_gen4_vfmig_open_device;
> > + vfmig_ops->close = adf_gen4_vfmig_close_device;
> > + vfmig_ops->suspend = adf_gen4_vfmig_suspend_device;
> > + vfmig_ops->resume = adf_gen4_vfmig_resume_device;
> > + vfmig_ops->save_state = adf_gen4_vfmig_save_state;
> > + vfmig_ops->load_state = adf_gen4_vfmig_load_state;
> > + vfmig_ops->load_setup = adf_gen4_vfmig_load_setup;
> > + vfmig_ops->save_setup = adf_gen4_vfmig_save_setup;
> > +}
> > +EXPORT_SYMBOL_GPL(adf_gen4_init_vf_mig_ops);
>
> This GEN4 device supports live migration functionality.
> The above part of the code supports the live migration function and
> has nothing to do with crypto.
>
> Therefore, these should be moved to the vfio/pci/qat directory.
>
Thanks for the suggestion, but
1. The migration operations of QAT VF rely on QAT PF driver sitting in
crypto tree to handle. Some of the states can only be accessed from PF by
PF driver. For each generation of PF, we will have a PF driver. It is obvious
more clear and nature to make these operations part QAT PF driver rather
than the variant VF driver.
2. The interfaces are defined clear enough to understand the dependency
of the variant migration VF driver to QAT PF driver.
3. A device driver sitting in crypto tree usually not only provides crypto stuff, but
also provide helpers to support other functionalities such as the non vfio
use space process access logic, it does make sense to provide migration helpers in
device driver as well.
Those why I prefer to put these helpers into QAT PF driver.
Thanks
next prev parent reply other threads:[~2024-04-30 9:49 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-03-06 13:58 [PATCH v5 00/10] crypto: qat - enable QAT GEN4 SRIOV VF live migration for QAT GEN4 Xin Zeng
2024-03-06 13:58 ` [PATCH v5 01/10] crypto: qat - adf_get_etr_base() helper Xin Zeng
2024-03-06 13:58 ` [PATCH v5 02/10] crypto: qat - relocate and rename 4xxx PF2VM definitions Xin Zeng
2024-03-06 13:58 ` [PATCH v5 03/10] crypto: qat - move PFVF compat checker to a function Xin Zeng
2024-03-06 13:58 ` [PATCH v5 04/10] crypto: qat - relocate CSR access code Xin Zeng
2024-03-06 13:58 ` [PATCH v5 05/10] crypto: qat - rename get_sla_arr_of_type() Xin Zeng
2024-03-06 13:58 ` [PATCH v5 06/10] crypto: qat - expand CSR operations for QAT GEN4 devices Xin Zeng
2024-03-06 13:58 ` [PATCH v5 07/10] crypto: qat - add bank save and restore flows Xin Zeng
2024-03-06 13:58 ` [PATCH v5 08/10] crypto: qat - add interface for live migration Xin Zeng
2024-03-08 16:52 ` Jason Gunthorpe
2024-03-11 17:57 ` Cabiddu, Giovanni
2024-03-06 13:58 ` [PATCH v5 09/10] crypto: qat - implement " Xin Zeng
2024-04-30 3:10 ` liulongfang
2024-04-30 9:49 ` Zeng, Xin [this message]
2024-03-06 13:58 ` [PATCH v5 10/10] vfio/qat: Add vfio_pci driver for Intel QAT VF devices Xin Zeng
2024-03-08 7:56 ` Tian, Kevin
2024-03-28 10:51 ` [PATCH v5 00/10] crypto: qat - enable QAT GEN4 SRIOV VF live migration for QAT GEN4 Herbert Xu
2024-03-28 15:03 ` Alex Williamson
2024-04-02 2:52 ` Herbert Xu
2024-04-12 14:19 ` Cabiddu, Giovanni
2024-04-12 22:59 ` Alex Williamson
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=DM4PR11MB55026F0FF713BCB3317F5AC7881A2@DM4PR11MB5502.namprd11.prod.outlook.com \
--to=xin.zeng@intel.com \
--cc=alex.williamson@redhat.com \
--cc=herbert@gondor.apana.org.au \
--cc=jgg@nvidia.com \
--cc=kevin.tian@intel.com \
--cc=kvm@vger.kernel.org \
--cc=linux-crypto@vger.kernel.org \
--cc=liulongfang@huawei.com \
--cc=qat-linux@intel.com \
--cc=shameerali.kolothum.thodi@huawei.com \
--cc=yishaih@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).