From: Przemek Kitszel <przemyslaw.kitszel@intel.com>
To: Jakub Kicinski <kuba@kernel.org>, Jiri Pirko <jiri@resnulli.us>
Cc: Pawel Chmielewski <pawel.chmielewski@intel.com>,
"netdev@vger.kernel.org" <netdev@vger.kernel.org>,
"Knitter, Konrad" <konrad.knitter@intel.com>,
Ahmed Zaki <ahmed.zaki@intel.com>,
"Samudrala, Sridhar" <sridhar.samudrala@intel.com>,
"intel-wired-lan@lists.osuosl.org"
<intel-wired-lan@lists.osuosl.org>,
Simon Horman <horms@kernel.org>,
Mateusz Polchlopek <mateusz.polchlopek@intel.com>,
"Nguyen, Anthony L" <anthony.l.nguyen@intel.com>,
Paolo Abeni <pabeni@redhat.com>
Subject: Re: [Intel-wired-lan] [RFC net-next (what uAPI?) ice: add support for more than 16 RSS queues for VF
Date: Mon, 6 May 2024 16:34:38 +0200 [thread overview]
Message-ID: <9be9efee-6adc-4812-b07b-007c0922183a@intel.com> (raw)
In-Reply-To: <20240429185941.6229b944@kernel.org>
On 4/30/24 03:59, Jakub Kicinski wrote:
> On Fri, 26 Apr 2024 15:22:02 +0200 Przemek Kitszel wrote:
>> ## devlink resources (with current API)
>> `devlink resource` is compelling, partially given the name sounds like a
>> perfect match. But when we dig just a little bit, the current Path+sizes
>> (min,max,step) is totally off to what is the most elegant picture of the
>> situation. In order to fit into existing uAPI, I would need to register
>> VFs as PF's resource, then GLOBAL LUT and PF LUT as a sub resource to
>> that (each VF gets two entries under it; plus two additional ones for
>> PF) I don't like it, I also feel like there is not that much use of
>> current resources API (it's not natural to use it for distribution, only
>> for limitation).
>
> Can you share more on how that would look like?
something like below (though I have added one more layer to illustrate
broader idea, it could be chopped down)
[1]
>
> From the description it does not sound so bad. Maybe with some CLI / UI
> changes it will be fine?
>
>> ## devlink resources (with extended API)
>> It is possible to extend current `devlink resource` so instead of only
>> Path+size, there would be also Path+Owner option to use.
>> The default state for ice driver would be that PFs owns PF LUTs, GLOBAL
>> LUTs are all free.
>>
>> example proposed flow to assign a GLOBAL LUT to VF0 and PF LUT to VF1:
>> pf=0000:03:00.0 # likely more meaningful than VSI idx, but open for
>> vf0=0000:03:00.1 # suggestions
>> vf1=0000:03:00.2
>> devlink resource set pci/$pf path /lut/lut_table_512 owner $pf
>> devlink resource set pci/$pf path /lut/lut_table_2048 owner free
>> devlink resource set pci/$pf path /lut/lut_table_512 owner $vf0
>> devlink resource set pci/$pf path /lut/lut_table_2048 owner $vf1
>
> Don't we want some level of over-subscription to be allowed?
In theory we could reuse/share tables, especially with the case of auto
filled ones, not sure if I would want to implement that with the very
first series, but good point to design interface with that in mind.
To move in this direction we could make numbered LUTs an entity that is
set/show'ed (digression: this would feel more like mmap() than malloc())
(The imaginary output below does not do that).
The main problem with the [1] above for "current API" for me is lack of
aggregate device [2] in the structures represented by devlink. Let me
show what it would be with one more layer (so I put PFs under that, and
VFs one layer down).
Here is an example with 2 PFs, one of with with 3 VFs, each of them with
different LUT
$ devlink resource show $device
$device:
name lut size 4 unit entry
resources:
name lut_table_2048 size 2 unit entry size_min 0 size_max 8
size_gran 1
name lut_table_512 size 2 unit entry size_min 0 size_max 16
size_gran 1
name functions size 5 unit entry
resources:
name pf0
resources:
name lut_table_2048 size 0 size_max 1 ...
name lut_table_512 size 1 size_max 1 ...
name vf0
resources:
name lut_table_2048 size 1 size_max 1 ...
name lut_table_512 size 0 size_max 1 ...
name vf1
resources:
name lut_table_2048 size 0 size_max 1 ...
name lut_table_512 size 1 size_max 1 ...
name vf2
resources:
name lut_table_2048 size 0 size_max 1 ...
name lut_table_512 size 0 size_max 1 ...
name pf1
resources:
name lut_table_2048 size 1 ...
name lut_table_512 size 0 ...
where $device stands for the aggregate device (covers 8 PFs in case of
8-port split used)
and all named PF/VFs in the output would need "dummy" size calculations
(I would like to remove this need though)
When all resources are assigned, all "size 0" entries should have also
"size_max 0" to indicate no room for growth.
[2] aggregate device discussion:
https://lore.kernel.org/intel-wired-lan/cfe84890-f999-4b97-a012-6f9fd16da8e3@intel.com/
prev parent reply other threads:[~2024-05-06 14:34 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-04-26 13:22 [Intel-wired-lan] [RFC net-next (what uAPI?) ice: add support for more than 16 RSS queues for VF Przemek Kitszel
2024-04-30 1:59 ` Jakub Kicinski
2024-05-06 14:34 ` Przemek Kitszel [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=9be9efee-6adc-4812-b07b-007c0922183a@intel.com \
--to=przemyslaw.kitszel@intel.com \
--cc=ahmed.zaki@intel.com \
--cc=anthony.l.nguyen@intel.com \
--cc=horms@kernel.org \
--cc=intel-wired-lan@lists.osuosl.org \
--cc=jiri@resnulli.us \
--cc=konrad.knitter@intel.com \
--cc=kuba@kernel.org \
--cc=mateusz.polchlopek@intel.com \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=pawel.chmielewski@intel.com \
--cc=sridhar.samudrala@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).