From: Wen Gu <guwen@linux.alibaba.com>
To: Jan Karcher <jaka@linux.ibm.com>,
wintera@linux.ibm.com, twinkler@linux.ibm.com, hca@linux.ibm.com,
gor@linux.ibm.com, agordeev@linux.ibm.com, davem@davemloft.net,
edumazet@google.com, kuba@kernel.org, pabeni@redhat.com,
wenjia@linux.ibm.com
Cc: borntraeger@linux.ibm.com, svens@linux.ibm.com,
alibuda@linux.alibaba.com, tonylu@linux.alibaba.com,
linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org,
netdev@vger.kernel.org
Subject: Re: [PATCH net-next v3 01/11] net/smc: adapt SMC-D device dump for Emulated-ISM
Date: Fri, 15 Mar 2024 20:29:26 +0800 [thread overview]
Message-ID: <d69885a1-c85f-4d26-a615-fbf6968e2c40@linux.alibaba.com> (raw)
In-Reply-To: <ef98b16d-2965-4297-a2ed-143b0b592a25@linux.ibm.com>
On 2024/3/15 18:27, Jan Karcher wrote:
>
>
> On 15/03/2024 04:44, Wen Gu wrote:
>>
>>
>> On 2024/3/14 18:23, Jan Karcher wrote:
>>>
>>>
>>> On 12/03/2024 15:27, Wen Gu wrote:
>>>> The introduction of Emulated-ISM requires adaptation of SMC-D device
>>>> dump. Software implemented non-PCI device (loopback-ism) should be
>>>> handled correctly and the CHID reserved for Emulated-ISM should be got
>>>> from smcd_ops interface instead of PCI information.
>>>>
>>>> Signed-off-by: Wen Gu <guwen@linux.alibaba.com>
>>>> ---
>>>> net/smc/smc_ism.c | 13 ++++++++++---
>>>> 1 file changed, 10 insertions(+), 3 deletions(-)
>>>>
>>>> diff --git a/net/smc/smc_ism.c b/net/smc/smc_ism.c
>>>> index ac88de2a06a0..b6eca4231913 100644
>>>> --- a/net/smc/smc_ism.c
>>>> +++ b/net/smc/smc_ism.c
>>>> @@ -252,12 +252,11 @@ static int smc_nl_handle_smcd_dev(struct smcd_dev *smcd,
>>>> char smc_pnet[SMC_MAX_PNETID_LEN + 1];
>>>> struct smc_pci_dev smc_pci_dev;
>>>> struct nlattr *port_attrs;
>>>> + struct device *device;
>>>> struct nlattr *attrs;
>>>> - struct ism_dev *ism;
>>>> int use_cnt = 0;
>>>> void *nlh;
>>>> - ism = smcd->priv;
>>>> nlh = genlmsg_put(skb, NETLINK_CB(cb->skb).portid, cb->nlh->nlmsg_seq,
>>>> &smc_gen_nl_family, NLM_F_MULTI,
>>>> SMC_NETLINK_GET_DEV_SMCD);
>>>> @@ -272,7 +271,15 @@ static int smc_nl_handle_smcd_dev(struct smcd_dev *smcd,
>>>> if (nla_put_u8(skb, SMC_NLA_DEV_IS_CRIT, use_cnt > 0))
>>>> goto errattr;
>>>> memset(&smc_pci_dev, 0, sizeof(smc_pci_dev));
>>>> - smc_set_pci_values(to_pci_dev(ism->dev.parent), &smc_pci_dev);
>>>> + device = smcd->ops->get_dev(smcd);
>>>> + if (device->parent)
>>>> + smc_set_pci_values(to_pci_dev(device->parent), &smc_pci_dev);
>>>> + if (smc_ism_is_emulated(smcd)) {
>>>> + smc_pci_dev.pci_pchid = smc_ism_get_chid(smcd);
>>>> + if (!device->parent)
>>>> + snprintf(smc_pci_dev.pci_id, sizeof(smc_pci_dev.pci_id),
>>>> + "%s", dev_name(device));
>>>> + }
>>>
>>> Hi Wen Gu,
>>>
>>> playing around with the loopback-ism device and testing it, i developed some concerns regarding this exposure.
>>> Basically this enables us to see the loopback device in the `smcd device` tool without any changes.
>>> E.g.:
>>> ```
>>> # smcd device
>>> FID Type PCI-ID PCHID InUse #LGs PNET-ID
>>> 0000 0 loopback-ism ffff No 0
>>> 102a ISM 0000:00:00.0 07c2 No 0 NET1
>>> ```
>>>
>>> Now the problem with this is that "loopback-ism" is no valid PCI-ID and should not be there. My first thought was to
>>> put an "n/a" instead, but that opens up another problem. Currently you could set - even if it does not make sense - a
>>> PNET_ID for the loopback device:
>>> ```
>>
>> Yes, and we can exclude loopback-ism in smc_pnet_enter() if necessary.
>
> We could, but we have to make sure we implement those distinctions at the right location. E.g.: if virtio-ism is coming.
> Does a PNETID make sense for a virtio-ism device? Do we want to exclude is also there or do we want an abstracted
> layer/mechanism to recognize if a device has a PNETId capability or not?
>
Understand, a long-term view is better.
>>
>>> # smc_pnet -a -D loopback-ism NET1
>>> # smcd device
>>> FID Type PCI-ID PCHID InUse #LGs PNET-ID
>>> 0000 0 loopback-ism ffff No 0 *NET1
>>> 102a ISM 0000:00:00.0 07c2 No 0 NET1
>>> ```
>>> If we would change the PCI-ID to "n/a" it would be a valid input parameter for the tooling which is... to put it
>>> nice... not so beautiful.
>>
>> FID Type PCI-ID PCHID InUse #LGs PNET-ID
>> 0000 0 n/a ffff No 0
>>
>> IIUC, the problem is that the 'n/a', which would be an input for other tools, is somewhat strange?
>
> Exactly.
>
>>
>> Since PCHID 0xffff is clear defined for loopback-ism, I am wondering if it can be a specific sign
>> of loopback-ism for tooling to not take PCI-ID into account? Would you also consider that inelegant?
>
> I think deciding on PCHID is the only way we can currently differentiate what kind of device we are talking about. So my
> guess would be that PCHID is going to play a big role in future design decisions.
>
Make sense to me.
>>
>>> I brainstormed this with them team and the problem is more complex.
>>> In theory there shouldn't be PCI information set for the loopback device. There should be a better abstraction in the
>>> netlink interface that creates this output and the tooling should be made aware of it.
>>>
>>
>> Yes, it is better. But I couldn't surely picture how the abstraction looks like, and if it is necessary
>> to introduce it just for a special case of loopback-ism (note that virtio-ISM also has PCI information),
>> since we can identify loopback-ism by CHID.
>
> Please take the following with a grain of salt. I just want to give you a bit insight of our current train of thought.
> None of it is final or set in stone. The idea would be to have device core information that contain the information
> which other fields are important for this device. And corresponding "extensions" that contain the information. The
> tooling cvould then decide soley on the core information which features are supported by a device and which are not.
> If that is really needed: Not sure yet. Is this the best solution: Propably not.
> E.g.:
>
> SMC-D netlink abstraction
>
> +------------------------------------+
> | Core information |
> | (e.g. PCHID, InUse, isPCI, isS390) |
> +------------------------------------+
>
> +----------------+
> | s390 extension |
> | (e.g.FID) |
> +----------------+
>
> +--------------------+
> | PCI extension |
> | (e.g. PCI-ID, ...) |
> +--------------------+
>
>
I like this diagram and it is very clear. So core information will be applicable to all ISM devices,
and the extension information will be specific to some certain kinds.
>>
>>> Do you rely on the output currently? What are your thoughts about it?
>>> If not I'd ask you to not fill the netlink interface for the loopback device and refactor it with the next stage when
>>> we create a right interface for it.
>>>
>>
>> Currently we don't rely on the output, and I have no objection to the proposal that not fill the netlink
>> interface for loopback-ism and refactor it in the next stage.
>
> Thank you! If you have ideas regarding the design of the interface hit us up. As soon as we are going to think about
> this further I'm going to invite you to those discussions.
>
Sure! and thank you very much!
Best regards,
Wen Gu
>> >>> Since the Merge-Window is closed feel free to send new versions as RFC.
>>
>> OK, I will send the new version as an RFC.
>>
>> Thank you!
>
> Thanks!
> - Jan
>
>>
>>> Thank you
>>> - Jan
>>>
>>>> if (nla_put_u32(skb, SMC_NLA_DEV_PCI_FID, smc_pci_dev.pci_fid))
>>>> goto errattr;
>>>> if (nla_put_u16(skb, SMC_NLA_DEV_PCI_CHID, smc_pci_dev.pci_pchid))
next prev parent reply other threads:[~2024-03-15 12:29 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-03-12 14:27 [PATCH net-next v3 00/11] net/smc: SMC intra-OS shortcut with loopback-ism Wen Gu
2024-03-12 14:27 ` [PATCH net-next v3 01/11] net/smc: adapt SMC-D device dump for Emulated-ISM Wen Gu
2024-03-14 10:23 ` Jan Karcher
2024-03-15 3:44 ` Wen Gu
2024-03-15 10:27 ` Jan Karcher
2024-03-15 12:29 ` Wen Gu [this message]
2024-03-12 14:27 ` [PATCH net-next v3 02/11] net/smc: decouple ism_client from SMC-D DMB registration Wen Gu
2024-03-12 14:27 ` [PATCH net-next v3 03/11] net/smc: introduce loopback-ism for SMC intra-OS shortcut Wen Gu
2024-03-12 14:27 ` [PATCH net-next v3 04/11] net/smc: implement ID-related operations of loopback-ism Wen Gu
2024-03-12 14:27 ` [PATCH net-next v3 05/11] net/smc: implement some unsupported " Wen Gu
2024-03-12 14:27 ` [PATCH net-next v3 06/11] net/smc: implement DMB-related " Wen Gu
2024-03-12 14:27 ` [PATCH net-next v3 07/11] net/smc: register loopback-ism into SMC-D device list Wen Gu
2024-03-12 14:27 ` [PATCH net-next v3 08/11] net/smc: add operations to merge sndbuf with peer DMB Wen Gu
2024-03-12 14:27 ` [PATCH net-next v3 09/11] net/smc: attach or detach ghost sndbuf to " Wen Gu
2024-03-12 14:27 ` [PATCH net-next v3 10/11] net/smc: adapt cursor update when sndbuf and peer DMB are merged Wen Gu
2024-03-12 14:27 ` [PATCH net-next v3 11/11] net/smc: implement DMB-merged operations of loopback-ism Wen Gu
2024-03-12 14:33 ` [PATCH net-next v3 00/11] net/smc: SMC intra-OS shortcut with loopback-ism Jan Karcher
2024-03-12 14:43 ` Wen Gu
2024-03-12 15:00 ` Paolo Abeni
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=d69885a1-c85f-4d26-a615-fbf6968e2c40@linux.alibaba.com \
--to=guwen@linux.alibaba.com \
--cc=agordeev@linux.ibm.com \
--cc=alibuda@linux.alibaba.com \
--cc=borntraeger@linux.ibm.com \
--cc=davem@davemloft.net \
--cc=edumazet@google.com \
--cc=gor@linux.ibm.com \
--cc=hca@linux.ibm.com \
--cc=jaka@linux.ibm.com \
--cc=kuba@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-s390@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=svens@linux.ibm.com \
--cc=tonylu@linux.alibaba.com \
--cc=twinkler@linux.ibm.com \
--cc=wenjia@linux.ibm.com \
--cc=wintera@linux.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).