From: Jesse Brandeburg <jesse.brandeburg@intel.com>
To: Jay Vosburgh <jay.vosburgh@canonical.com>,
Federico Parola <federico.parola@polito.it>
Cc: Maciej Fijalkowski <maciej.fijalkowski@intel.com>,
<xdp-newbies@vger.kernel.org>,
Piotr Raczynski <piotr.raczynski@intel.com>
Subject: Re: AF_XDP busy poll receives packets in batches of 8 on i40e
Date: Fri, 18 Mar 2022 12:29:27 -0700 [thread overview]
Message-ID: <cf883b44-c192-cbcd-e430-1247b2335cf2@intel.com> (raw)
In-Reply-To: <20413.1647628390@famine>
On 3/18/2022 11:33 AM, Jay Vosburgh wrote:
>> I tested it on 3 different XL710 NICs, all updated to the latest 8.50
>> firmware, and they all present the same 8 packets "batching" behavior.
>> How do I check the PCI Cache Line Size (I didn't find such entry in lspci
>> -vvv)? Normal cache line size is 64B on my system, but I guess they are
>> two different things.
>
> It's in the lspci -vvv, near the top, e.g.,
>
> 08:00.0 Ethernet controller: Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ (rev 02)
> Subsystem: Intel Corporation Ethernet Converged Network Adapter XL710-Q2
> Physical Slot: 1
> Control: [...bunch of stuff...]
> Status: [...bunch of stuff...]
> Latency: 0, Cache Line Size: 64 bytes
>
> Checking the PCIe spec, though, I'm not sure why this would
> matter, as PCIe 4.0 7.5.1.3 Cache Line Size Register says it "has no
> effect on any PCI Express device behavior."
Thanks Jay, that might just be the sound of my brain short circuiting,
as I thought that held some relevance for some reason. Our receive
descriptors are 16 bytes (with options for 32 bytes sometimes), and the
transmit are 16. 8*16=128, so I was suspecting something related to
cache line size in the PCIe is triggering a writeback when we get to
that boundary, it's probably some other trigger I don't remember.
I know the XL710 has a documented erratum where while interrupts are
disabled (like while polling) that hardware won't write back until it
hits a writeback criteria, or unless the interrupt is re-enabled. We
have a bunch of workarounds in the driver trying to address this, I was
thinking maybe one of those workarounds isn't hitting in the XDP code.
Oh, and WB_ON_ITR didn't appear until the X722 generation, so it's not
there on XL710. Hope this helps!
Jesse
next prev parent reply other threads:[~2022-03-18 19:29 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-03-17 14:36 AF_XDP busy poll receives packets in batches of 8 on i40e Federico Parola
2022-03-17 14:59 ` Jay Vosburgh
2022-03-17 15:34 ` Federico Parola
2022-03-17 16:41 ` Jesse Brandeburg
2022-03-17 23:26 ` Jay Vosburgh
2022-03-18 10:30 ` Federico Parola
2022-03-18 18:33 ` Jay Vosburgh
2022-03-18 19:29 ` Jesse Brandeburg [this message]
2022-03-17 16:59 ` Jay Vosburgh
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=cf883b44-c192-cbcd-e430-1247b2335cf2@intel.com \
--to=jesse.brandeburg@intel.com \
--cc=federico.parola@polito.it \
--cc=jay.vosburgh@canonical.com \
--cc=maciej.fijalkowski@intel.com \
--cc=piotr.raczynski@intel.com \
--cc=xdp-newbies@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).