From: Mateusz Polchlopek <mateusz.polchlopek@intel.com>
To: intel-wired-lan@lists.osuosl.org
Cc: andrew@lunn.ch, jiri@resnulli.us, michal.wilczynski@intel.com,
Mateusz Polchlopek <mateusz.polchlopek@intel.com>,
netdev@vger.kernel.org, lukasz.czapnik@intel.com,
victor.raj@intel.com, Jiri Pirko <jiri@nvidia.com>,
anthony.l.nguyen@intel.com, horms@kernel.org,
przemyslaw.kitszel@intel.com, kuba@kernel.org
Subject: [Intel-wired-lan] [PATCH net-next v10 6/6] ice: Document tx_scheduling_layers parameter
Date: Fri, 19 Apr 2024 04:08:54 -0400 [thread overview]
Message-ID: <20240419080854.10000-7-mateusz.polchlopek@intel.com> (raw)
In-Reply-To: <20240419080854.10000-1-mateusz.polchlopek@intel.com>
From: Michal Wilczynski <michal.wilczynski@intel.com>
New driver specific parameter 'tx_scheduling_layers' was introduced.
Describe parameter in the documentation.
Signed-off-by: Michal Wilczynski <michal.wilczynski@intel.com>
Acked-by: Jakub Kicinski <kuba@kernel.org>
Reviewed-by: Jiri Pirko <jiri@nvidia.com>
Reviewed-by: Przemek Kitszel <przemyslaw.kitszel@intel.com>
Co-developed-by: Mateusz Polchlopek <mateusz.polchlopek@intel.com>
Signed-off-by: Mateusz Polchlopek <mateusz.polchlopek@intel.com>
---
Documentation/networking/devlink/ice.rst | 47 ++++++++++++++++++++++++
1 file changed, 47 insertions(+)
diff --git a/Documentation/networking/devlink/ice.rst b/Documentation/networking/devlink/ice.rst
index 7f30ebd5debb..830c04354222 100644
--- a/Documentation/networking/devlink/ice.rst
+++ b/Documentation/networking/devlink/ice.rst
@@ -21,6 +21,53 @@ Parameters
* - ``enable_iwarp``
- runtime
- mutually exclusive with ``enable_roce``
+ * - ``tx_scheduling_layers``
+ - permanent
+ - The ice hardware uses hierarchical scheduling for Tx with a fixed
+ number of layers in the scheduling tree. Each of them are decision
+ points. Root node represents a port, while all the leaves represent
+ the queues. This way of configuring the Tx scheduler allows features
+ like DCB or devlink-rate (documented below) to configure how much
+ bandwidth is given to any given queue or group of queues, enabling
+ fine-grained control because scheduling parameters can be configured
+ at any given layer of the tree.
+
+ The default 9-layer tree topology was deemed best for most workloads,
+ as it gives an optimal ratio of performance to configurability. However,
+ for some specific cases, this 9-layer topology might not be desired.
+ One example would be sending traffic to queues that are not a multiple
+ of 8. Because the maximum radix is limited to 8 in 9-layer topology,
+ the 9th queue has a different parent than the rest, and it's given
+ more bandwidth credits. This causes a problem when the system is
+ sending traffic to 9 queues:
+
+ | tx_queue_0_packets: 24163396
+ | tx_queue_1_packets: 24164623
+ | tx_queue_2_packets: 24163188
+ | tx_queue_3_packets: 24163701
+ | tx_queue_4_packets: 24163683
+ | tx_queue_5_packets: 24164668
+ | tx_queue_6_packets: 23327200
+ | tx_queue_7_packets: 24163853
+ | tx_queue_8_packets: 91101417 < Too much traffic is sent from 9th
+
+ To address this need, you can switch to a 5-layer topology, which
+ changes the maximum topology radix to 512. With this enhancement,
+ the performance characteristic is equal as all queues can be assigned
+ to the same parent in the tree. The obvious drawback of this solution
+ is a lower configuration depth of the tree.
+
+ Use the ``tx_scheduling_layer`` parameter with the devlink command
+ to change the transmit scheduler topology. To use 5-layer topology,
+ use a value of 5. For example:
+ $ devlink dev param set pci/0000:16:00.0 name tx_scheduling_layers
+ value 5 cmode permanent
+ Use a value of 9 to set it back to the default value.
+
+ You must do PCI slot powercycle for the selected topology to take effect.
+
+ To verify that value has been set:
+ $ devlink dev param show pci/0000:16:00.0 name tx_scheduling_layers
Info versions
=============
--
2.38.1
prev parent reply other threads:[~2024-04-19 8:18 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-04-19 8:08 [Intel-wired-lan] [PATCH net-next v10 0/6] ice: Support 5 layer Tx scheduler topology Mateusz Polchlopek
2024-04-19 8:08 ` [Intel-wired-lan] [PATCH net-next v10 1/6] devlink: extend devlink_param *set pointer Mateusz Polchlopek
2024-04-19 8:08 ` [Intel-wired-lan] [PATCH net-next v10 2/6] ice: Support 5 layer topology Mateusz Polchlopek
2024-04-19 8:08 ` [Intel-wired-lan] [PATCH net-next v10 3/6] ice: Adjust the VSI/Aggregator layers Mateusz Polchlopek
2024-04-19 8:08 ` [Intel-wired-lan] [PATCH net-next v10 4/6] ice: Enable switching default Tx scheduler topology Mateusz Polchlopek
2024-04-19 8:08 ` [Intel-wired-lan] [PATCH net-next v10 5/6] ice: Add tx_scheduling_layers devlink param Mateusz Polchlopek
2024-04-19 8:08 ` Mateusz Polchlopek [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240419080854.10000-7-mateusz.polchlopek@intel.com \
--to=mateusz.polchlopek@intel.com \
--cc=andrew@lunn.ch \
--cc=anthony.l.nguyen@intel.com \
--cc=horms@kernel.org \
--cc=intel-wired-lan@lists.osuosl.org \
--cc=jiri@nvidia.com \
--cc=jiri@resnulli.us \
--cc=kuba@kernel.org \
--cc=lukasz.czapnik@intel.com \
--cc=michal.wilczynski@intel.com \
--cc=netdev@vger.kernel.org \
--cc=przemyslaw.kitszel@intel.com \
--cc=victor.raj@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).