From: "Ilpo Järvinen" <ilpo.jarvinen@linux.intel.com>
To: linux-pci@vger.kernel.org, "Bjorn Helgaas" <bhelgaas@google.com>,
"Lorenzo Pieralisi" <lorenzo.pieralisi@arm.com>,
"Rob Herring" <robh@kernel.org>,
"Krzysztof Wilczyński" <kw@linux.com>,
"Maciej W . Rozycki" <macro@orcam.me.uk>,
"Jonathan Cameron" <Jonathan.Cameron@Huawei.com>,
"Lukas Wunner" <lukas@wunner.de>,
"Alexandru Gagniuc" <mr.nuke.me@gmail.com>,
"Krishna chaitanya chundru" <quic_krichai@quicinc.com>,
"Srinivas Pandruvada" <srinivas.pandruvada@linux.intel.com>,
"Rafael J. Wysocki" <rafael@kernel.org>,
linux-pm@vger.kernel.org,
"Ilpo Järvinen" <ilpo.jarvinen@linux.intel.com>,
"Daniel Lezcano" <daniel.lezcano@linaro.org>,
"Zhang Rui" <rui.zhang@intel.com>,
"Lukasz Luba" <lukasz.luba@arm.com>,
linux-kernel@vger.kernel.org
Cc: Amit Kucheria <amitk@kernel.org>,
Christophe JAILLET <christophe.jaillet@wanadoo.fr>,
Jonathan Cameron <Jonathan.Cameron@huawei.com>
Subject: [PATCH v6 7/8] thermal: Add PCIe cooling driver
Date: Thu, 16 May 2024 12:32:21 +0300 [thread overview]
Message-ID: <20240516093222.1684-8-ilpo.jarvinen@linux.intel.com> (raw)
In-Reply-To: <20240516093222.1684-1-ilpo.jarvinen@linux.intel.com>
Add a thermal cooling driver to provide path to access PCIe bandwidth
controller using the usual thermal interfaces.
A cooling device is instantiated for controllable PCIe Ports from the
bwctrl service driver.
If registering the cooling device fails, allow bwctrl's probe to
succeed regardless. As cdev in that case contains IS_ERR() pseudo
"pointer", clean that up inside the probe function so the remove side
doesn't need to suddenly make an odd looking IS_ERR() check.
The thermal side state 0 means no throttling, i.e., maximum supported
PCIe Link Speed.
Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Acked-by: Rafael J. Wysocki <rafael@kernel.org> # From the cooling device interface perspective
---
MAINTAINERS | 2 +
drivers/pci/pcie/bwctrl.c | 11 +++++
drivers/thermal/Kconfig | 10 +++++
drivers/thermal/Makefile | 2 +
drivers/thermal/pcie_cooling.c | 80 ++++++++++++++++++++++++++++++++++
include/linux/pci-bwctrl.h | 28 ++++++++++++
6 files changed, 133 insertions(+)
create mode 100644 drivers/thermal/pcie_cooling.c
create mode 100644 include/linux/pci-bwctrl.h
diff --git a/MAINTAINERS b/MAINTAINERS
index bd420cb09821..3a94ae81b13f 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -17104,6 +17104,8 @@ M: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
L: linux-pci@vger.kernel.org
S: Supported
F: drivers/pci/pcie/bwctrl.c
+F: drivers/thermal/pcie_cooling.c
+F: include/linux/pci-bwctrl.h
PCIE DRIVER FOR AMAZON ANNAPURNA LABS
M: Jonathan Chocron <jonnyc@amazon.com>
diff --git a/drivers/pci/pcie/bwctrl.c b/drivers/pci/pcie/bwctrl.c
index c44da36ef81a..c1d0ccbcc3ab 100644
--- a/drivers/pci/pcie/bwctrl.c
+++ b/drivers/pci/pcie/bwctrl.c
@@ -27,6 +27,7 @@
#include <linux/interrupt.h>
#include <linux/mutex.h>
#include <linux/pci.h>
+#include <linux/pci-bwctrl.h>
#include <linux/rwsem.h>
#include <linux/slab.h>
#include <linux/types.h>
@@ -38,10 +39,12 @@
* struct pcie_bwctrl_data - PCIe bandwidth controller
* @set_speed_mutex: Serializes link speed changes
* @lbms_count: Count for LBMS (since last reset)
+ * @cdev: thermal cooling device associated with the port
*/
struct pcie_bwctrl_data {
struct mutex set_speed_mutex;
atomic_t lbms_count;
+ struct thermal_cooling_device *cdev;
};
static bool pcie_valid_speed(enum pci_bus_speed speed)
@@ -290,6 +293,11 @@ static int pcie_bwnotif_probe(struct pcie_device *srv)
pci_dbg(port, "enabled with IRQ %d\n", srv->irq);
+ /* Don't fail on errors. Don't leave IS_ERR() "pointer" into ->cdev */
+ port->link_bwctrl->cdev = pcie_cooling_device_register(port);
+ if (IS_ERR(port->link_bwctrl->cdev))
+ port->link_bwctrl->cdev = NULL;
+
return 0;
}
@@ -297,6 +305,9 @@ static void pcie_bwnotif_remove(struct pcie_device *srv)
{
struct pcie_bwctrl_data *data = get_service_data(srv);
+ if (data->cdev)
+ pcie_cooling_device_unregister(data->cdev);
+
pcie_bwnotif_disable(srv->port);
scoped_guard(rwsem_write, &pcie_bwctrl_remove_rwsem)
srv->port->link_bwctrl = NULL;
diff --git a/drivers/thermal/Kconfig b/drivers/thermal/Kconfig
index 204ed89a3ec9..7ddc44be4701 100644
--- a/drivers/thermal/Kconfig
+++ b/drivers/thermal/Kconfig
@@ -211,6 +211,16 @@ config DEVFREQ_THERMAL
If you want this support, you should say Y here.
+config PCIE_THERMAL
+ bool "PCIe cooling support"
+ depends on PCIEPORTBUS
+ select PCIE_BWCTRL
+ help
+ This implements PCIe cooling mechanism through bandwidth reduction
+ for PCIe devices.
+
+ If you want this support, you should say Y here.
+
config THERMAL_EMULATION
bool "Thermal emulation mode support"
help
diff --git a/drivers/thermal/Makefile b/drivers/thermal/Makefile
index 5cdf7d68687f..5b9bf8e80eb6 100644
--- a/drivers/thermal/Makefile
+++ b/drivers/thermal/Makefile
@@ -31,6 +31,8 @@ thermal_sys-$(CONFIG_CPU_IDLE_THERMAL) += cpuidle_cooling.o
# devfreq cooling
thermal_sys-$(CONFIG_DEVFREQ_THERMAL) += devfreq_cooling.o
+thermal_sys-$(CONFIG_PCIE_THERMAL) += pcie_cooling.o
+
obj-$(CONFIG_K3_THERMAL) += k3_bandgap.o k3_j72xx_bandgap.o
# platform thermal drivers
obj-y += broadcom/
diff --git a/drivers/thermal/pcie_cooling.c b/drivers/thermal/pcie_cooling.c
new file mode 100644
index 000000000000..a876d64f1582
--- /dev/null
+++ b/drivers/thermal/pcie_cooling.c
@@ -0,0 +1,80 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * PCIe cooling device
+ *
+ * Copyright (C) 2023-2024 Intel Corporation
+ */
+
+#include <linux/build_bug.h>
+#include <linux/cleanup.h>
+#include <linux/err.h>
+#include <linux/module.h>
+#include <linux/pci.h>
+#include <linux/pci-bwctrl.h>
+#include <linux/slab.h>
+#include <linux/sprintf.h>
+#include <linux/thermal.h>
+
+#define COOLING_DEV_TYPE_PREFIX "PCIe_Port_Link_Speed_"
+
+static int pcie_cooling_get_max_level(struct thermal_cooling_device *cdev, unsigned long *state)
+{
+ struct pci_dev *port = cdev->devdata;
+
+ /* cooling state 0 is same as the maximum PCIe speed */
+ *state = port->subordinate->max_bus_speed - PCIE_SPEED_2_5GT;
+
+ return 0;
+}
+
+static int pcie_cooling_get_cur_level(struct thermal_cooling_device *cdev, unsigned long *state)
+{
+ struct pci_dev *port = cdev->devdata;
+
+ /* cooling state 0 is same as the maximum PCIe speed */
+ *state = cdev->max_state - (port->subordinate->cur_bus_speed - PCIE_SPEED_2_5GT);
+
+ return 0;
+}
+
+static int pcie_cooling_set_cur_level(struct thermal_cooling_device *cdev, unsigned long state)
+{
+ struct pci_dev *port = cdev->devdata;
+ enum pci_bus_speed speed;
+
+ /* cooling state 0 is same as the maximum PCIe speed */
+ speed = (cdev->max_state - state) + PCIE_SPEED_2_5GT;
+
+ return pcie_set_target_speed(port, speed, true);
+}
+
+static struct thermal_cooling_device_ops pcie_cooling_ops = {
+ .get_max_state = pcie_cooling_get_max_level,
+ .get_cur_state = pcie_cooling_get_cur_level,
+ .set_cur_state = pcie_cooling_set_cur_level,
+};
+
+struct thermal_cooling_device *pcie_cooling_device_register(struct pci_dev *port)
+{
+ char *name __free(kfree) =
+ kasprintf(GFP_KERNEL, COOLING_DEV_TYPE_PREFIX "%s", pci_name(port));
+ if (!name)
+ return ERR_PTR(-ENOMEM);
+
+ return thermal_cooling_device_register(name, port, &pcie_cooling_ops);
+}
+
+void pcie_cooling_device_unregister(struct thermal_cooling_device *cdev)
+{
+ thermal_cooling_device_unregister(cdev);
+}
+
+/* For bus_speed <-> state arithmetic */
+static_assert(PCIE_SPEED_2_5GT + 1 == PCIE_SPEED_5_0GT);
+static_assert(PCIE_SPEED_5_0GT + 1 == PCIE_SPEED_8_0GT);
+static_assert(PCIE_SPEED_8_0GT + 1 == PCIE_SPEED_16_0GT);
+static_assert(PCIE_SPEED_16_0GT + 1 == PCIE_SPEED_32_0GT);
+static_assert(PCIE_SPEED_32_0GT + 1 == PCIE_SPEED_64_0GT);
+
+MODULE_AUTHOR("Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>");
+MODULE_DESCRIPTION("PCIe cooling driver");
diff --git a/include/linux/pci-bwctrl.h b/include/linux/pci-bwctrl.h
new file mode 100644
index 000000000000..cee07127455b
--- /dev/null
+++ b/include/linux/pci-bwctrl.h
@@ -0,0 +1,28 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * PCIe bandwidth controller
+ *
+ * Copyright (C) 2023-2024 Intel Corporation
+ */
+
+#ifndef LINUX_PCI_BWCTRL_H
+#define LINUX_PCI_BWCTRL_H
+
+#include <linux/pci.h>
+
+struct thermal_cooling_device;
+
+#ifdef CONFIG_PCIE_THERMAL
+struct thermal_cooling_device *pcie_cooling_device_register(struct pci_dev *port);
+void pcie_cooling_device_unregister(struct thermal_cooling_device *cdev);
+#else
+static inline struct thermal_cooling_device *pcie_cooling_device_register(struct pci_dev *port)
+{
+ return NULL;
+}
+static inline void pcie_cooling_device_unregister(struct thermal_cooling_device *cdev)
+{
+}
+#endif
+
+#endif
--
2.39.2
next prev parent reply other threads:[~2024-05-16 9:34 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-05-16 9:32 [PATCH v6 0/8] PCI: Add PCIe bandwidth controller Ilpo Järvinen
2024-05-16 9:32 ` [PATCH v6 1/8] PCI: Protect Link Control 2 Register with RMW locking Ilpo Järvinen
2024-05-16 9:32 ` [PATCH v6 2/8] PCI: Store all PCIe Supported Link Speeds Ilpo Järvinen
2024-05-16 9:32 ` [PATCH v6 3/8] PCI: Refactor pcie_update_link_speed() Ilpo Järvinen
2024-05-16 9:32 ` [PATCH v6 4/8] PCI/quirks: Abstract LBMS seen check into own function Ilpo Järvinen
2024-05-16 9:32 ` [PATCH v6 5/8] PCI/bwctrl: Re-add BW notification portdrv as PCIe BW controller Ilpo Järvinen
2024-05-16 9:32 ` [PATCH v6 6/8] PCI/bwctrl: Add API to set PCIe Link Speed Ilpo Järvinen
2024-05-16 9:32 ` Ilpo Järvinen [this message]
2024-05-16 9:32 ` [PATCH v6 8/8] selftests/pcie_bwctrl: Create selftests Ilpo Järvinen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240516093222.1684-8-ilpo.jarvinen@linux.intel.com \
--to=ilpo.jarvinen@linux.intel.com \
--cc=Jonathan.Cameron@Huawei.com \
--cc=amitk@kernel.org \
--cc=bhelgaas@google.com \
--cc=christophe.jaillet@wanadoo.fr \
--cc=daniel.lezcano@linaro.org \
--cc=kw@linux.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=linux-pm@vger.kernel.org \
--cc=lorenzo.pieralisi@arm.com \
--cc=lukas@wunner.de \
--cc=lukasz.luba@arm.com \
--cc=macro@orcam.me.uk \
--cc=mr.nuke.me@gmail.com \
--cc=quic_krichai@quicinc.com \
--cc=rafael@kernel.org \
--cc=robh@kernel.org \
--cc=rui.zhang@intel.com \
--cc=srinivas.pandruvada@linux.intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).