* [PATCH net-next v2 0/3] bnxt: add rx discards stats for oom and netpool
@ 2021-08-26 1:47 Jakub Kicinski
2021-08-26 1:47 ` [PATCH net-next v2 1/3] bnxt: reorder logic in bnxt_get_stats64() Jakub Kicinski
` (2 more replies)
0 siblings, 3 replies; 5+ messages in thread
From: Jakub Kicinski @ 2021-08-26 1:47 UTC (permalink / raw
To: davem, michael.chan; +Cc: netdev, olteanv, Jakub Kicinski
Drivers should avoid silently dropping frames. This set adds two
stats for previously unaccounted events to bnxt - packets dropped
due to allocation failures and packets dropped during emergency
ring polling.
Jakub Kicinski (3):
bnxt: reorder logic in bnxt_get_stats64()
bnxt: count packets discarded because of netpoll
bnxt: count discards due to memory allocation errors
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 54 ++++++++++++++-----
drivers/net/ethernet/broadcom/bnxt/bnxt.h | 3 ++
.../net/ethernet/broadcom/bnxt/bnxt_ethtool.c | 6 +++
3 files changed, 50 insertions(+), 13 deletions(-)
--
2.31.1
^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH net-next v2 1/3] bnxt: reorder logic in bnxt_get_stats64()
2021-08-26 1:47 [PATCH net-next v2 0/3] bnxt: add rx discards stats for oom and netpool Jakub Kicinski
@ 2021-08-26 1:47 ` Jakub Kicinski
2021-08-26 1:47 ` [PATCH net-next v2 2/3] bnxt: count packets discarded because of netpoll Jakub Kicinski
2021-08-26 1:47 ` [PATCH net-next v2 3/3] bnxt: count discards due to memory allocation errors Jakub Kicinski
2 siblings, 0 replies; 5+ messages in thread
From: Jakub Kicinski @ 2021-08-26 1:47 UTC (permalink / raw
To: davem, michael.chan; +Cc: netdev, olteanv, Jakub Kicinski
Saved ring stats and port stats are completely disjoint.
We can reorder getting the old stats and collecting new
stats from rings with reading port stats.
We can also use bnxt_add_prev_stats() instead of doing
a struct assignment.
With that we can use the same code for closed and open
device, next commits will add more stats at the end
of bnxt_get_stats64().
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
---
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 15 +++++++--------
1 file changed, 7 insertions(+), 8 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index ee66d410c82c..d39449e7b236 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -10673,14 +10673,8 @@ bnxt_get_stats64(struct net_device *dev, struct rtnl_link_stats64 *stats)
* we check the BNXT_STATE_OPEN flag.
*/
smp_mb__after_atomic();
- if (!test_bit(BNXT_STATE_OPEN, &bp->state)) {
- clear_bit(BNXT_STATE_READ_STATS, &bp->state);
- *stats = bp->net_stats_prev;
- return;
- }
-
- bnxt_get_ring_stats(bp, stats);
- bnxt_add_prev_stats(bp, stats);
+ if (!test_bit(BNXT_STATE_OPEN, &bp->state))
+ goto skip_current;
if (bp->flags & BNXT_FLAG_PORT_STATS) {
u64 *rx = bp->port_stats.sw_stats;
@@ -10704,6 +10698,11 @@ bnxt_get_stats64(struct net_device *dev, struct rtnl_link_stats64 *stats)
BNXT_GET_TX_PORT_STATS64(tx, tx_fifo_underruns);
stats->tx_errors = BNXT_GET_TX_PORT_STATS64(tx, tx_err);
}
+
+ bnxt_get_ring_stats(bp, stats);
+skip_current:
+ bnxt_add_prev_stats(bp, stats);
+
clear_bit(BNXT_STATE_READ_STATS, &bp->state);
}
--
2.31.1
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH net-next v2 2/3] bnxt: count packets discarded because of netpoll
2021-08-26 1:47 [PATCH net-next v2 0/3] bnxt: add rx discards stats for oom and netpool Jakub Kicinski
2021-08-26 1:47 ` [PATCH net-next v2 1/3] bnxt: reorder logic in bnxt_get_stats64() Jakub Kicinski
@ 2021-08-26 1:47 ` Jakub Kicinski
2021-08-26 6:58 ` Michael Chan
2021-08-26 1:47 ` [PATCH net-next v2 3/3] bnxt: count discards due to memory allocation errors Jakub Kicinski
2 siblings, 1 reply; 5+ messages in thread
From: Jakub Kicinski @ 2021-08-26 1:47 UTC (permalink / raw
To: davem, michael.chan; +Cc: netdev, olteanv, Jakub Kicinski
bnxt may discard packets if Rx completions are consumed
in an attempt to let netpoll make progress. It should be
exteremely rare in practice but nonetheless such events
should be counted.
Since completion ring memory is allocated dynamically use
a similar scheme to what is done for HW stats to save them.
Report the stats in rx_dropped and per-netdev ethtool
counter. Chances that users care which ring dropped are
very low.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
---
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 31 ++++++++++++++-----
drivers/net/ethernet/broadcom/bnxt/bnxt.h | 2 ++
.../net/ethernet/broadcom/bnxt/bnxt_ethtool.c | 6 ++++
3 files changed, 32 insertions(+), 7 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index d39449e7b236..d12a9052388f 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -2003,6 +2003,7 @@ static int bnxt_force_rx_discard(struct bnxt *bp,
struct rx_cmp *rxcmp;
u16 cp_cons;
u8 cmp_type;
+ int ret;
cp_cons = RING_CMP(tmp_raw_cons);
rxcmp = (struct rx_cmp *)
@@ -2031,7 +2032,10 @@ static int bnxt_force_rx_discard(struct bnxt *bp,
tpa_end1->rx_tpa_end_cmp_errors_v2 |=
cpu_to_le32(RX_TPA_END_CMP_ERRORS);
}
- return bnxt_rx_pkt(bp, cpr, raw_cons, event);
+ ret = bnxt_rx_pkt(bp, cpr, raw_cons, event);
+ if (ret != -EBUSY)
+ cpr->sw_stats.rx.rx_netpoll_discards += 1;
+ return ret;
}
u32 bnxt_fw_health_readl(struct bnxt *bp, int reg_idx)
@@ -10441,7 +10445,8 @@ static bool bnxt_drv_busy(struct bnxt *bp)
}
static void bnxt_get_ring_stats(struct bnxt *bp,
- struct rtnl_link_stats64 *stats);
+ struct rtnl_link_stats64 *stats,
+ struct bnxt_sw_stats *bsw_stats);
static void __bnxt_close_nic(struct bnxt *bp, bool irq_re_init,
bool link_re_init)
@@ -10470,7 +10475,8 @@ static void __bnxt_close_nic(struct bnxt *bp, bool irq_re_init,
/* Save ring stats before shutdown */
if (bp->bnapi && irq_re_init)
- bnxt_get_ring_stats(bp, &bp->net_stats_prev);
+ bnxt_get_ring_stats(bp, &bp->net_stats_prev,
+ &bp->sw_stats_prev);
if (irq_re_init) {
bnxt_free_irq(bp);
bnxt_del_napi(bp);
@@ -10615,7 +10621,8 @@ static int bnxt_ioctl(struct net_device *dev, struct ifreq *ifr, int cmd)
}
static void bnxt_get_ring_stats(struct bnxt *bp,
- struct rtnl_link_stats64 *stats)
+ struct rtnl_link_stats64 *stats,
+ struct bnxt_sw_stats *bsw_stats)
{
int i;
@@ -10646,11 +10653,15 @@ static void bnxt_get_ring_stats(struct bnxt *bp,
stats->multicast += BNXT_GET_RING_STATS64(sw, rx_mcast_pkts);
stats->tx_dropped += BNXT_GET_RING_STATS64(sw, tx_error_pkts);
+
+ bsw_stats->rx.rx_netpoll_discards +=
+ cpr->sw_stats.rx.rx_netpoll_discards;
}
}
static void bnxt_add_prev_stats(struct bnxt *bp,
- struct rtnl_link_stats64 *stats)
+ struct rtnl_link_stats64 *stats,
+ struct bnxt_sw_stats *bsw_stats)
{
struct rtnl_link_stats64 *prev_stats = &bp->net_stats_prev;
@@ -10661,11 +10672,15 @@ static void bnxt_add_prev_stats(struct bnxt *bp,
stats->rx_missed_errors += prev_stats->rx_missed_errors;
stats->multicast += prev_stats->multicast;
stats->tx_dropped += prev_stats->tx_dropped;
+
+ bsw_stats->rx.rx_netpoll_discards +=
+ bp->sw_stats_prev.rx.rx_netpoll_discards;
}
static void
bnxt_get_stats64(struct net_device *dev, struct rtnl_link_stats64 *stats)
{
+ struct bnxt_sw_stats bsw_stats = {};
struct bnxt *bp = netdev_priv(dev);
set_bit(BNXT_STATE_READ_STATS, &bp->state);
@@ -10699,9 +10714,11 @@ bnxt_get_stats64(struct net_device *dev, struct rtnl_link_stats64 *stats)
stats->tx_errors = BNXT_GET_TX_PORT_STATS64(tx, tx_err);
}
- bnxt_get_ring_stats(bp, stats);
+ bnxt_get_ring_stats(bp, stats, &bsw_stats);
skip_current:
- bnxt_add_prev_stats(bp, stats);
+ bnxt_add_prev_stats(bp, stats, &bsw_stats);
+
+ stats->rx_dropped += bsw_stats.rx.rx_netpoll_discards;
clear_bit(BNXT_STATE_READ_STATS, &bp->state);
}
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
index 7b989b6e4f6e..5c2e9a06e959 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
@@ -939,6 +939,7 @@ struct bnxt_rx_sw_stats {
u64 rx_l4_csum_errors;
u64 rx_resets;
u64 rx_buf_errors;
+ u64 rx_netpoll_discards;
};
struct bnxt_cmn_sw_stats {
@@ -1917,6 +1918,7 @@ struct bnxt {
dma_addr_t hwrm_cmd_kong_resp_dma_addr;
struct rtnl_link_stats64 net_stats_prev;
+ struct bnxt_sw_stats sw_stats_prev;
struct bnxt_stats_mem port_stats;
struct bnxt_stats_mem rx_port_stats_ext;
struct bnxt_stats_mem tx_port_stats_ext;
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
index 9f8c72d95228..25f1327aedb6 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
@@ -307,6 +307,7 @@ static const char * const bnxt_cmn_sw_stats_str[] = {
enum {
RX_TOTAL_DISCARDS,
TX_TOTAL_DISCARDS,
+ RX_NETPOLL_DISCARDS,
};
static struct {
@@ -315,6 +316,7 @@ static struct {
} bnxt_sw_func_stats[] = {
{0, "rx_total_discard_pkts"},
{0, "tx_total_discard_pkts"},
+ {0, "rx_netpoll_discards"},
};
#define NUM_RING_RX_SW_STATS ARRAY_SIZE(bnxt_rx_sw_stats_str)
@@ -561,6 +563,8 @@ static void bnxt_get_ethtool_stats(struct net_device *dev,
for (i = 0; i < BNXT_NUM_SW_FUNC_STATS; i++)
bnxt_sw_func_stats[i].counter = 0;
+ bnxt_sw_func_stats[RX_NETPOLL_DISCARDS].counter =
+ bp->sw_stats_prev.rx.rx_netpoll_discards;
tpa_stats = bnxt_get_num_tpa_ring_stats(bp);
for (i = 0; i < bp->cp_nr_rings; i++) {
@@ -603,6 +607,8 @@ static void bnxt_get_ethtool_stats(struct net_device *dev,
BNXT_GET_RING_STATS64(sw_stats, rx_discard_pkts);
bnxt_sw_func_stats[TX_TOTAL_DISCARDS].counter +=
BNXT_GET_RING_STATS64(sw_stats, tx_discard_pkts);
+ bnxt_sw_func_stats[RX_NETPOLL_DISCARDS].counter +=
+ cpr->sw_stats.rx.rx_netpoll_discards;
}
for (i = 0; i < BNXT_NUM_SW_FUNC_STATS; i++, j++)
--
2.31.1
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH net-next v2 3/3] bnxt: count discards due to memory allocation errors
2021-08-26 1:47 [PATCH net-next v2 0/3] bnxt: add rx discards stats for oom and netpool Jakub Kicinski
2021-08-26 1:47 ` [PATCH net-next v2 1/3] bnxt: reorder logic in bnxt_get_stats64() Jakub Kicinski
2021-08-26 1:47 ` [PATCH net-next v2 2/3] bnxt: count packets discarded because of netpoll Jakub Kicinski
@ 2021-08-26 1:47 ` Jakub Kicinski
2 siblings, 0 replies; 5+ messages in thread
From: Jakub Kicinski @ 2021-08-26 1:47 UTC (permalink / raw
To: davem, michael.chan; +Cc: netdev, olteanv, Jakub Kicinski
Count packets dropped due to buffer or skb allocation errors.
Report as part of rx_dropped.
v2: drop the ethtool -S entry [Vladimir]
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
---
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 14 +++++++++++++-
drivers/net/ethernet/broadcom/bnxt/bnxt.h | 1 +
2 files changed, 14 insertions(+), 1 deletion(-)
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index d12a9052388f..bdc5eb42f55b 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -1651,6 +1651,7 @@ static inline struct sk_buff *bnxt_tpa_end(struct bnxt *bp,
skb = bnxt_copy_skb(bnapi, data_ptr, len, mapping);
if (!skb) {
bnxt_abort_tpa(cpr, idx, agg_bufs);
+ cpr->sw_stats.rx.rx_oom_discards += 1;
return NULL;
}
} else {
@@ -1660,6 +1661,7 @@ static inline struct sk_buff *bnxt_tpa_end(struct bnxt *bp,
new_data = __bnxt_alloc_rx_data(bp, &new_mapping, GFP_ATOMIC);
if (!new_data) {
bnxt_abort_tpa(cpr, idx, agg_bufs);
+ cpr->sw_stats.rx.rx_oom_discards += 1;
return NULL;
}
@@ -1675,6 +1677,7 @@ static inline struct sk_buff *bnxt_tpa_end(struct bnxt *bp,
if (!skb) {
kfree(data);
bnxt_abort_tpa(cpr, idx, agg_bufs);
+ cpr->sw_stats.rx.rx_oom_discards += 1;
return NULL;
}
skb_reserve(skb, bp->rx_offset);
@@ -1685,6 +1688,7 @@ static inline struct sk_buff *bnxt_tpa_end(struct bnxt *bp,
skb = bnxt_rx_pages(bp, cpr, skb, idx, agg_bufs, true);
if (!skb) {
/* Page reuse already handled by bnxt_rx_pages(). */
+ cpr->sw_stats.rx.rx_oom_discards += 1;
return NULL;
}
}
@@ -1888,6 +1892,7 @@ static int bnxt_rx_pkt(struct bnxt *bp, struct bnxt_cp_ring_info *cpr,
if (agg_bufs)
bnxt_reuse_rx_agg_bufs(cpr, cp_cons, 0,
agg_bufs, false);
+ cpr->sw_stats.rx.rx_oom_discards += 1;
rc = -ENOMEM;
goto next_rx;
}
@@ -1901,6 +1906,7 @@ static int bnxt_rx_pkt(struct bnxt *bp, struct bnxt_cp_ring_info *cpr,
skb = bp->rx_skb_func(bp, rxr, cons, data, data_ptr, dma_addr,
payload | len);
if (!skb) {
+ cpr->sw_stats.rx.rx_oom_discards += 1;
rc = -ENOMEM;
goto next_rx;
}
@@ -1909,6 +1915,7 @@ static int bnxt_rx_pkt(struct bnxt *bp, struct bnxt_cp_ring_info *cpr,
if (agg_bufs) {
skb = bnxt_rx_pages(bp, cpr, skb, cp_cons, agg_bufs, false);
if (!skb) {
+ cpr->sw_stats.rx.rx_oom_discards += 1;
rc = -ENOMEM;
goto next_rx;
}
@@ -10656,6 +10663,8 @@ static void bnxt_get_ring_stats(struct bnxt *bp,
bsw_stats->rx.rx_netpoll_discards +=
cpr->sw_stats.rx.rx_netpoll_discards;
+ bsw_stats->rx.rx_oom_discards +=
+ cpr->sw_stats.rx.rx_oom_discards;
}
}
@@ -10675,6 +10684,7 @@ static void bnxt_add_prev_stats(struct bnxt *bp,
bsw_stats->rx.rx_netpoll_discards +=
bp->sw_stats_prev.rx.rx_netpoll_discards;
+ bsw_stats->rx.rx_oom_discards += bp->sw_stats_prev.rx.rx_oom_discards;
}
static void
@@ -10718,7 +10728,9 @@ bnxt_get_stats64(struct net_device *dev, struct rtnl_link_stats64 *stats)
skip_current:
bnxt_add_prev_stats(bp, stats, &bsw_stats);
- stats->rx_dropped += bsw_stats.rx.rx_netpoll_discards;
+ stats->rx_dropped +=
+ bsw_stats.rx.rx_netpoll_discards +
+ bsw_stats.rx.rx_oom_discards;
clear_bit(BNXT_STATE_READ_STATS, &bp->state);
}
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
index 5c2e9a06e959..2f37f03b7e2d 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
@@ -939,6 +939,7 @@ struct bnxt_rx_sw_stats {
u64 rx_l4_csum_errors;
u64 rx_resets;
u64 rx_buf_errors;
+ u64 rx_oom_discards;
u64 rx_netpoll_discards;
};
--
2.31.1
^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH net-next v2 2/3] bnxt: count packets discarded because of netpoll
2021-08-26 1:47 ` [PATCH net-next v2 2/3] bnxt: count packets discarded because of netpoll Jakub Kicinski
@ 2021-08-26 6:58 ` Michael Chan
0 siblings, 0 replies; 5+ messages in thread
From: Michael Chan @ 2021-08-26 6:58 UTC (permalink / raw
To: Jakub Kicinski; +Cc: David Miller, Netdev, olteanv
[-- Attachment #1: Type: text/plain, Size: 2044 bytes --]
On Wed, Aug 25, 2021 at 6:47 PM Jakub Kicinski <kuba@kernel.org> wrote:
>
> bnxt may discard packets if Rx completions are consumed
> in an attempt to let netpoll make progress. It should be
> exteremely rare in practice but nonetheless such events
> should be counted.
>
> Since completion ring memory is allocated dynamically use
> a similar scheme to what is done for HW stats to save them.
>
> Report the stats in rx_dropped and per-netdev ethtool
> counter. Chances that users care which ring dropped are
> very low.
>
> Signed-off-by: Jakub Kicinski <kuba@kernel.org>
> ---
> drivers/net/ethernet/broadcom/bnxt/bnxt.c | 31 ++++++++++++++-----
> drivers/net/ethernet/broadcom/bnxt/bnxt.h | 2 ++
> .../net/ethernet/broadcom/bnxt/bnxt_ethtool.c | 6 ++++
> 3 files changed, 32 insertions(+), 7 deletions(-)
>
> diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
> index d39449e7b236..d12a9052388f 100644
> --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
> +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
> @@ -2003,6 +2003,7 @@ static int bnxt_force_rx_discard(struct bnxt *bp,
> struct rx_cmp *rxcmp;
> u16 cp_cons;
> u8 cmp_type;
> + int ret;
>
> cp_cons = RING_CMP(tmp_raw_cons);
> rxcmp = (struct rx_cmp *)
> @@ -2031,7 +2032,10 @@ static int bnxt_force_rx_discard(struct bnxt *bp,
> tpa_end1->rx_tpa_end_cmp_errors_v2 |=
> cpu_to_le32(RX_TPA_END_CMP_ERRORS);
> }
> - return bnxt_rx_pkt(bp, cpr, raw_cons, event);
> + ret = bnxt_rx_pkt(bp, cpr, raw_cons, event);
> + if (ret != -EBUSY)
> + cpr->sw_stats.rx.rx_netpoll_discards += 1;
To be more accurate, we should also skip counting if ret is 0. 0
means it is a TPA_START completion for the start of an aggregated
packet. We haven't dropped this aggregated packet yet until we
process the TPA_END completion.
> + return ret;
> }
>
> u32 bnxt_fw_health_readl(struct bnxt *bp, int reg_idx)
[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4209 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2021-08-26 6:58 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2021-08-26 1:47 [PATCH net-next v2 0/3] bnxt: add rx discards stats for oom and netpool Jakub Kicinski
2021-08-26 1:47 ` [PATCH net-next v2 1/3] bnxt: reorder logic in bnxt_get_stats64() Jakub Kicinski
2021-08-26 1:47 ` [PATCH net-next v2 2/3] bnxt: count packets discarded because of netpoll Jakub Kicinski
2021-08-26 6:58 ` Michael Chan
2021-08-26 1:47 ` [PATCH net-next v2 3/3] bnxt: count discards due to memory allocation errors Jakub Kicinski
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.