From: Florian Westphal <fw@strlen.de>
To: <netfilter-devel@vger.kernel.org>
Cc: Florian Westphal <fw@strlen.de>
Subject: [PATCH nf-next 00/11] netfilter: nf_tables: reduce transaction log memory usage
Date: Mon, 13 May 2024 15:00:40 +0200 [thread overview]
Message-ID: <20240513130057.11014-1-fw@strlen.de> (raw)
The transaction log can grow to huge values.
Insertion of 1.000.000 elements into a set, or flushing a set with
1.000.000 elements will eat 128 byte per element, i.e. 128 MiBi.
This series compacts the structures. After this series, struct
nft_trans_elem can be allocated from kmalloc-96 slab, resulting
in a 25% memory reduction.
To further reduce flush/mass-insert several approaches come
to mind:
1. allow struct nft_trans_elem to hold several elements.
2. add a kernel-internal, dedicated nft_trans_elem_batch that
is only used for flushing (similar to 1).
3. Remove 'struct net' from nft_trans struct. This reduces
size of nft_trans_elem to 64 bytes, which would halve memory
needs compared to the current state.
I have tried to do 3), its possible but not very elegant.
You can have a look at the general idea at
https://git.kernel.org/pub/scm/linux/kernel/git/fwestphal/nf-next.git/commit/?h=nft_trans_compact_01&id=5269e591563204490b9fad6ae1e33810a9f4c39d
I have started to look at 1) too, but unlike this compaction
series it looks like this will make things even more complex
as we'll need to be careful wrt. appending more set elements to
an already-queued nft_trans_elem (must be same msg_type, same set,
etc).
This series has seen brief testing with kasan+kmemleak and
nftables.git selftests.
Feedback and comments welcome.
Florian Westphal (11):
netfilter: nf_tables: make struct nft_trans first member of derived subtypes
netfilter: nf_tables: move bind list_head into relevant subtypes
netfilter: nf_tables: compact chain+ft transaction objects
netfilter: nf_tables: reduce trans->ctx.table references
netfilter: nf_tables: pass nft_chain to destroy function, not nft_ctx
netfilter: nf_tables: pass more specific nft_trans_chain where possible
netfilter: nf_tables: avoid usage of embedded nft_ctx
netfilter: nf_tables: store chain pointer in rule transaction
netfilter: nf_tables: reduce trans->ctx.chain references
netfilter: nf_tables: pass nft_table to destroy function
netfilter: nf_tables: do not store nft_ctx in transaction objects
include/net/netfilter/nf_tables.h | 152 +++++++----
net/netfilter/nf_tables_api.c | 402 +++++++++++++++++-------------
net/netfilter/nf_tables_offload.c | 40 +--
net/netfilter/nft_immediate.c | 2 +-
4 files changed, 363 insertions(+), 233 deletions(-)
--
2.43.2
next reply other threads:[~2024-05-13 13:09 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-05-13 13:00 Florian Westphal [this message]
2024-05-13 13:00 ` [PATCH nf-next 01/11] netfilter: nf_tables: make struct nft_trans first member of derived subtypes Florian Westphal
2024-05-13 13:00 ` [PATCH nf-next 02/11] netfilter: nf_tables: move bind list_head into relevant subtypes Florian Westphal
2024-05-13 13:00 ` [PATCH nf-next 03/11] netfilter: nf_tables: compact chain+ft transaction objects Florian Westphal
2024-05-13 13:00 ` [PATCH nf-next 04/11] netfilter: nf_tables: reduce trans->ctx.table references Florian Westphal
2024-05-13 13:00 ` [PATCH nf-next 05/11] netfilter: nf_tables: pass nft_chain to destroy function, not nft_ctx Florian Westphal
2024-05-13 13:00 ` [PATCH nf-next 06/11] netfilter: nf_tables: pass more specific nft_trans_chain where possible Florian Westphal
2024-05-13 13:00 ` [PATCH nf-next 07/11] netfilter: nf_tables: avoid usage of embedded nft_ctx Florian Westphal
2024-05-13 13:00 ` [PATCH nf-next 08/11] netfilter: nf_tables: store chain pointer in rule transaction Florian Westphal
2024-05-13 13:00 ` [PATCH nf-next 09/11] netfilter: nf_tables: reduce trans->ctx.chain references Florian Westphal
2024-05-13 13:00 ` [PATCH nf-next 10/11] netfilter: nf_tables: pass nft_table to destroy function Florian Westphal
2024-05-13 13:00 ` [PATCH nf-next 11/11] netfilter: nf_tables: do not store nft_ctx in transaction objects Florian Westphal
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240513130057.11014-1-fw@strlen.de \
--to=fw@strlen.de \
--cc=netfilter-devel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).