From: Julian Anastasov <ja@ssi.bg>
To: Terin Stock <terin@cloudflare.com>
Cc: horms@verge.net.au, netdev@vger.kernel.org,
lvs-devel@vger.kernel.org, kernel-team@cloudflare.com
Subject: Re: [PATCH] ipvs: align inner_mac_header for encapsulation
Date: Fri, 9 Jun 2023 22:03:39 +0300 (EEST) [thread overview]
Message-ID: <98a12cef-2220-3f92-3b6a-0efc2dd3dfba@ssi.bg> (raw)
In-Reply-To: <20230609110714.2015477-1-terin@cloudflare.com>
Hello,
On Fri, 9 Jun 2023, Terin Stock wrote:
> When using encapsulation the original packet's headers are copied to the
> inner headers. This preserves the space for an inner mac header, which
> is not used by the inner payloads for the encapsulation types supported
> by IPVS. If a packet is using GUE or GRE encapsulation and needs to be
> segmented, flow can be passed to __skb_udp_tunnel_segment() which
> calculates a negative tunnel header length. A negative tunnel header
> length causes pskb_may_pull() to fail, dropping the packet.
>
> This can be observed by attaching probes to ip_vs_in_hook(),
> __dev_queue_xmit(), and __skb_udp_tunnel_segment():
>
> perf probe --add '__dev_queue_xmit skb->inner_mac_header \
> skb->inner_network_header skb->mac_header skb->network_header'
> perf probe --add '__skb_udp_tunnel_segment:7 tnl_hlen'
> perf probe -m ip_vs --add 'ip_vs_in_hook skb->inner_mac_header \
> skb->inner_network_header skb->mac_header skb->network_header'
>
> These probes the headers and tunnel header length for packets which
> traverse the IPVS encapsulation path. A TCP packet can be forced into
> the segmentation path by being smaller than a calculated clamped MSS,
> but larger than the advertised MSS.
>
> probe:ip_vs_in_hook: inner_mac_header=0x0 inner_network_header=0x0 mac_header=0x44 network_header=0x52
> probe:ip_vs_in_hook: inner_mac_header=0x44 inner_network_header=0x52 mac_header=0x44 network_header=0x32
> probe:dev_queue_xmit: inner_mac_header=0x44 inner_network_header=0x52 mac_header=0x44 network_header=0x32
> probe:__skb_udp_tunnel_segment_L7: tnl_hlen=-2
>
> When using veth-based encapsulation, the interfaces are set to be
> mac-less, which does not preserve space for an inner mac header. This
> prevents this issue from occurring.
>
> In our real-world testing of sending a 32KB file we observed operation
> time increasing from ~75ms for veth-based encapsulation to over 1.5s
> using IPVS encapsulation due to retries from dropped packets.
>
> This changeset modifies the packet on the encapsulation path in
> ip_vs_tunnel_xmit() to remove the inner mac header offset. This fixes
> UDP segmentation for both encapsulation types, and corrects the inner
> headers for any IPIP flows that may use it.
>
> Fixes: 84c0d5e96f3a ("ipvs: allow tunneling with gue encapsulation")
> Signed-off-by: Terin Stock <terin@cloudflare.com>
> ---
> net/netfilter/ipvs/ip_vs_xmit.c | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/net/netfilter/ipvs/ip_vs_xmit.c b/net/netfilter/ipvs/ip_vs_xmit.c
> index c7652da78c88..4d20b89dd765 100644
> --- a/net/netfilter/ipvs/ip_vs_xmit.c
> +++ b/net/netfilter/ipvs/ip_vs_xmit.c
> @@ -1207,6 +1207,7 @@ ip_vs_tunnel_xmit(struct sk_buff *skb, struct ip_vs_conn *cp,
> skb->transport_header = skb->network_header;
>
> skb_set_inner_ipproto(skb, next_protocol);
> + skb_set_inner_mac_header(skb, skb_inner_network_offset(skb));
Can you send v2 after including the same line also in
ip_vs_tunnel_xmit_v6?
>
> if (tun_type == IP_VS_CONN_F_TUNNEL_TYPE_GUE) {
> bool check = false;
> --
> 2.40.1
Regards
--
Julian Anastasov <ja@ssi.bg>
prev parent reply other threads:[~2023-06-09 19:03 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-06-09 11:07 [PATCH] ipvs: align inner_mac_header for encapsulation Terin Stock
2023-06-09 19:03 ` Julian Anastasov [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=98a12cef-2220-3f92-3b6a-0efc2dd3dfba@ssi.bg \
--to=ja@ssi.bg \
--cc=horms@verge.net.au \
--cc=kernel-team@cloudflare.com \
--cc=lvs-devel@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=terin@cloudflare.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).