lvs-devel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed

From: Sven Bartscher <sven.bartscher@weltraumschlangen.de>
To: lvs-devel@vger.kernel.org
Cc: netfilter-devel@vger.kernel.org
Subject: vs conntrack changes TCP ports mid-stream
Date: Wed, 21 Jun 2023 18:46:43 +0200	[thread overview]
Message-ID: <d3c93e6a-f8a0-5c72-d044-dcec7d8d235c@weltraumschlangen.de> (raw)

[-- Attachment #1: Type: text/plain, Size: 3307 bytes --]

I'm forwarding the report from 
https://bugzilla.netfilter.org/show_bug.cgi?id=1669 here, since it was 
pointed out there, that this list would be more appropriate.

When using an ipvs service in combination with SNAT and a NOTRACK rule, 
specific circumstances can lead to TCP ports of packets being changed 
mid-stream, which results in successful connections that no data can be 
effectively sent over.

Consider the following example:

```
root@router:~# sysctl net.ipv4.vs.conntrack
net.ipv4.vs.conntrack = 1

root@router:~# iptables -t raw -L -n -v
Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
  pkts bytes target     prot opt in     out     source 
destination
    24  1296 CT         tcp  --  enp1s0 *       0.0.0.0/0 
10.0.0.1             tcp dpt:1234 NOTRACK

root@router:~# iptables -t nat -L -n -v
Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
  pkts bytes target     prot opt in     out     source 
destination

Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
  pkts bytes target     prot opt in     out     source 
destination

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
  pkts bytes target     prot opt in     out     source 
destination

Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
  pkts bytes target     prot opt in     out     source 
destination
     4   240 SNAT       all  --  *      *       10.0.0.0/24 
10.0.1.0/24          to:10.0.1.1

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
  pkts bytes target     prot opt in     out     source 
destination

root@router:~# ipvsadm -L -n
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
   -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  10.0.0.1:1234 rr
   -> 10.0.1.2:1234                Masq    1      0          0
   -> 10.0.1.3:1234                Masq    1      0          0
```

The reals servers are running

```
socat TCP4-LISTEN:1234,fork 'EXEC:sh -c 
echo${IFS}hello;read${IFS}r${IFS}L;sleep${IFS}1'
```

We dump the network traffic between the router and client on the ipvs 
router as follows:

```
root@router:~# tcpdump -pXXni enp1s0 icmp or tcp -w 
/tmp/ipvs_port_reuse.pcap
tcpdump: listening on enp1s0, link-type EN10MB (Ethernet), capture size 
262144 bytes
^C16 packets captured
16 packets received by filter
0 packets dropped by kernel
```

While the capture is running, we run the following commands on a client 
to trigger the buggy behavior:

```
root@debian:~# netcat -p 4321 -v 10.0.01 1234
Connection to 10.0.01 1234 port [tcp/*] succeeded!
hello
^C
root@debian:~# sleep 60
root@debian:~# netcat -p 4321 -v 10.0.01 1234
Connection to 10.0.01 1234 port [tcp/*] succeeded!
^C
root@debian:~#
```

We can see that on the first connection attempt we successfully receive 
a reply with payload from the server and then terminate the connection 
with Ctrl+C. Then we wait 60 seconds, which is necessary for the 
previous connection to move out of the TIME_WAIT state. Afterwards we 
open another connection, reusing the same src port as on the first 
connection and don't receive a reply from the server. The captured 
traffic shows, that after the three-way handshake for the second TCP 
connection, packets from the router to the clients use another server 
port than the one used for the initiation of the connection.

Regards
Sven

[-- Attachment #2: ipvs_port_reuse.pcap --]
[-- Type: application/vnd.tcpdump.pcap, Size: 1356 bytes --]

next             reply	other threads:[~2023-06-21 16:46 UTC|newest]

Thread overview: 2+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-06-21 16:46 Sven Bartscher [this message]
2023-06-21 18:26 ` vs conntrack changes TCP ports mid-stream Julian Anastasov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=d3c93e6a-f8a0-5c72-d044-dcec7d8d235c@weltraumschlangen.de \
    --to=sven.bartscher@weltraumschlangen.de \
    --cc=lvs-devel@vger.kernel.org \
    --cc=netfilter-devel@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).