From: Simon Kirby <sim@hostway.ca>
To: Florian Westphal <fw@strlen.de>
Cc: netdev@vger.kernel.org, netfilter-devel@vger.kernel.org,
lvs-devel@vger.kernel.org
Subject: Inability to IPVS DR with nft dnat since 9971a514ed26
Date: Tue, 26 Mar 2019 23:26:50 -0700 [thread overview]
Message-ID: <20190327062650.GA10700@hostway.ca> (raw)
Hello!
We have been successfully using nft dnat and IPVS in DR mode on 4.9, 4.14
kernels, but since upgrading to 4.19, such rules now appear to miss the
IPVS input hook and instead appear to hit localhost (and "tcpdump -ni lo"
shows the packets) instead of being forwarded to a real server.
I bisected this to 9971a514ed2697e542f3984a6162eac54bb1da98 ("netfilter:
nf_nat: add nat type hooks to nat core").
It should be pretty easy to see this with a minimal setup:
/etc/nftables.conf:
table ip nat {
chain prerouting {
type nat hook prerouting priority 0;
ip daddr $ext_ip dnat to $vip
}
chain postrouting {
type nat hook postrouting priority 100;
# In theory this hook no longer needed since this commit,
# but we also need to do some unrelated snatting.
}
}
/etc/sysctl.conf:
net.ipv4.conf.all.accept_local = 1
net.ipv4.vs.conntrack = 1
IPVS DR setup:
ipvsadm -A -t $vip:80 -s wrr
ipvsadm -a -t $vip:80 -r $real_ip:80 -g -w 100
On the real server, the vip has to be bound to lo or similar and
net.ipv4.conf.all.arp_announce=2 and net.ipv4.conf.all.arp_ignore=1 as
usual for DR, with a symmetric gateway setup (with accept_local above).
Actually, a real server isn't needed to show the issue here, just another
neighbor to route at.
When it works, the inbound frame (TCP connectin to $ext_ip:80) should be
dnatted and then L2-routed (like a static route) to the MAC of $real_ip,
and sent out that interface. Since this commit, it hits lo instead.
Any ideas on what is going wrong here?
Note that we ended up using originally using nftables here because it let
us do one more thing: hairpin NAT _with_ IPVS all on the same host with
"type nat hook input priority -99" and applying snat there. The abillity
to specify hook priorities made this possible. I haven't checked if this
is still working or not, yet, though.
Simon-
next reply other threads:[~2019-03-27 6:26 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-03-27 6:26 Simon Kirby [this message]
2019-03-27 9:30 ` Inability to IPVS DR with nft dnat since 9971a514ed26 Florian Westphal
2019-03-27 15:34 ` Simon Kirby
2021-12-03 8:34 ` Simon Kirby
2021-12-03 9:40 ` Pablo Neira Ayuso
2021-12-03 21:48 ` Julian Anastasov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190327062650.GA10700@hostway.ca \
--to=sim@hostway.ca \
--cc=fw@strlen.de \
--cc=lvs-devel@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=netfilter-devel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).