bpf: nodeport: forward L7 svc traffic straight to proxy by julianwiedmann · Pull Request #36383 · cilium/cilium

julianwiedmann · 2024-12-05T08:42:38Z

L7 svc requests are currently redirected to cilium_net / cilium_host, where they get marked with MARK_MAGIC_TO_PROXY and then get rerouted to the proxy.

Simplify this by marking the packet in from-netdev and from-overlay, and letting it pass up the stack. This avoids the dependency on skb->cb surviving the transfer from the native program to cilium_net.

The BPF TPROXY path needs further investigation, so only apply this change when BPF TPROXY is disabled.

julianwiedmann · 2024-12-05T09:03:11Z

/test

julianwiedmann · 2024-12-05T10:22:52Z

I'm thinking whether this will change anything from a netfilter-visibility / "asymmetric path" perspective ...

jschwinger233

So the original "from_netdev/from_overlay -> cilium_net -> cilium_host -> stack -> tproxy hijack" is shortened to "from_netdev/from_overlay -> stack -> tproxy hijack".

Left a comment to understand if it's a missing part or intentional design.

bpf/bpf_host.c

L7 svc requests are currently redirected to cilium_net / cilium_host, where they get marked with MARK_MAGIC_TO_PROXY and then get rerouted to the proxy. Simplify this by marking the packet in from-netdev and from-overlay, and letting it pass up the stack. This avoids the dependency on skb->cb surviving the transfer from the native program to cilium_net. The BPF TPROXY path needs further investigation, so only apply this change when BPF TPROXY is disabled. Signed-off-by: Julian Wiedmann <jwi@isovalent.com>

julianwiedmann · 2024-12-06T07:23:16Z

/test

Fix L7 proxy traffic disruption (DROP_REASON_NOSOCKET) on nodes where the BPF program is attached to a Linux bridge device and the br_netfilter module is loaded with net.bridge.bridge-nf-call-iptables=1. With #36383, we changed the non-BPF-TPROXY L7 LB redirect path to mark packets with MARK_MAGIC_TO_PROXY and punt them to the stack, relying on an iptables TPROXY rule in PREROUTING to deliver the packet to the proxy. This optimization avoids a hairpin redirect through cilium_net. However, when br_netfilter is active on a bridge device, the kernel's ip_sabotage_in() function interferes: 1. Packet arrives on bridge port -> br_netfilter evaluates PREROUTING (1st time, BPF hasn't run yet -> TPROXY rule doesn't match) 2. ip_sabotage_in() sets NF_HOOK_STATE_SABOTAGED on the packet 3. tc-ingress BPF runs, sets MARK_MAGIC_TO_PROXY, punts to stack 4. IP stack calls PREROUTING again, but ip_sabotage causes it to be SKIPPED entirely 5. TPROXY rule never fires -> no listening socket on VIP -> DROP_REASON_NOSOCKET In previous commits, we added a datapath config variable so that each program is aware of its attached link type. With this, if the device is a bridge, the L7 LB path falls back to the pre-#36383 hairpin redirect via cilium_net, bypassing the punt-to-stack path that ip_sabotage would break. Non-bridge devices continue using the optimized punt-to-stack path. Signed-off-by: Simone Magnani <simone.magnani@isovalent.com>

[ upstream commit 0ba47cd ] [ backporter's notes: * Added bits into bpf/tests/pktgen.h without backporting whole unrelated PRs * adjusted LB4_SERVICES_MAP_V2, LB4_REVERSE_NAT_MAP, and their respective IPv6 versions references, different than in upstream * added `ENABLE_SERVICE_PROTOCOL_DIFFERENTIATION` in the new test, alongside with some different infra to tail call the ingress program, and macros to check redirect interface ] Fix L7 proxy traffic disruption (DROP_REASON_NOSOCKET) on nodes where the BPF program is attached to a Linux bridge device and the br_netfilter module is loaded with net.bridge.bridge-nf-call-iptables=1. With #36383, we changed the non-BPF-TPROXY L7 LB redirect path to mark packets with MARK_MAGIC_TO_PROXY and punt them to the stack, relying on an iptables TPROXY rule in PREROUTING to deliver the packet to the proxy. This optimization avoids a hairpin redirect through cilium_net. However, when br_netfilter is active on a bridge device, the kernel's ip_sabotage_in() function interferes: 1. Packet arrives on bridge port -> br_netfilter evaluates PREROUTING (1st time, BPF hasn't run yet -> TPROXY rule doesn't match) 2. ip_sabotage_in() sets NF_HOOK_STATE_SABOTAGED on the packet 3. tc-ingress BPF runs, sets MARK_MAGIC_TO_PROXY, punts to stack 4. IP stack calls PREROUTING again, but ip_sabotage causes it to be SKIPPED entirely 5. TPROXY rule never fires -> no listening socket on VIP -> DROP_REASON_NOSOCKET In previous commits, we added a datapath config variable so that each program is aware of its attached link type. With this, if the device is a bridge, the L7 LB path falls back to the pre-#36383 hairpin redirect via cilium_net, bypassing the punt-to-stack path that ip_sabotage would break. Non-bridge devices continue using the optimized punt-to-stack path. Signed-off-by: Simone Magnani <simone.magnani@isovalent.com>

Fix L7 proxy traffic disruption (DROP_REASON_NOSOCKET) on nodes where the BPF program is attached to a Linux bridge device and the br_netfilter module is loaded with net.bridge.bridge-nf-call-iptables=1. With #36383, we changed the non-BPF-TPROXY L7 LB redirect path to mark packets with MARK_MAGIC_TO_PROXY and punt them to the stack, relying on an iptables TPROXY rule in PREROUTING to deliver the packet to the proxy. This optimization avoids a hairpin redirect through cilium_net. However, when br_netfilter is active on a bridge device, the kernel's ip_sabotage_in() function interferes: 1. Packet arrives on bridge port -> br_netfilter evaluates PREROUTING (1st time, BPF hasn't run yet -> TPROXY rule doesn't match) 2. ip_sabotage_in() sets NF_HOOK_STATE_SABOTAGED on the packet 3. tc-ingress BPF runs, sets MARK_MAGIC_TO_PROXY, punts to stack 4. IP stack calls PREROUTING again, but ip_sabotage causes it to be SKIPPED entirely 5. TPROXY rule never fires -> no listening socket on VIP -> DROP_REASON_NOSOCKET We therefore add a new datapath config variable that will be set to true only when compiling the Netdev datapath program (cil_{from,to}_netdev) and the network device is a bridge. This allows us to fall back to the pre-#36383 hairpin redirect via cilium_net for L7 LB traffic, bypassing the ip_sabotage_in() function and allowing the TPROXY rule to match as expected. Non-bridge devices continue using the optimized punt-to-stack path. Signed-off-by: Simone Magnani <simone.magnani@isovalent.com>

[ upstream commit 473a9e6 ] [ backporter's notes: fixed conflicts: * adapting to the old loader implementation, adding changes only to the Netdev program attached to network devices. * added bits into bpf/tests/pktgen.h without backporting whole unrelated PRs * adjusted LB4_SERVICES_MAP_V2, LB4_REVERSE_NAT_MAP, and their respective IPv6 versions references, different than in upstream * added `ENABLE_SERVICE_PROTOCOL_DIFFERENTIATION` in the new test, alongside with some different infra to tail call the ingress program, and macros to check redirect interface * changed check in bpf/tests/l7_lb_hairpin.c to check for HOST_IFINDEX instead of cilium_net, as the latter is not used in this backport ] Fix L7 proxy traffic disruption (DROP_REASON_NOSOCKET) on nodes where the BPF program is attached to a Linux bridge device and the br_netfilter module is loaded with net.bridge.bridge-nf-call-iptables=1. With #36383, we changed the non-BPF-TPROXY L7 LB redirect path to mark packets with MARK_MAGIC_TO_PROXY and punt them to the stack, relying on an iptables TPROXY rule in PREROUTING to deliver the packet to the proxy. This optimization avoids a hairpin redirect through cilium_net. However, when br_netfilter is active on a bridge device, the kernel's ip_sabotage_in() function interferes: 1. Packet arrives on bridge port -> br_netfilter evaluates PREROUTING (1st time, BPF hasn't run yet -> TPROXY rule doesn't match) 2. ip_sabotage_in() sets NF_HOOK_STATE_SABOTAGED on the packet 3. tc-ingress BPF runs, sets MARK_MAGIC_TO_PROXY, punts to stack 4. IP stack calls PREROUTING again, but ip_sabotage causes it to be SKIPPED entirely 5. TPROXY rule never fires -> no listening socket on VIP -> DROP_REASON_NOSOCKET We therefore add a new datapath config variable that will be set to true only when compiling the Netdev datapath program (cil_{from,to}_netdev) and the network device is a bridge. This allows us to fall back to the pre-#36383 hairpin redirect via cilium_net for L7 LB traffic, bypassing the ip_sabotage_in() function and allowing the TPROXY rule to match as expected. Non-bridge devices continue using the optimized punt-to-stack path. Signed-off-by: Simone Magnani <simone.magnani@isovalent.com>

Fix L7 proxy traffic disruption (DROP_REASON_NOSOCKET) on nodes where the BPF program is attached to a Linux bridge device and the br_netfilter module is loaded with net.bridge.bridge-nf-call-iptables=1. With #36383, we changed the non-BPF-TPROXY L7 LB redirect path to mark packets with MARK_MAGIC_TO_PROXY and punt them to the stack, relying on an iptables TPROXY rule in PREROUTING to deliver the packet to the proxy. This optimization avoids a hairpin redirect through cilium_net. However, when br_netfilter is active on a bridge device, the kernel's ip_sabotage_in() function interferes: 1. Packet arrives on bridge port -> br_netfilter evaluates PREROUTING (1st time, BPF hasn't run yet -> TPROXY rule doesn't match) 2. ip_sabotage_in() sets NF_HOOK_STATE_SABOTAGED on the packet 3. tc-ingress BPF runs, sets MARK_MAGIC_TO_PROXY, punts to stack 4. IP stack calls PREROUTING again, but ip_sabotage causes it to be SKIPPED entirely 5. TPROXY rule never fires -> no listening socket on VIP -> DROP_REASON_NOSOCKET We therefore add a new datapath config variable that will be set to true only when compiling the Netdev datapath program (cil_{from,to}_netdev) and the network device is a bridge. This allows us to fall back to the pre-#36383 hairpin redirect via cilium_net for L7 LB traffic, bypassing the ip_sabotage_in() function and allowing the TPROXY rule to match as expected. Non-bridge devices continue using the optimized punt-to-stack path. Signed-off-by: Simone Magnani <simone.magnani@isovalent.com>

[ upstream commit 52f35a9 ] [ backporter's notes: fixed conflicts: * adapting to the old loader implementation, adding changes only to the Netdev program attached to network devices. * added bits into bpf/tests/pktgen.h without backporting whole unrelated PRs * adjusted LB4_SERVICES_MAP_V2, LB4_REVERSE_NAT_MAP, and their respective IPv6 versions references, different than in upstream * added `ENABLE_SERVICE_PROTOCOL_DIFFERENTIATION` in the new test, alongside with some different infra to tail call the ingress program, and macros to check redirect interface * changed check in bpf/tests/l7_lb_hairpin.c to check for HOST_IFINDEX instead of cilium_net, as the latter is not used in this backport ] Fix L7 proxy traffic disruption (DROP_REASON_NOSOCKET) on nodes where the BPF program is attached to a Linux bridge device and the br_netfilter module is loaded with net.bridge.bridge-nf-call-iptables=1. With #36383, we changed the non-BPF-TPROXY L7 LB redirect path to mark packets with MARK_MAGIC_TO_PROXY and punt them to the stack, relying on an iptables TPROXY rule in PREROUTING to deliver the packet to the proxy. This optimization avoids a hairpin redirect through cilium_net. However, when br_netfilter is active on a bridge device, the kernel's ip_sabotage_in() function interferes: 1. Packet arrives on bridge port -> br_netfilter evaluates PREROUTING (1st time, BPF hasn't run yet -> TPROXY rule doesn't match) 2. ip_sabotage_in() sets NF_HOOK_STATE_SABOTAGED on the packet 3. tc-ingress BPF runs, sets MARK_MAGIC_TO_PROXY, punts to stack 4. IP stack calls PREROUTING again, but ip_sabotage causes it to be SKIPPED entirely 5. TPROXY rule never fires -> no listening socket on VIP -> DROP_REASON_NOSOCKET We therefore add a new datapath config variable that will be set to true only when compiling the Netdev datapath program (cil_{from,to}_netdev) and the network device is a bridge. This allows us to fall back to the pre-#36383 hairpin redirect via cilium_net for L7 LB traffic, bypassing the ip_sabotage_in() function and allowing the TPROXY rule to match as expected. Non-bridge devices continue using the optimized punt-to-stack path. Signed-off-by: Simone Magnani <simone.magnani@isovalent.com>

[ upstream commit 52f35a9 ] [ backporter's notes: fixed conflicts: * adapting to the old loader implementation, adding changes only to the Netdev program attached to network devices. * added bits into bpf/tests/pktgen.h without backporting whole unrelated PRs * changed check in bpf/tests/l7_lb_hairpin.c to check for CILIUM_NET_IFINDEX instead of CONFIG(cilium_net_ifindex), as the latter is not used in this backport ] Fix L7 proxy traffic disruption (DROP_REASON_NOSOCKET) on nodes where the BPF program is attached to a Linux bridge device and the br_netfilter module is loaded with net.bridge.bridge-nf-call-iptables=1. With #36383, we changed the non-BPF-TPROXY L7 LB redirect path to mark packets with MARK_MAGIC_TO_PROXY and punt them to the stack, relying on an iptables TPROXY rule in PREROUTING to deliver the packet to the proxy. This optimization avoids a hairpin redirect through cilium_net. However, when br_netfilter is active on a bridge device, the kernel's ip_sabotage_in() function interferes: 1. Packet arrives on bridge port -> br_netfilter evaluates PREROUTING (1st time, BPF hasn't run yet -> TPROXY rule doesn't match) 2. ip_sabotage_in() sets NF_HOOK_STATE_SABOTAGED on the packet 3. tc-ingress BPF runs, sets MARK_MAGIC_TO_PROXY, punts to stack 4. IP stack calls PREROUTING again, but ip_sabotage causes it to be SKIPPED entirely 5. TPROXY rule never fires -> no listening socket on VIP -> DROP_REASON_NOSOCKET We therefore add a new datapath config variable that will be set to true only when compiling the Netdev datapath program (cil_{from,to}_netdev) and the network device is a bridge. This allows us to fall back to the pre-#36383 hairpin redirect via cilium_net for L7 LB traffic, bypassing the ip_sabotage_in() function and allowing the TPROXY rule to match as expected. Non-bridge devices continue using the optimized punt-to-stack path. Signed-off-by: Simone Magnani <simone.magnani@isovalent.com>

[ upstream commit 52f35a9 ] [ backporter's notes: fixed conflicts: * adapting to the old loader implementation, adding changes only to the Netdev program attached to network devices. * changed check in bpf/tests/l7_lb_hairpin.c to check for CILIUM_NET_IFINDEX instead of CONFIG(cilium_net_ifindex), as the latter is not used in this backport ] Fix L7 proxy traffic disruption (DROP_REASON_NOSOCKET) on nodes where the BPF program is attached to a Linux bridge device and the br_netfilter module is loaded with net.bridge.bridge-nf-call-iptables=1. With #36383, we changed the non-BPF-TPROXY L7 LB redirect path to mark packets with MARK_MAGIC_TO_PROXY and punt them to the stack, relying on an iptables TPROXY rule in PREROUTING to deliver the packet to the proxy. This optimization avoids a hairpin redirect through cilium_net. However, when br_netfilter is active on a bridge device, the kernel's ip_sabotage_in() function interferes: 1. Packet arrives on bridge port -> br_netfilter evaluates PREROUTING (1st time, BPF hasn't run yet -> TPROXY rule doesn't match) 2. ip_sabotage_in() sets NF_HOOK_STATE_SABOTAGED on the packet 3. tc-ingress BPF runs, sets MARK_MAGIC_TO_PROXY, punts to stack 4. IP stack calls PREROUTING again, but ip_sabotage causes it to be SKIPPED entirely 5. TPROXY rule never fires -> no listening socket on VIP -> DROP_REASON_NOSOCKET We therefore add a new datapath config variable that will be set to true only when compiling the Netdev datapath program (cil_{from,to}_netdev) and the network device is a bridge. This allows us to fall back to the pre-#36383 hairpin redirect via cilium_net for L7 LB traffic, bypassing the ip_sabotage_in() function and allowing the TPROXY rule to match as expected. Non-bridge devices continue using the optimized punt-to-stack path. Signed-off-by: Simone Magnani <simone.magnani@isovalent.com>

[ upstream commit 52f35a9 ] [ backporter's notes: fixed conflicts: * adapting to the old loader implementation, adding changes only to the Netdev program attached to network devices. * added bits into bpf/tests/pktgen.h without backporting whole unrelated PRs * changed check in bpf/tests/l7_lb_hairpin.c to check for CILIUM_NET_IFINDEX instead of CONFIG(cilium_net_ifindex), as the latter is not used in this backport ] Fix L7 proxy traffic disruption (DROP_REASON_NOSOCKET) on nodes where the BPF program is attached to a Linux bridge device and the br_netfilter module is loaded with net.bridge.bridge-nf-call-iptables=1. With #36383, we changed the non-BPF-TPROXY L7 LB redirect path to mark packets with MARK_MAGIC_TO_PROXY and punt them to the stack, relying on an iptables TPROXY rule in PREROUTING to deliver the packet to the proxy. This optimization avoids a hairpin redirect through cilium_net. However, when br_netfilter is active on a bridge device, the kernel's ip_sabotage_in() function interferes: 1. Packet arrives on bridge port -> br_netfilter evaluates PREROUTING (1st time, BPF hasn't run yet -> TPROXY rule doesn't match) 2. ip_sabotage_in() sets NF_HOOK_STATE_SABOTAGED on the packet 3. tc-ingress BPF runs, sets MARK_MAGIC_TO_PROXY, punts to stack 4. IP stack calls PREROUTING again, but ip_sabotage causes it to be SKIPPED entirely 5. TPROXY rule never fires -> no listening socket on VIP -> DROP_REASON_NOSOCKET We therefore add a new datapath config variable that will be set to true only when compiling the Netdev datapath program (cil_{from,to}_netdev) and the network device is a bridge. This allows us to fall back to the pre-#36383 hairpin redirect via cilium_net for L7 LB traffic, bypassing the ip_sabotage_in() function and allowing the TPROXY rule to match as expected. Non-bridge devices continue using the optimized punt-to-stack path. Signed-off-by: Simone Magnani <simone.magnani@isovalent.com>

[ upstream commit 52f35a9 ] [ backporter's notes: fixed conflicts: * adapting to the old loader implementation, adding changes only to the Netdev program attached to network devices. * added bits into bpf/tests/pktgen.h without backporting whole unrelated PRs * adjusted LB4_SERVICES_MAP_V2, LB4_REVERSE_NAT_MAP, and their respective IPv6 versions references, different than in upstream * added `ENABLE_SERVICE_PROTOCOL_DIFFERENTIATION` in the new test, alongside with some different infra to tail call the ingress program, and macros to check redirect interface * changed check in bpf/tests/l7_lb_hairpin.c to check for HOST_IFINDEX instead of cilium_net, as the latter is not used in this backport ] Fix L7 proxy traffic disruption (DROP_REASON_NOSOCKET) on nodes where the BPF program is attached to a Linux bridge device and the br_netfilter module is loaded with net.bridge.bridge-nf-call-iptables=1. With #36383, we changed the non-BPF-TPROXY L7 LB redirect path to mark packets with MARK_MAGIC_TO_PROXY and punt them to the stack, relying on an iptables TPROXY rule in PREROUTING to deliver the packet to the proxy. This optimization avoids a hairpin redirect through cilium_net. However, when br_netfilter is active on a bridge device, the kernel's ip_sabotage_in() function interferes: 1. Packet arrives on bridge port -> br_netfilter evaluates PREROUTING (1st time, BPF hasn't run yet -> TPROXY rule doesn't match) 2. ip_sabotage_in() sets NF_HOOK_STATE_SABOTAGED on the packet 3. tc-ingress BPF runs, sets MARK_MAGIC_TO_PROXY, punts to stack 4. IP stack calls PREROUTING again, but ip_sabotage causes it to be SKIPPED entirely 5. TPROXY rule never fires -> no listening socket on VIP -> DROP_REASON_NOSOCKET We therefore add a new datapath config variable that will be set to true only when compiling the Netdev datapath program (cil_{from,to}_netdev) and the network device is a bridge. This allows us to fall back to the pre-#36383 hairpin redirect via cilium_net for L7 LB traffic, bypassing the ip_sabotage_in() function and allowing the TPROXY rule to match as expected. Non-bridge devices continue using the optimized punt-to-stack path. Signed-off-by: Simone Magnani <simone.magnani@isovalent.com>

[ upstream commit 52f35a9 ] [ backporter's notes: fixed conflicts: * adapting to the old loader implementation, adding changes only to the Netdev program attached to network devices. * added bits into bpf/tests/pktgen.h without backporting whole unrelated PRs * changed check in bpf/tests/l7_lb_hairpin.c to check for CILIUM_NET_IFINDEX instead of CONFIG(cilium_net_ifindex), as the latter is not used in this backport ] Fix L7 proxy traffic disruption (DROP_REASON_NOSOCKET) on nodes where the BPF program is attached to a Linux bridge device and the br_netfilter module is loaded with net.bridge.bridge-nf-call-iptables=1. With #36383, we changed the non-BPF-TPROXY L7 LB redirect path to mark packets with MARK_MAGIC_TO_PROXY and punt them to the stack, relying on an iptables TPROXY rule in PREROUTING to deliver the packet to the proxy. This optimization avoids a hairpin redirect through cilium_net. However, when br_netfilter is active on a bridge device, the kernel's ip_sabotage_in() function interferes: 1. Packet arrives on bridge port -> br_netfilter evaluates PREROUTING (1st time, BPF hasn't run yet -> TPROXY rule doesn't match) 2. ip_sabotage_in() sets NF_HOOK_STATE_SABOTAGED on the packet 3. tc-ingress BPF runs, sets MARK_MAGIC_TO_PROXY, punts to stack 4. IP stack calls PREROUTING again, but ip_sabotage causes it to be SKIPPED entirely 5. TPROXY rule never fires -> no listening socket on VIP -> DROP_REASON_NOSOCKET We therefore add a new datapath config variable that will be set to true only when compiling the Netdev datapath program (cil_{from,to}_netdev) and the network device is a bridge. This allows us to fall back to the pre-#36383 hairpin redirect via cilium_net for L7 LB traffic, bypassing the ip_sabotage_in() function and allowing the TPROXY rule to match as expected. Non-bridge devices continue using the optimized punt-to-stack path. Signed-off-by: Simone Magnani <simone.magnani@isovalent.com>

[ upstream commit 52f35a9 ] [ backporter's notes: fixed conflicts: * adapting to the old loader implementation, adding changes only to the Netdev program attached to network devices. * added bits into bpf/tests/pktgen.h without backporting whole unrelated PRs * adjusted LB4_SERVICES_MAP_V2, LB4_REVERSE_NAT_MAP, and their respective IPv6 versions references, different than in upstream * added `ENABLE_SERVICE_PROTOCOL_DIFFERENTIATION` in the new test, alongside with some different infra to tail call the ingress program, and macros to check redirect interface * changed check in bpf/tests/l7_lb_hairpin.c to check for HOST_IFINDEX instead of cilium_net, as the latter is not used in this backport ] Fix L7 proxy traffic disruption (DROP_REASON_NOSOCKET) on nodes where the BPF program is attached to a Linux bridge device and the br_netfilter module is loaded with net.bridge.bridge-nf-call-iptables=1. With #36383, we changed the non-BPF-TPROXY L7 LB redirect path to mark packets with MARK_MAGIC_TO_PROXY and punt them to the stack, relying on an iptables TPROXY rule in PREROUTING to deliver the packet to the proxy. This optimization avoids a hairpin redirect through cilium_net. However, when br_netfilter is active on a bridge device, the kernel's ip_sabotage_in() function interferes: 1. Packet arrives on bridge port -> br_netfilter evaluates PREROUTING (1st time, BPF hasn't run yet -> TPROXY rule doesn't match) 2. ip_sabotage_in() sets NF_HOOK_STATE_SABOTAGED on the packet 3. tc-ingress BPF runs, sets MARK_MAGIC_TO_PROXY, punts to stack 4. IP stack calls PREROUTING again, but ip_sabotage causes it to be SKIPPED entirely 5. TPROXY rule never fires -> no listening socket on VIP -> DROP_REASON_NOSOCKET We therefore add a new datapath config variable that will be set to true only when compiling the Netdev datapath program (cil_{from,to}_netdev) and the network device is a bridge. This allows us to fall back to the pre-#36383 hairpin redirect via cilium_net for L7 LB traffic, bypassing the ip_sabotage_in() function and allowing the TPROXY rule to match as expected. Non-bridge devices continue using the optimized punt-to-stack path. Signed-off-by: Simone Magnani <simone.magnani@isovalent.com>

Fix L7 proxy traffic disruption (DROP_REASON_NOSOCKET) on nodes where the BPF program is attached to a Linux bridge device and the br_netfilter module is loaded with net.bridge.bridge-nf-call-iptables=1. With cilium#36383, we changed the non-BPF-TPROXY L7 LB redirect path to mark packets with MARK_MAGIC_TO_PROXY and punt them to the stack, relying on an iptables TPROXY rule in PREROUTING to deliver the packet to the proxy. This optimization avoids a hairpin redirect through cilium_net. However, when br_netfilter is active on a bridge device, the kernel's ip_sabotage_in() function interferes: 1. Packet arrives on bridge port -> br_netfilter evaluates PREROUTING (1st time, BPF hasn't run yet -> TPROXY rule doesn't match) 2. ip_sabotage_in() sets NF_HOOK_STATE_SABOTAGED on the packet 3. tc-ingress BPF runs, sets MARK_MAGIC_TO_PROXY, punts to stack 4. IP stack calls PREROUTING again, but ip_sabotage causes it to be SKIPPED entirely 5. TPROXY rule never fires -> no listening socket on VIP -> DROP_REASON_NOSOCKET We therefore add a new datapath config variable that will be set to true only when compiling the Netdev datapath program (cil_{from,to}_netdev) and the network device is a bridge. This allows us to fall back to the pre-cilium#36383 hairpin redirect via cilium_net for L7 LB traffic, bypassing the ip_sabotage_in() function and allowing the TPROXY rule to match as expected. Non-bridge devices continue using the optimized punt-to-stack path. Signed-off-by: Simone Magnani <simone.magnani@isovalent.com>

maintainer-s-little-helper bot added the dont-merge/needs-release-note-label The author needs to describe the release impact of these changes. label Dec 5, 2024

maintainer-s-little-helper bot removed the dont-merge/needs-release-note-label The author needs to describe the release impact of these changes. label Dec 5, 2024

julianwiedmann added the dont-merge/preview-only Only for preview or testing, don't merge it. label Dec 5, 2024

julianwiedmann requested review from brb and jrajahalme December 5, 2024 10:17

julianwiedmann marked this pull request as ready for review December 5, 2024 10:17

julianwiedmann requested a review from a team as a code owner December 5, 2024 10:17

julianwiedmann requested a review from jschwinger233 December 5, 2024 11:30

jschwinger233 approved these changes Dec 6, 2024

View reviewed changes

bpf/bpf_host.c Show resolved Hide resolved

julianwiedmann force-pushed the 1.17-bpf-redirect-2 branch from 9cbb9c5 to 0b80b4f Compare December 6, 2024 07:21

julianwiedmann removed the dont-merge/preview-only Only for preview or testing, don't merge it. label Dec 6, 2024

jrajahalme approved these changes Dec 6, 2024

View reviewed changes

brb approved these changes Dec 6, 2024

View reviewed changes

maintainer-s-little-helper bot added the ready-to-merge This PR has passed all tests and received consensus from code owners to merge. label Dec 6, 2024

julianwiedmann added this pull request to the merge queue Dec 6, 2024

Merged via the queue into cilium:main with commit 1c7f706 Dec 6, 2024

julianwiedmann deleted the 1.17-bpf-redirect-2 branch December 6, 2024 10:14

cilium-release-bot bot mentioned this pull request Dec 18, 2024

Prepare for release v1.17.0-rc.0 #36692

Merged

smagnani96 mentioned this pull request Mar 11, 2026

bpf: nodeport: use hairpin redirect for L7 LB on bridge devices #44658

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

bpf: nodeport: forward L7 svc traffic straight to proxy#36383

bpf: nodeport: forward L7 svc traffic straight to proxy#36383
julianwiedmann merged 1 commit intocilium:mainfrom
julianwiedmann:1.17-bpf-redirect-2

julianwiedmann commented Dec 5, 2024

Uh oh!

julianwiedmann commented Dec 5, 2024

Uh oh!

julianwiedmann commented Dec 5, 2024

Uh oh!

jschwinger233 left a comment •

edited

Loading

Uh oh!

Uh oh!

julianwiedmann commented Dec 6, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

julianwiedmann commented Dec 5, 2024

Uh oh!

julianwiedmann commented Dec 5, 2024

Uh oh!

julianwiedmann commented Dec 5, 2024

Uh oh!

jschwinger233 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

julianwiedmann commented Dec 6, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

jschwinger233 left a comment •

edited

Loading