Skip to content

bpf: Introduce wildcard service entries to drop traffic towards unknown protocol/port combinations#40684

Merged
borkmann merged 8 commits intocilium:mainfrom
ajmmm:pr/slb-drop-wildcard-vip
Aug 27, 2025
Merged

bpf: Introduce wildcard service entries to drop traffic towards unknown protocol/port combinations#40684
borkmann merged 8 commits intocilium:mainfrom
ajmmm:pr/slb-drop-wildcard-vip

Conversation

@ajmmm
Copy link
Copy Markdown
Member

@ajmmm ajmmm commented Jul 23, 2025

PR to resolve a forwarding bug in data path.

This work is derived from work undertaken by Mikael Knutsson mikael.knutsson@gmail.com so has been credited in the relevant commit logs.

Summary of issue

Given a configured LoadBalancer or Cluster IP on a specific L4 destination port and protocol, ingress traffic towards the IP on an alternate destination port or protocol will not match eBPF service map lookup, so traffic be punted back to the network. Where the upstream network re-delivers the traffic back to Cilium, a packet loop is formed.

Summary of Changes

  • Introduces the concept of "wildcard" service entries in eBPF Service Maps, being a service entry with a zero destination and protocol field.
  • LoadBalancer and ClusterIP frontend services with external scope will be cross-referenced to a wildcard entry:
    • The first reference will trigger addition of wildcard entry in eBPF
    • The last reference deleted will trigger removal of wildcard entry from eBPF
  • Data path behaviour modified on traffic ingress. Where service lookup fails, an additional lookup is undertaken with zero destination and protocol. On successful match to such a wildcard entry, traffic is dropped.
  • Removal of data path logic that maintained compatibility with eBPF maps programmed by older versions of Cilium. Protocol differentiation was deprecated in 1.17, and automatic conversion of undifferentiated protocol was added in 1.18.
  • Appropriate unit test updates for the new logic
  • Minor typo fixes in unit test logic

Release Notes:

Introduce wildcard service entries to ensure traffic towards a LoadBalancer and ClusterIPs with an unknown protocol/port combination is dropped by the data path, rather than being forwarded back to the network.

@maintainer-s-little-helper maintainer-s-little-helper bot added the dont-merge/needs-release-note-label The author needs to describe the release impact of these changes. label Jul 23, 2025
@github-actions github-actions bot added the kind/community-contribution This was a contribution made by a community member. label Jul 23, 2025
@ajmmm ajmmm force-pushed the pr/slb-drop-wildcard-vip branch 12 times, most recently from bd5a7e7 to f5a6f60 Compare July 29, 2025 14:01
@ajmmm ajmmm marked this pull request as ready for review July 29, 2025 14:31
@ajmmm ajmmm requested review from a team as code owners July 29, 2025 14:31
@ajmmm ajmmm requested a review from dylandreimerink July 29, 2025 14:31
@ajmmm ajmmm force-pushed the pr/slb-drop-wildcard-vip branch from fb75646 to adf633a Compare July 29, 2025 16:02
@ajmmm ajmmm requested a review from joamaki July 29, 2025 16:10
@ajmmm ajmmm force-pushed the pr/slb-drop-wildcard-vip branch from adf633a to c391c35 Compare July 31, 2025 08:50
@ajmmm ajmmm force-pushed the pr/slb-drop-wildcard-vip branch 4 times, most recently from 8b5e388 to f408d79 Compare August 1, 2025 08:42
Copy link
Copy Markdown
Member

@dylandreimerink dylandreimerink left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great overall. One point of feedback though.

@ajmmm ajmmm force-pushed the pr/slb-drop-wildcard-vip branch from f408d79 to 6b2266c Compare August 1, 2025 11:27
@ajmmm
Copy link
Copy Markdown
Member Author

ajmmm commented Aug 22, 2025

/ci-e2e-upgrade

@ajmmm
Copy link
Copy Markdown
Member Author

ajmmm commented Aug 27, 2025

/ci-ginkgo

ajmmm added 8 commits August 27, 2025 10:28
…ice().

Previously, if the lookup of a service derived from ingress 5-tuple results
in no match, we undertake an additional lookup with IPPROTO_ANY. This was
necessary to maintain backwards compatibility with earlier versions of Cilium
that did not support protocol differentiation when programming the data path.

Cilium 1.18 introduced logic to migrate IPPROTO_ANY service/backend BPF map
entries to an appropriate protocol automatically, such that the additional
lookups are no longer required.

Appropriate tests updated to include explicit lookups for IPPROTO_TCP,
IPPROTO_UDP, and mismatched protocol checks.

Signed-off-by: Alasdair McWilliam <alasdair.mcwilliam@isovalent.com>
This change modifies the behaviour of lbXX_lookup_service() such that if an external
service lookup does not find an entry, we undertake an additional lookup with DPort=0
and Protocol=0. This will facilitate a wildcard lookup that, when programmed with a
suitable wildcard entry by the control plane, will trigger the data path to drop
unmatched ingress traffic.

This change modifies the semantics of said routines such that they no longer propagate
changes to the key protocol field, unless a wildcard service entry was found in the
external scope, and returned. In this case, the dport and protocol field will be zero.

Signed-off-by: Alasdair McWilliam <alasdair.mcwilliam@isovalent.com>
Signed-off-by: Mikael Knutsson <mikael.knutsson@gmail.com>
…paths.

Previously, lbX_lookup_service() would zeroise the key protocol field,
such that callers would	need to	reinstate the key protocol field if their
logic re-used the key.

Following the previous commit, the lbX_lookup_service()	routines will now
only zeroise the key protocol (and dport) fields if an external	wild-card
entry is returned. Thus, the restoration of key values can be removed.

This commit also removes lbX_key_set_protocol() helpers as they became
broadly unnecessary.

Signed-off-by: Alasdair McWilliam <alasdair.mcwilliam@isovalent.com>
This set of tests verifies that if we receive a packet towards a VIP on either
an unknown destination port, or unknown protocol, the data plane will correctly
drop if a wildcard entry is present in the service map.

Signed-off-by: Alasdair McWilliam <alasdair.mcwilliam@isovalent.com>
This change extends the nodePortAddrKey structure to include the L4 Protocol
number of an address, such that addresses with differing addresses will be
better distributed in the nodePortAddrByPort map.

This required the introduction of a helper routine to translate the L4Addr
protocol field into the real IANA number.

Signed-off-by: Alasdair McWilliam <alasdair.mcwilliam@isovalent.com>
Introduce logic in bpf_reconciler to map frontend service entries to wildcard
service entries, and program them in the data path, for LoadBalancer and ClusterIP
frontends of external scope. This will ensure the data path drops traffic towards
unknown destination ports and/or protocols.

Wildcard state is tracked as a map (indexed by IP Address) that contains a slice
of Frontend ServiceIDs. The first entry that requires a wildcard will trigger
programming of the data path entry. The last entry that depends on a wildcard
entry to be removed will trigger removal of the wildcard entry.

Signed-off-by: Alasdair McWilliam <alasdair.mcwilliam@isovalent.com>
Signed-off-by: Mikael Knutsson <mikael.knutsson@gmail.com>
Update the unit tests for loadbalancer and ciliumenvoyconfig to include
wildcard entries.

Signed-off-by: Alasdair McWilliam <alasdair.mcwilliam@isovalent.com>
Remove lb_v4_add_mixed_proto_service_with_flags() as it no longer makes sense.

Signed-off-by: Alasdair McWilliam <alasdair.mcwilliam@isovalent.com>
@ajmmm ajmmm force-pushed the pr/slb-drop-wildcard-vip branch from fe966e5 to abf0fec Compare August 27, 2025 09:28
@ajmmm
Copy link
Copy Markdown
Member Author

ajmmm commented Aug 27, 2025

/test

@ajmmm
Copy link
Copy Markdown
Member Author

ajmmm commented Aug 27, 2025

ci-ginkgo failures is likely a flake - happens across different tests on main, eg:

https://github.com/cilium/cilium/actions/workflows/conformance-ginkgo.yaml?query=is%3Afailure+branch%3Amain

@borkmann borkmann merged commit 3091b2e into cilium:main Aug 27, 2025
67 of 68 checks passed
@pippolo84 pippolo84 mentioned this pull request Sep 24, 2025
17 tasks
@ajmmm ajmmm deleted the pr/slb-drop-wildcard-vip branch September 29, 2025 10:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/community-contribution This was a contribution made by a community member. release-note/minor This PR changes functionality that users may find relevant to operating Cilium.

Projects

No open projects
Status: Released

Development

Successfully merging this pull request may close these issues.

9 participants