Skip to content

cilium: hostport service map fixes#12446

Merged
borkmann merged 7 commits intomasterfrom
pr/hostport-fixes
Jul 9, 2020
Merged

cilium: hostport service map fixes#12446
borkmann merged 7 commits intomasterfrom
pr/hostport-fixes

Conversation

@borkmann
Copy link
Copy Markdown
Member

@borkmann borkmann commented Jul 7, 2020

See commit msg; targeted for 1.8.2.

Fixes: #12116

@maintainer-s-little-helper maintainer-s-little-helper Bot added the dont-merge/needs-release-note-label The author needs to describe the release impact of these changes. label Jul 7, 2020
@borkmann borkmann added the release-note/misc This PR makes changes that have no direct user impact. label Jul 7, 2020
@maintainer-s-little-helper maintainer-s-little-helper Bot removed the dont-merge/needs-release-note-label The author needs to describe the release impact of these changes. label Jul 7, 2020
@borkmann borkmann added dont-merge/needs-release-note-label The author needs to describe the release impact of these changes. needs-backport/1.8 labels Jul 7, 2020
@maintainer-s-little-helper maintainer-s-little-helper Bot removed the dont-merge/needs-release-note-label The author needs to describe the release impact of these changes. label Jul 7, 2020
@coveralls
Copy link
Copy Markdown

coveralls commented Jul 7, 2020

Coverage Status

Coverage decreased (-0.03%) to 36.93% when pulling 15476a9 on pr/hostport-fixes into 31f1306 on master.

Name it generically and add another bool that allows to control whether
the dst port must be in the nodeport range or outside of it. For the
current users, it is set so that they match on the configured NodePort
range.

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
@borkmann borkmann force-pushed the pr/hostport-fixes branch 11 times, most recently from 2864c48 to b279163 Compare July 8, 2020 14:44
@borkmann borkmann requested a review from brb July 8, 2020 14:45
@borkmann borkmann marked this pull request as ready for review July 8, 2020 14:45
@borkmann borkmann requested a review from a team July 8, 2020 14:45
@borkmann borkmann requested review from a team as code owners July 8, 2020 14:45
@borkmann borkmann requested a review from a team July 8, 2020 14:45
@borkmann
Copy link
Copy Markdown
Member Author

borkmann commented Jul 8, 2020

retest-runtime

@borkmann
Copy link
Copy Markdown
Member Author

borkmann commented Jul 8, 2020

retest-4.19

@borkmann
Copy link
Copy Markdown
Member Author

borkmann commented Jul 8, 2020

retest-net-next

@borkmann borkmann force-pushed the pr/hostport-fixes branch from e2e6836 to d47ab81 Compare July 9, 2020 06:43
@borkmann
Copy link
Copy Markdown
Member Author

borkmann commented Jul 9, 2020

test-me-please

Copy link
Copy Markdown
Member

@pchaigno pchaigno left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly refactoring nits on my side. LGTM otherwise, but I'm not very familiar with this code and even less with the Kubernetes implication.

Comment thread bpf/bpf_sock.c Outdated
Comment thread bpf/lib/lb.h Outdated
Comment thread pkg/k8s/watchers/pod.go Outdated
Comment thread pkg/k8s/watchers/pod.go Outdated
@pchaigno pchaigno requested a review from a team July 9, 2020 08:07
@borkmann
Copy link
Copy Markdown
Member Author

borkmann commented Jul 9, 2020

(all checks green)

@borkmann borkmann force-pushed the pr/hostport-fixes branch from d47ab81 to 53d15f9 Compare July 9, 2020 09:06
borkmann added 6 commits July 9, 2020 12:02
Prepare the socket LB for a hostport-related wildcard lookup. The NodePort
and HostPort range are exclusive, so the lookup is still fast given we can
bail out early w/o map lookup in sock{4,6}_wildcard_lookup(). Aside from
the range, we must match on HOST_ID dst address for the hostport lookup.

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Ensure that we don't process such packets from outside if they
ever land on our node. Instead of doing a more expensive memcmp()
on the address, just include this into the test for the underlying
type. The latter is still present for the case where the agent
is restarted & reconfigured and map entries are still stale before
k8s resync.

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
In IPv6 case we already use net.IPv6zero in LoadBalancerNodeAddresses().

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
In order to be compatible with portmap's HostPort mode, we need to expose the
host port on all addresses for the case where no hostIP was specified. This also
includes the loopback address. For the case where hostIP was specified, we only
expose the user provided IP address under HostPort.

A typical portmap rule looks like (removing CNI-HOSTPORT-SETMARK from here):

  [...]
  [6:408] -A PREROUTING -m addrtype --dst-type LOCAL -j CNI-HOSTPORT-DNAT
  [...]
  [0:0] -A CNI-DN-1ba85aa0447ede8ba6dab -d 8.8.8.8/32 -p tcp -m tcp --dport 999 -j DNAT --to-destination 10.0.0.217:80
  [0:0] -A CNI-DN-489a67438eed03d272897 -d 127.0.0.1/32 -p tcp -m tcp --dport 298 -j DNAT --to-destination 10.0.0.27:80
  [2:120] -A CNI-DN-526ac7f749f8260cb0d9f -p tcp -m tcp --dport 666 -j DNAT --to-destination 10.0.0.188:80
  [2:120] -A CNI-DN-81de08afe40acdfde3fac -d 192.168.178.29/32 -p tcp -m tcp --dport 777 -j DNAT --to-destination 10.0.0.138:80
  [2:120] -A CNI-HOSTPORT-DNAT -p tcp -m comment --comment "dnat name: \"portmap\" id: \"dd111758b3cc7d46e0c027725f4c1cc3d5ac349c6cb1d6ebb919a5f433f155aa\"" -m multiport --dports 666 -j CNI-DN-526ac7f749f8260cb0d9f
  [...]

In above example, hostPort 666 has no hostIP specified; hostPort 777 had the hostIP
of 192.168.178.29 specified. In such case access via 127.0.0.1:777 is not possible.
hostPort 298 had 127.0.0.1 as hostIP specified, so it's only available via loopback
but not from outside world. And hostPort 999 had 8.8.8.8 as hostIP specified. It's
not forbidden but it won't match/xlate if 8.8.8.8 is not a local address on the node.
Implement these semantics also for the BPF-based replacement by reworking the k8s
watcher's genServiceMappings().

Fixes: #12116
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Final one to enforce drop of surrogate entries for external traffic.

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Extend the batch of hostPort tests to improve coverage.

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
@borkmann borkmann force-pushed the pr/hostport-fixes branch from 53d15f9 to 15476a9 Compare July 9, 2020 10:03
@borkmann
Copy link
Copy Markdown
Member Author

borkmann commented Jul 9, 2020

test-me-please

@borkmann
Copy link
Copy Markdown
Member Author

borkmann commented Jul 9, 2020

(ignoring checkpatch in this case since line goes over ~85 chars)

@borkmann borkmann added the ready-to-merge This PR has passed all tests and received consensus from code owners to merge. label Jul 9, 2020
@borkmann borkmann merged commit 56e0e72 into master Jul 9, 2020
@borkmann borkmann deleted the pr/hostport-fixes branch July 9, 2020 13:06
Comment thread pkg/k8s/watchers/pod.go
Comment thread pkg/k8s/watchers/pod.go
@maintainer-s-little-helper maintainer-s-little-helper Bot removed ready-to-merge This PR has passed all tests and received consensus from code owners to merge. labels Jul 9, 2020
@brb brb mentioned this pull request Jul 15, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release-note/misc This PR makes changes that have no direct user impact.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cilium native hostport only listens on a internal IP, unlike CNI portmap

6 participants