Skip to content

cilium: bpf-based hostport implementation#10592

Merged
borkmann merged 3 commits intomasterfrom
pr/bpf-host-port
Mar 18, 2020
Merged

cilium: bpf-based hostport implementation#10592
borkmann merged 3 commits intomasterfrom
pr/bpf-host-port

Conversation

@borkmann
Copy link
Copy Markdown
Member

@borkmann borkmann commented Mar 16, 2020 •
edited
Loading

See commit msg.

This change is Reviewable

@borkmann borkmann added wip area/datapath Impacts bpf/ or low-level forwarding details, including map management and monitor messages. area/daemon Impacts operation of the Cilium daemon. release-note/minor This PR changes functionality that users may find relevant to operating Cilium. labels Mar 16, 2020
@borkmann borkmann requested a review from a team March 16, 2020 16:09
@borkmann borkmann requested a review from a team as a code owner March 16, 2020 16:09
@borkmann borkmann requested a review from a team March 16, 2020 16:09
@borkmann borkmann requested review from a team as code owners March 16, 2020 16:09
@borkmann borkmann requested review from a team March 16, 2020 16:09
tklauser
tklauser reviewed Mar 16, 2020
View reviewed changes
Comment thread cilium/cmd/service_update.go Outdated
@borkmann borkmann force-pushed the pr/bpf-host-port branch 4 times, most recently from ae0add1 to 1da3d05 Compare March 16, 2020 21:39
@borkmann borkmann changed the title cilium: bpf-based hostport implementation [wip] cilium: bpf-based hostport implementation Mar 16, 2020
@borkmann
Copy link
Copy Markdown
Member Author

borkmann commented Mar 16, 2020

test-me-please

@borkmann borkmann requested a review from brb March 16, 2020 21:51
@coveralls
Copy link
Copy Markdown

coveralls commented Mar 16, 2020 •
edited
Loading

Coverage Status

Coverage decreased (-0.1%) to 45.597% when pulling 7616447 on pr/bpf-host-port into f6e4ad5 on master.

@borkmann
Copy link
Copy Markdown
Member Author

borkmann commented Mar 16, 2020

test-me-please

@borkmann
Copy link
Copy Markdown
Member Author

borkmann commented Mar 16, 2020

test-docs-please

@borkmann
Copy link
Copy Markdown
Member Author

borkmann commented Mar 16, 2020

test-docs-please

@borkmann
Copy link
Copy Markdown
Member Author

borkmann commented Mar 16, 2020

test-me-please

@borkmann
Copy link
Copy Markdown
Member Author

borkmann commented Mar 17, 2020

test-docs-please

brb
brb approved these changes Mar 17, 2020
View reviewed changes
Copy link
Copy Markdown
Member

@brb brb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Couple nits and questions.

Comment thread Documentation/gettingstarted/kubeproxy-free.rst Outdated
Copy link
Copy Markdown
Member

@brb brb Mar 17, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: kubectl run nginx --generator=run-pod/v1 --image=nginx --replicas=1 --port=80 --hostport=8080?

Copy link
Copy Markdown
Member Author

@borkmann borkmann Mar 17, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was referring to the example yaml from earlier in the same doc, so I kept it as is but with the added hostPort.

Comment thread Documentation/gettingstarted/kubeproxy-free.rst Outdated
Comment thread install/kubernetes/cilium/values.yaml Outdated
Copy link
Copy Markdown
Member

@brb brb Mar 17, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to disable the hostPort feature for new users? Asking, as for the existing users after the upgrade to v1.8 the feature will be enabled due to the agent defaulting the flag to true.

Copy link
Copy Markdown
Member Author

@borkmann borkmann Mar 17, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would keep it enabled otherwise people might miss it and still deploy the portmap CNI chaining. I'll do a follow-up for the upgrade guide to document the support and actions needed.

@borkmann
Copy link
Copy Markdown
Member Author

borkmann commented Mar 17, 2020

test-docs-please

@borkmann
Copy link
Copy Markdown
Member Author

borkmann commented Mar 17, 2020

test-me-please

@borkmann
Copy link
Copy Markdown
Member Author

borkmann commented Mar 17, 2020

test-docs-please

@borkmann
Copy link
Copy Markdown
Member Author

borkmann commented Mar 18, 2020

test-me-please

@borkmann
Copy link
Copy Markdown
Member Author

borkmann commented Mar 18, 2020

test-me-please

@borkmann
Copy link
Copy Markdown
Member Author

borkmann commented Mar 18, 2020

test-docs-please

@borkmann
Copy link
Copy Markdown
Member Author

borkmann commented Mar 18, 2020

test-me-please

@borkmann
Copy link
Copy Markdown
Member Author

borkmann commented Mar 18, 2020

test-docs-please

@borkmann
Copy link
Copy Markdown
Member Author

borkmann commented Mar 18, 2020

(prior docs test was green)

@borkmann
Copy link
Copy Markdown
Member Author

borkmann commented Mar 18, 2020

test-me-please

@borkmann
Copy link
Copy Markdown
Member Author

borkmann commented Mar 18, 2020

(prior travis ci was green, x86 is green here, arm64 flaky in unrelated spot)

@joestringer joestringer mentioned this pull request Mar 18, 2020
Closed
borkmann added 3 commits March 18, 2020 21:27
@borkmann
cilium, bpf: add native HostPort mapping support
399bbd6 
In order to use hostPort today, users can only consume it via deploying
Cilium through --set global.cni.chainingMode=portmap in order to then
have iptables set up for each pod that specifies a hostPort. We almost
have all the BPF infrastructure in place for natively supporting BPF
based hostPort. This work implements the hostPort feature as a service
mapping. It is part of the kube-proxy-free implementation since it
depends on NodePort infrastructure.

Example:

  # ./daemon/cilium-agent --identity-allocation-mode=crd --enable-ipv6=true --enable-ipv4=true --disable-envoy-version-check=true --tunnel=disabled --k8s-kubeconfig-path=$HOME/.kube/config --kube-proxy-replacement=strict --enable-l7-proxy=false

  # cat hostport.yaml
  apiVersion: v1
  kind: Pod
  metadata:
    name: nginx
  spec:
    containers:
      - name: nginx
        image: nginx
        ports:
          - containerPort: 80
            hostPort: 90
  # kubectl apply -f ./hostport.yaml
  pod/nginx created

  # ./cilium/cilium service list
  ID   Frontend            Service Type   Backend
  1    10.96.0.1:443       ClusterIP      1 => 192.168.178.29:6443
  2    10.96.0.10:53       ClusterIP      1 => 10.29.6.103:53
                                          2 => 10.29.24.228:53
  3    10.96.0.10:9153     ClusterIP      1 => 10.29.6.103:9153
                                          2 => 10.29.24.228:9153
  4    192.168.178.29:90   HostPort       1 => 10.29.245.35:80

From own node:

  # curl 192.168.178.29:90
  <!DOCTYPE html>
  <html>
  <head>
  [...]

From remote node:

  root@tank:~# curl 192.168.178.29:90
  <!DOCTYPE html>
  <html>
  <head>
  [...]

The HostPort implementation is a hybrid of ClusterIP and NodePort in
terms of implementation in that making it reachable on the own node is
just a single service map entry (as opposed to further exposing it via
loopback address, etc) and for external traffic the backend is always
local to the node and can be mapped from any port.

Fixes: #10359
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
@borkmann
cilium, helm, docs: update kube-proxy free's gsg to document HostPort
2391e53 
Document the new HostPort mapping feature under the advanced section and
add Helm support for it. I've also added a small setup validation as I
think it's quite useful.

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
@borkmann
tests: add HostPort connectivity tests to k8sT service tests
7616447 
Add connectivity test cases from internal and external.

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
@borkmann
Copy link
Copy Markdown
Member Author

borkmann commented Mar 18, 2020

test-me-please

@borkmann borkmann merged commit ebd7392 into master Mar 18, 2020
@borkmann borkmann deleted the pr/bpf-host-port branch March 18, 2020 22:58
@borkmann borkmann mentioned this pull request Mar 18, 2020
Closed
@brb brb mentioned this pull request Mar 27, 2020
Merged
@dghubble dghubble mentioned this pull request Jun 16, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@brb brb brb approved these changes

@tklauser tklauser tklauser left review comments

@joestringer joestringer joestringer approved these changes

@aanm aanm aanm approved these changes

Assignees

No one assigned

Labels

area/daemon Impacts operation of the Cilium daemon. area/datapath Impacts bpf/ or low-level forwarding details, including map management and monitor messages. release-note/minor This PR changes functionality that users may find relevant to operating Cilium.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@borkmann @coveralls @aanm @brb @tklauser @joestringer