Skip to content

Cilium endpoint list is empty and network policies doesn't work #2053

@thoslin

Description

@thoslin

Bug reports

Title

Cilium endpoint list is empty and network policies doesn't work

General Information

  • Cilium version (run cilium version)
Cilium 0.13.90 3a5da2c Thu, 16 Nov 2017 00:39:22 +0100 go version go1.9 linux/amd64

Actually I'm using this docker image https://hub.docker.com/r/cilium/cilium/builds/bkdezkwu8ga7pjtbtzhaggj/

  • Kernel version (run uname -a)
Linux ip-172-20-34-175.ec2.internal 4.13.9-coreos #1 SMP Thu Oct 26 03:21:00 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
  • Orchestration system version in use (e.g. kubectl version, Mesos, ...)
Client Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.1", GitCommit:"f38e43b221d08850172a9a4ea785a86a3ffa3b3a", GitTreeState:"clean", BuildDate:"2017-10-11T23:27:35Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"7", GitVersion:"v1.7.2", GitCommit:"922a86cfcd65915a9b2f69f3f193b8907d741d9c", GitTreeState:"clean", BuildDate:"2017-07-21T08:08:00Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}

The cluster is created using Kops with 4 worker nodes.

How to reproduce the issue

I'm following the instructions on the Getting Start with Kubernetes page http://cilium.readthedocs.io/en/latest/gettingstarted/#getting-started-using-kubernetes.

  1. Install cilium
    Since the version of etcd on my cluster is lower than v3.1.0. As a workround, I deployed a consul cluster as kv store.
    I have 4 worker nodes. Cilium on each node starts Ok. However with a few error logs as followed:
time="2017-11-16T07:41:09Z" level=error msg="bpf: Unable to update in tunnel endpoint map" error="Unable to get object /sys/fs/bpf/tc/globals/tunnel_endpoint_map: no such file or directory" ipAddr=100.96.3.0/24
time="2017-11-16T07:41:09Z" level=error msg="bpf: Unable to update in tunnel endpoint map" error="Unable to update element: invalid argument" ipAddr="f00d::6460:300:0:0/112"

time="2017-11-16T07:41:16Z" level=error msg="bpf: Unable to delete in tunnel endpoint map" error="Unable to delete element: invalid argument" ipAddr=100.96.3.0/24
time="2017-11-16T07:41:16Z" level=error msg="bpf: Unable to delete in tunnel endpoint map" error="Unable to delete element: invalid argument" ipAddr="f00d::6460:300:0:0/112"
time="2017-11-16T07:41:16Z" level=error msg="bpf: Unable to update in tunnel endpoint map" error="Unable to update element: invalid argument" ipAddr=100.96.3.0/24
time="2017-11-16T07:41:16Z" level=error msg="bpf: Unable to update in tunnel endpoint map" error="Unable to update element: invalid argument" ipAddr="f00d::6460:300:0:0/96"

The full log could be found from here: https://pastebin.com/E9Ecv6us. I'm not sure if this error is related to the issue.

Cilium status shows OK on all nodes though:

/# cilium status                                                                                                                                                 
KVStore:            Ok   Consul: 100.124.241.78:8300
ContainerRuntime:   Ok   
Kubernetes:         Ok   OK
Kubernetes APIs:    ["core/v1::Service", "core/v1::Endpoint", "extensions/v1beta1::Ingress", "core/v1::Node", "CustomResourceDefinition", "cilium/v2::CiliumNetworkPolicy", "extensions/v1beta1::NetworkPolicy", "networking.k8s.io/v1::NetworkPolicy"]
Cilium:             Ok   OK
NodeMonitor:        Listening for events on 2 CPUs with 64x4096 of shared memory
  1. Deploy the demo

Deploy the demo from https://github.com/cilium/cilium/blob/master/examples/minikube/demo.yaml

  1. Check endpoint list
$ kubectl -n kube-system exec cilium-xhpvt cilium endpoint list
ENDPOINT   POLICY        IDENTITY   LABELS (source:key[=value])   IPv6   IPv4   STATUS   
           ENFORCEMENT                                                                   

Nothing shows up in the endpoint list. And network policies doesn't work, ie, restricted apps are still accessible.

Now I'm not sure how to proceed with this issue, which part should I check? Any tools to debug? I tried cilium monitor, but no packets are captured.

Any advice would be appreciated.Thanks!

Metadata

Metadata

Assignees

Labels

kind/bugThis is a bug in the Cilium logic.kind/community-reportThis was reported by a user in the Cilium community, eg via Slack.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions