Bug report
General Information
- Cilium version (run
cilium version)
Client: 1.8.4 17d623853 2020-09-30T17:31:44-07:00 go version go1.14.9 linux/amd64
Daemon: 1.8.4 17d623853 2020-09-30T17:31:44-07:00 go version go1.14.9 linux/amd64
- Kernel version (run
uname -a)
Linux ip-10-105-14-33 5.4.0-1028-aws #29-Ubuntu SMP Mon Oct 5 15:30:10 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
- Orchestration system version in use (e.g.
kubectl version, Mesos, ...)
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.8", GitCommit:"9f2892aab98fe339f3bd70e3c470144299398ace", GitTreeState:"clean", BuildDate:"2020-08-13T16:12:48Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.13", GitCommit:"30d651da517185653e34e7ab99a792be6a3d9495", GitTreeState:"clean", BuildDate:"2020-10-15T00:59:17Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"linux/amd64"}
- Link to relevant artifacts (policies, deployments scripts, ...)
On these nodes, ephemeral port range is
$ cat /proc/sys/net/ipv4/ip_local_port_range
32768 60999
- Generate and upload a system zip:
curl -sLO https://git.io/cilium-sysdump-latest.zip && python cilium-sysdump-latest.zip
was unable to generate the dump, sorry
How to reproduce the issue
- Enable Hubble metrics
- Make sure
port-distribution metric is enabled
- See
port-distribution metric cardinality explode due to alleged UDP requests (from ephemeral port range).
On the target system, running Cilium monitor shows UDP traffic:
$ cilium monitor | grep ud
-> endpoint 2669 flow 0x0 identity 11980->49072 state reply ifindex lxcdaea9e2f5946 orig-ip 100.64.9.148: 100.96.0.10:53 -> 100.64.9.230:44005 udp
-> endpoint 2631 flow 0x3a6e9cf4 identity 49072->11980 state established ifindex lxc1775c9458c08 orig-ip 100.64.9.230: 100.64.9.230:44005 -> 100.64.9.148:53 udp
-> endpoint 2669 flow 0x0 identity 11980->49072 state reply ifindex lxcdaea9e2f5946 orig-ip 100.64.9.148: 100.96.0.10:53 -> 100.64.9.230:44005 udp
-> endpoint 2631 flow 0x3a6e9cf4 identity 49072->11980 state established ifindex lxc1775c9458c08 orig-ip 100.64.9.230: 100.64.9.230:44005 -> 100.64.9.148:53 udp
...
but adding grep -v ':53' produces no results. This suggests that all UDP traffic observed is DNS. There are no services that would generate UDP traffic in this cluster other than CoreDNS.
Bug report
General Information
cilium version)uname -a)kubectl version, Mesos, ...)On these nodes, ephemeral port range is
was unable to generate the dump, sorry
How to reproduce the issue
port-distributionmetric is enabledport-distributionmetric cardinality explode due to alleged UDP requests (from ephemeral port range).On the target system, running Cilium monitor shows UDP traffic:
but adding
grep -v ':53'produces no results. This suggests that all UDP traffic observed is DNS. There are no services that would generate UDP traffic in this cluster other than CoreDNS.