Setting --device with the private iface seems to break multi-node connectivity in multiple scenarios. For example, commit 7dc7409 implements that change in CI and https://jenkins.cilium.io/job/Cilium-PR-K8s-newest-kernel-4.9/654/testReport/ has the corresponding results. The following tests are failing:
Suite-k8s-1.18.K8sPolicyTest Basic Test Allows traffic with k8s default-allow egress policy
Suite-k8s-1.18.K8sPolicyTest Basic Test Validate to-entities policies Validate toEntities All
Suite-k8s-1.18.K8sPolicyTest Basic Test Validate to-entities policies Validate toEntities World
Suite-k8s-1.18.K8sPolicyTest Basic Test Validate to-entities policies Validate toEntities Cluster
Suite-k8s-1.18.K8sPolicyTest Basic Test Traffic redirections to proxy Tests DNS proxy visibility without policy
Suite-k8s-1.18.K8sPolicyTest Multi-node policy test validates fromEntities policies with remote-node identity disabled Allows from all hosts with cnp fromEntities host policy
Suite-k8s-1.18.K8sPolicyTest Multi-node policy test validates fromEntities policies with remote-node identity enabled Validates fromEntities remote-node policy
Suite-k8s-1.18.K8sIstioTest Istio Bookinfo Demo Tests bookinfo inter-service connectivity
Suite-k8s-1.18.K8sHealthTest checks cilium-health status between nodes
Suite-k8s-1.18.K8sFQDNTest Validate that multiple specs are working correctly
Suite-k8s-1.18.K8sConformance Portmap Chaining Check connectivity-check compliance with portmap chaining
Suite-k8s-1.18.K8sConformance Portmap Chaining Check one node connectivity-check compliance with portmap chaining
Suite-k8s-1.18.K8sDatapathConfig MonitorAggregation Checks that monitor aggregation restricts notifications
Suite-k8s-1.18.K8sDatapathConfig MonitorAggregation Checks that monitor aggregation flags send notifications
Suite-k8s-1.18.K8sDatapathConfig Encapsulation Check connectivity with sockops and VXLAN encapsulation
Suite-k8s-1.18.K8sDatapathConfig Encapsulation Check connectivity with VXLAN encapsulation
Suite-k8s-1.18.K8sDatapathConfig Encapsulation Check connectivity with Geneve encapsulation
Suite-k8s-1.18.K8sDatapathConfig AutoDirectNodeRoutes Check connectivity with automatic direct nodes routes
Suite-k8s-1.18.K8sDatapathConfig AutoDirectNodeRoutes Check direct connectivity with per endpoint routes
Suite-k8s-1.18.K8sDatapathConfig AutoDirectNodeRoutes Check connectivity with sockops and direct routing
Suite-k8s-1.18.K8sDatapathConfig Transparent encryption DirectRouting Check connectivity with transparent encryption and direct routing
Suite-k8s-1.18.K8sDatapathConfig IPv4Only Check connectivity with IPv6 disabled
Suite-k8s-1.18.K8sDatapathConfig Etcd Check connectivity
Suite-k8s-1.18.K8sHubbleTest Hubble Observe Test L3/L4 Flow with hubble-relay
Suite-k8s-1.18.K8sHubbleTest Hubble Observe Test L7 Flow with hubble-relay
Suite-k8s-1.18.K8sServicesTest Checks service across nodes Checks ClusterIP Connectivity
Suite-k8s-1.18.K8sServicesTest Checks service across nodes Tests NodePort (kube-proxy)
Suite-k8s-1.18.K8sServicesTest Checks service across nodes Tests NodePort (kube-proxy) with externalTrafficPolicy=Local
Suite-k8s-1.18.K8sServicesTest Checks service across nodes with L4 policy Tests NodePort with L4 Policy
Suite-k8s-1.18.K8sServicesTest Checks service across nodes with L7 policy Tests NodePort with L7 Policy
Suite-k8s-1.18.K8sKafkaPolicyTest Kafka Policy Tests KafkaPolicies
Suite-k8s-1.18.K8sChaosTest Connectivity demo application Endpoint can still connect while Cilium is not running
Suite-k8s-1.18.K8sDemosTest Tests Star Wars Demo
I am able to reproduce locally, although the connectivity checks work fine with --device=enp0s3,enp0s8 in the multi-node dev. Vagrant setup.
I'm setting as priority/release-blocker because it is needed for the host firewall.
/cc @brb
Related: #11969, #11799.
Setting
--devicewith the private iface seems to break multi-node connectivity in multiple scenarios. For example, commit 7dc7409 implements that change in CI and https://jenkins.cilium.io/job/Cilium-PR-K8s-newest-kernel-4.9/654/testReport/ has the corresponding results. The following tests are failing:I am able to reproduce locally, although the connectivity checks work fine with
--device=enp0s3,enp0s8in the multi-node dev. Vagrant setup.I'm setting as
priority/release-blockerbecause it is needed for the host firewall./cc @brb
Related: #11969, #11799.