Skip to content

CI: K8sServicesTest Checks service across nodes Tests NodePort BPF Tests with direct routing #9297

@joestringer

Description

@joestringer

https://jenkins.cilium.io/job/Cilium-PR-Ginkgo-Tests-Validated/15184/testReport/junit/Suite-k8s-1/11/K8sServicesTest_Checks_service_across_nodes_Tests_NodePort_BPF_Tests_with_direct_routing/

Found in a PR build, but the PR was unrelated to nodeport or datapath changes.

Stacktrace

/home/jenkins/workspace/Cilium-PR-Ginkgo-Tests-Validated/k8s-1.11-gopath/src/github.com/cilium/cilium/test/ginkgo-ext/scopes.go:420
Pod "testclient-kw9ct" can not connect to service "http://192.168.36.11:31365"
Expected command: kubectl exec -n default testclient-kw9ct -- curl --path-as-is -s -D /dev/stderr --fail --connect-timeout 5 --max-time 8 http://192.168.36.11:31365 -w "time-> DNS: '%{time_namelookup}(%{remote_ip})', Connect: '%{time_connect}',Transfer '%{time_starttransfer}', total '%{time_total}'" 
To succeed, but it failed:
Exitcode: 28 
Stdout:
 	 time-> DNS: '0.000020()', Connect: '0.000000',Transfer '0.000000', total '5.001223'
Stderr:
 	 command terminated with exit code 28
	 

/home/jenkins/workspace/Cilium-PR-Ginkgo-Tests-Validated/k8s-1.11-gopath/src/github.com/cilium/cilium/test/k8sT/Services.go:235

Standard Output

Number of "context deadline exceeded" in logs: 0
Number of "level=error" in logs: 0
Number of "level=warning" in logs: 0
Number of "Cilium API handler panicked" in logs: 0
Number of "Goroutine took lock for more than" in logs: 0
No errors/warnings found in logs
Cilium pods: [cilium-7fvsd cilium-9g7bf]
Netpols loaded: 
CiliumNetworkPolicies loaded: 
Endpoint Policy Enforcement:
Pod                        Ingress   Egress
testds-sc4xb               false     false
coredns-687db6485c-vcr9k   false     false
testclient-fkhxr           false     false
testclient-kw9ct           false     false
testds-5jgmm               false     false
Cilium agent 'cilium-7fvsd': Status: Ok  Health: Ok Nodes "" ContinerRuntime: Ok Kubernetes: Ok KVstore: Ok Controllers: Total 23 Failed 0
Cilium agent 'cilium-9g7bf': Status: Ok  Health: Ok Nodes "" ContinerRuntime: Ok Kubernetes: Ok KVstore: Ok Controllers: Total 28 Failed 0

Standard Error

STEP: Installing Cilium
STEP: Installing DNS Deployment
STEP: Performing Cilium preflight check
STEP: Performing Cilium status preflight check
STEP: Performing Cilium controllers preflight check
STEP: Performing Cilium health check
STEP: Performing Cilium service preflight check
STEP: Performing K8s service preflight check
STEP: Waiting for cilium-operator to be ready
STEP: Waiting for kube-dns to be ready
STEP: Running kube-dns preflight check
STEP: Performing K8s service preflight check
STEP: Making ten HTTP requests from "testclient-fkhxr" to "http://10.109.184.104:10080"
STEP: Making ten HTTP requests from "testclient-kw9ct" to "http://10.109.184.104:10080"
STEP: Making 10 HTTP requests from k8s1 to "http://127.0.0.1:31365"
STEP: Making 10 HTTP requests from k8s1 to "http://192.168.36.11:31365"
STEP: Making 10 HTTP requests from k8s1 to "http://192.168.36.12:31365"
STEP: Making ten HTTP requests from "testclient-fkhxr" to "http://192.168.36.11:31365"
STEP: Making ten HTTP requests from "testclient-kw9ct" to "http://192.168.36.11:31365"
=== Test Finished at 2019-09-27T20:20:09Z====
===================== TEST FAILED =====================
cmd: kubectl get pods -o wide --all-namespaces
Exitcode: 0 
Stdout:
 	 NAMESPACE     NAME                              READY   STATUS    RESTARTS   AGE   IP              NODE   NOMINATED NODE
	 default       testclient-fkhxr                  1/1     Running   0          3m    10.10.1.237     k8s2   <none>
	 default       testclient-kw9ct                  1/1     Running   0          3m    10.10.0.110     k8s1   <none>
	 default       testds-5jgmm                      1/1     Running   0          3m    10.10.1.186     k8s2   <none>
	 default       testds-sc4xb                      1/1     Running   0          3m    10.10.0.158     k8s1   <none>
	 kube-system   cilium-7fvsd                      1/1     Running   0          1m    192.168.36.11   k8s1   <none>
	 kube-system   cilium-9g7bf                      1/1     Running   0          1m    192.168.36.12   k8s2   <none>
	 kube-system   cilium-operator-786886584-8g7zt   1/1     Running   0          5m    192.168.36.12   k8s2   <none>
	 kube-system   coredns-687db6485c-vcr9k          1/1     Running   0          17m   10.10.1.81      k8s2   <none>
	 kube-system   etcd-k8s1                         1/1     Running   0          16m   192.168.36.11   k8s1   <none>
	 kube-system   kube-apiserver-k8s1               1/1     Running   0          16m   192.168.36.11   k8s1   <none>
	 kube-system   kube-controller-manager-k8s1      1/1     Running   0          16m   192.168.36.11   k8s1   <none>
	 kube-system   kube-proxy-hjknv                  1/1     Running   0          17m   192.168.36.11   k8s1   <none>
	 kube-system   kube-proxy-m6rsf                  1/1     Running   0          6m    192.168.36.12   k8s2   <none>
	 kube-system   kube-scheduler-k8s1               1/1     Running   0          16m   192.168.36.11   k8s1   <none>
	 kube-system   log-gatherer-gtvnj                1/1     Running   0          5m    192.168.36.11   k8s1   <none>
	 kube-system   log-gatherer-tkjvd                1/1     Running   0          5m    192.168.36.12   k8s2   <none>
	 
Stderr:
 	 

Fetching command output from pods [cilium-7fvsd cilium-9g7bf]
cmd: kubectl exec -n kube-system cilium-7fvsd -- cilium service list
Exitcode: 0 
Stdout:
 	 ID   Frontend               Backend                   
	 1    10.96.0.1:443          1 => 192.168.36.11:6443   
	 2    10.96.0.10:53          1 => 10.10.1.81:53        
	 3    10.10.0.144:31365      1 => 10.10.1.186:80       
	                             2 => 10.10.0.158:80       
	 4    10.105.14.20:80        1 => 10.10.1.186:80       
	                             2 => 10.10.0.158:80       
	 5    10.109.184.104:10080   1 => 10.10.1.186:80       
	                             2 => 10.10.0.158:80       
	 6    0.0.0.0:31365          1 => 10.10.1.186:80       
	                             2 => 10.10.0.158:80       
	 7    192.168.36.11:31365    1 => 10.10.1.186:80       
	                             2 => 10.10.0.158:80       
	 
Stderr:
 	 

cmd: kubectl exec -n kube-system cilium-7fvsd -- cilium endpoint list
Exitcode: 0 
Stdout:
 	 ENDPOINT   POLICY (ingress)   POLICY (egress)   IDENTITY   LABELS (source:key[=value])                       IPv6                 IPv4          STATUS   
	            ENFORCEMENT        ENFORCEMENT                                                                                                       
	 429        Disabled           Disabled          38885      k8s:io.cilium.k8s.policy.cluster=default          f00d::a0a:0:0:98e2   10.10.0.110   ready   
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=default                                              
	                                                            k8s:io.kubernetes.pod.namespace=default                                                      
	                                                            k8s:zgroup=testDSClient                                                                      
	 660        Disabled           Disabled          43962      k8s:io.cilium.k8s.policy.cluster=default          f00d::a0a:0:0:1eb2   10.10.0.158   ready   
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=default                                              
	                                                            k8s:io.kubernetes.pod.namespace=default                                                      
	                                                            k8s:zgroup=testDS                                                                            
	 1292       Disabled           Disabled          4          reserved:health                                   f00d::a0a:0:0:631c   10.10.0.58    ready   
	 
Stderr:
 	 

cmd: kubectl exec -n kube-system cilium-9g7bf -- cilium service list
Exitcode: 0 
Stdout:
 	 ID   Frontend               Backend                   
	 1    10.96.0.1:443          1 => 192.168.36.11:6443   
	 2    10.96.0.10:53          1 => 10.10.1.81:53        
	 3    0.0.0.0:31365          1 => 10.10.1.186:80       
	                             2 => 10.10.0.158:80       
	 4    10.105.14.20:80        1 => 10.10.1.186:80       
	                             2 => 10.10.0.158:80       
	 5    10.109.184.104:10080   1 => 10.10.1.186:80       
	                             2 => 10.10.0.158:80       
	 6    192.168.36.12:31365    1 => 10.10.1.186:80       
	                             2 => 10.10.0.158:80       
	 7    10.10.1.122:31365      1 => 10.10.1.186:80       
	                             2 => 10.10.0.158:80       
	 
Stderr:
 	 

cmd: kubectl exec -n kube-system cilium-9g7bf -- cilium endpoint list
Exitcode: 0 
Stdout:
 	 ENDPOINT   POLICY (ingress)   POLICY (egress)   IDENTITY   LABELS (source:key[=value])                       IPv6                   IPv4          STATUS   
	            ENFORCEMENT        ENFORCEMENT                                                                                                         
	 762        Disabled           Disabled          38885      k8s:io.cilium.k8s.policy.cluster=default          f00d::a0a:100:0:fbff   10.10.1.237   ready   
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=default                                                
	                                                            k8s:io.kubernetes.pod.namespace=default                                                        
	                                                            k8s:zgroup=testDSClient                                                                        
	 1092       Disabled           Disabled          43962      k8s:io.cilium.k8s.policy.cluster=default          f00d::a0a:100:0:5f39   10.10.1.186   ready   
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=default                                                
	                                                            k8s:io.kubernetes.pod.namespace=default                                                        
	                                                            k8s:zgroup=testDS                                                                              
	 1727       Disabled           Disabled          104        k8s:io.cilium.k8s.policy.cluster=default          f00d::a0a:100:0:14d    10.10.1.81    ready   
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=coredns                                                
	                                                            k8s:io.kubernetes.pod.namespace=kube-system                                                    
	                                                            k8s:k8s-app=kube-dns                                                                           
	 2751       Disabled           Disabled          4          reserved:health                                   f00d::a0a:100:0:a3c6   10.10.1.175   ready   
	 
Stderr:
 	 

===================== Exiting AfterFailed =====================

[[ATTACHMENT|f9f3a0c9_K8sServicesTest_Checks_service_across_nodes_Tests_NodePort_BPF_Tests_with_direct_routing.zip]]
STEP: Installing Cilium
STEP: Installing DNS Deployment
STEP: Performing Cilium preflight check
STEP: Performing Cilium status preflight check
STEP: Performing Cilium controllers preflight check
STEP: Performing Cilium health check
STEP: Performing Cilium service preflight check
STEP: Performing K8s service preflight check
STEP: Waiting for cilium-operator to be ready
STEP: Waiting for kube-dns to be ready
STEP: Running kube-dns preflight check
STEP: Performing K8s service preflight check

f9f3a0c9_K8sServicesTest_Checks_service_across_nodes_Tests_NodePort_BPF_Tests_with_direct_routing.zip

Metadata

Metadata

Assignees

Labels

area/CIContinuous Integration testing issue or flakearea/datapathImpacts bpf/ or low-level forwarding details, including map management and monitor messages.kind/duplicateThere is another issue which contains additional details.needs/triageThis issue requires triaging to establish severity and next steps.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions