Skip to content

CI: Verifier error in connect6: back-edge from insn 624 to 570 #11731

@pchaigno

Description

@pchaigno

https://jenkins.cilium.io/job/Cilium-PR-Ginkgo-Tests-Kernel/1740/testReport/

Cilium logs contain the following verifier error message and tests are failing:

msg="+ tc exec bpf pin /sys/fs/bpf/tc/globals/cilium_cgroups_connect6 obj bpf_sock.o type sockaddr attach_type connect6 sec connect6" subsys=datapath-loader
subsys=datapath-loader
msg="Prog section 'connect6' rejected: Invalid argument (22)!" subsys=datapath-loader
msg=" - Type:         18" subsys=datapath-loader
msg=" - Attach Type:  11" subsys=datapath-loader
msg=" - Instructions: 740 (0 over limit)" subsys=datapath-loader
msg=" - License:      GPL" subsys=datapath-loader
subsys=datapath-loader
msg="Verifier analysis:" subsys=datapath-loader
subsys=datapath-loader
msg="back-edge from insn 624 to 570" subsys=datapath-loader
subsys=datapath-loader
msg="Error fetching program/map!" subsys=datapath-loader

I've seen the above error in these tests:

Suite-k8s-1.17.K8sHealthTest checks cilium-health status between nodes
Suite-k8s-1.17.K8sKafkaPolicyTest Kafka Policy Tests KafkaPolicies
Suite-k8s-1.17.K8sDatapathConfig MonitorAggregation Checks that monitor aggregation restricts notifications
Suite-k8s-1.17.K8sDatapathConfig MonitorAggregation Checks that monitor aggregation flags send notifications

Because we have another error that causes tests with broken Cilium deploy to stay stuck for 40min, the build reached the 3h timeout after running only these 4 tests. I'm guessing other tests would have failed with the same error otherwise.

I'm unsure whether this is a flake or a complete breakage. I haven't had time to investigate further. I guess we'll soon see. It's a complete breakage; 4.19 pipeline is down :'-(

75e6fd64_K8sDatapathConfig_MonitorAggregation_Checks_that_monitor_aggregation_restricts_notifications.zip
199f0502_K8sKafkaPolicyTest_Kafka_Policy_Tests_KafkaPolicies.zip
58819d2a_K8sHealthTest_checks_cilium-health_status_between_nodes.zip
89886bcc_K8sDatapathConfig_MonitorAggregation_Checks_that_monitor_aggregation_flags_send_notifications.zip

Metadata

Metadata

Assignees

Labels

area/CIContinuous Integration testing issue or flakearea/datapathImpacts bpf/ or low-level forwarding details, including map management and monitor messages.area/loaderImpacts the loading of BPF programs into the kernel.ci/flakeThis is a known failure that occurs in the tree. Please investigate me!

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions