Skip to content

macOS CI Network timeouts "transport: Error while dialing: dial tcp $IP: i/o timeout" #155110

@gaaclarke

Description

@gaaclarke

Update by @christopherfujino:

LUCI's data centers are having an incident, this is being tracked internally.


Known instances

BuildBucket link Bot Zone
https://ci.chromium.org/ui/p/flutter/builders/try/Mac%20Engine%20Drone/1152399/infra https://chromium-swarm.appspot.com/bot?id=mac-49-h526 ZONE_IAD97
https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20Production%20Engine%20Drone/455899/infra https://chromium-swarm.appspot.com/bot?id=mac-718-h526 ZONE_IAD97
https://ci.chromium.org/ui/p/flutter/builders/try/Mac%20Engine%20Drone/1155976/infra https://chromium-swarm.appspot.com/bot?id=mac-781-h526 ZONE_IAD97
https://ci.chromium.org/ui/p/flutter/builders/try/Mac%20Engine%20Drone/1156363/infra https://chromium-swarm.appspot.com/bot?id=mac-47-h526 ZONE_IAD97
https://ci.chromium.org/ui/p/flutter/builders/try/Mac%20customer_testing/83275/infra https://chromium-swarm.appspot.com/bot?id=mac-756-h526 ZONE_IAD97
https://ci.chromium.org/ui/p/flutter/builders/prod/Mac_arm64%20tool_host_cross_arch_tests/9934/infra https://chromium-swarm.appspot.com/bot?id=mac-704-h526 ZONE_IAD97

error

[E2024-09-12T10:41:38.940154-07:00 768 0 client.go:170] failed to create casclient: rpc error: code = Unavailable desc = rpc error: code = Unavailable desc = retry budget exhausted (6 attempts): connection error: desc = "transport: Error while dialing: dial tcp 172.253.115.95:443: i/o timeout", authentication type (identity) used="external authentication token (credentials helper)"
[E2024-09-12T10:41:38.940612-07:00 768 0 annotate.go:273] original error: rpc error: code = Unavailable desc = rpc error: code = Unavailable desc = retry budget exhausted (6 attempts): connection error: desc = "transport: Error while dialing: dial tcp 172.253.115.95:443: i/o timeout", authentication type (identity) used="external authentication token (credentials helper)"

goroutine 1:
#0 go.chromium.org/luci/client/casclient/client.go:171 - casclient.NewLegacy()
  reason: failed to create client

#1 cas/main.go:64 - main.(*authFlags).NewRBEClient()
#2 go.chromium.org/luci/client/cmd/cas/casimpl/download.go:338 - casimpl.(*downloadRun).doDownload()
#3 go.chromium.org/luci/client/cmd/cas/casimpl/download.go:574 - casimpl.(*downloadRun).Run()
#4 github.com/maruel/subcommands@v1.1.1/subcommands.go:395 - subcommands.Run()
#5 cas/main.go:95 - main.main()
#6 runtime/proc.go:271 - runtime.main()
#7 runtime/asm_amd64.s:1695 - runtime.goexit()
cas: failed to create client: rpc error: code = Unavailable desc = rpc error: code = Unavailable desc = retry budget exhausted (6 attempts): connection error: desc = "transport: Error while dialing: dial tcp 172.253.115.95:443: i/o timeout", authentication type (identity) used="external authentication token (credentials helper)"

Metadata

Metadata

Labels

P0Critical issues such as a build break or regressionc: flakeTests that sometimes, but not always, incorrectly passteam-infraOwned by Infrastructure team

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions