-
Notifications
You must be signed in to change notification settings - Fork 4.1k
roachtest: decommission/nodes=4/duration=1h0m0s failed #47738
Copy link
Copy link
Closed
Labels
C-test-failureBroken test (automatically or manually discovered).Broken test (automatically or manually discovered).O-roachtestO-robotOriginated from a bot.Originated from a bot.release-blockerIndicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked.Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked.
Milestone
Description
(roachtest).decommission/nodes=4/duration=1h0m0s failed on release-19.2@3b556da231bb5ac15ac6ff678d61cb1ba516e6f7:
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/decommission.go:205
| github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup.(*Group).Go.func1
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:57
| runtime.goexit
| /usr/local/go/src/runtime/asm_amd64.s:1357
Wraps: (2) 2 safe details enclosed
Wraps: (3) output in run_062023.051_n1_cockroach_quit
Wraps: (4) /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod run teamcity-1888224-1587449532-09-n4cpu4:1 -- ./cockroach quit --insecure --host=:{pgport:1} returned
| stderr:
| node is draining... remaining: 4
| node is draining... remaining: 0 (complete)
| Error: hard shutdown failed: rpc error: code = Unimplemented desc = grpc: Decompressor is not installed for grpc-encoding "snappy"
| Failed running "quit"
| Error: DEAD_ROACH_PROBLEM: exit status 1
| (1) DEAD_ROACH_PROBLEM
| Wraps: (2) Node 1. Command with error:
| | ```
| | ./cockroach quit --insecure --host=:{pgport:1}
| | ```
| Wraps: (3) exit status 1
| Error types: (1) errors.Cockroach (2) *hintdetail.withDetail (3) *exec.ExitError
|
| stdout:
Wraps: (5) exit status 30
Error types: (1) *withstack.withStack (2) *safedetails.withSafeDetails (3) *errutil.withMessage (4) *main.withCommandDetails (5) *exec.ExitError
cluster.go:1481,context.go:135,cluster.go:1470,test_runner.go:825: dead node detection: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod monitor teamcity-1888224-1587449532-09-n4cpu4 --oneshot --ignore-empty-nodes: exit status 1 1: dead
4: 4133
3: 4126
2: 4005
Error: UNCLASSIFIED_PROBLEM: 1: dead
(1) UNCLASSIFIED_PROBLEM
Wraps: (2) 1: dead
| main.glob..func13
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/main.go:1129
| main.wrap.func1
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/main.go:272
| github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra.(*Command).execute
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra/command.go:766
| github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra.(*Command).ExecuteC
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra/command.go:852
| github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra.(*Command).Execute
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra/command.go:800
| main.main
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/main.go:1793
| runtime.main
| /usr/local/go/src/runtime/proc.go:203
| runtime.goexit
| /usr/local/go/src/runtime/asm_amd64.s:1357
Error types: (1) errors.Unclassified (2) *errors.fundamental
More
Artifacts: /decommission/nodes=4/duration=1h0m0s
Related:
- roachtest: decommission/nodes=4/duration=1h0m0s failed #47656 roachtest: decommission/nodes=4/duration=1h0m0s failed C-test-failure O-roachtest O-robot branch-master release-blocker
See this test on roachdash
powered by pkg/cmd/internal/issues
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
C-test-failureBroken test (automatically or manually discovered).Broken test (automatically or manually discovered).O-roachtestO-robotOriginated from a bot.Originated from a bot.release-blockerIndicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked.Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked.