-
Notifications
You must be signed in to change notification settings - Fork 4.1k
roachtest: tpce/c=5000/nodes=3 failed #66843
Copy link
Copy link
Closed
Labels
C-test-failureBroken test (automatically or manually discovered).Broken test (automatically or manually discovered).O-roachtestO-robotOriginated from a bot.Originated from a bot.release-blockerIndicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked.Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked.
Milestone
Description
roachtest.tpce/c=5000/nodes=3 failed with artifacts on release-21.1 @ 92e342b8f0032a5771d3e50932df4c6d7d85fe41:
Reported tpsE : -- (not between 80% and 100%)
thread '<unnamed>' panicked at 'called ``Result::unwrap()`` on an ``Err`` value: "SendError(..)"', driver/src/market_exchange_emulator.rs:48:53
thread '<unnamed>' panicked at 'called ``Result::unwrap()`` on an ``Err`` value: "SendError(..)"', driver/src/customer_emulator.rs:37:53
cluster.go:2668,tpce.go:96,tpce.go:113,test_runner.go:767: monitor failure: unexpected node event: 1: dead
(1) attached stack trace
-- stack trace:
| main.(*monitor).WaitE
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2656
| main.(*monitor).Wait
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2664
| main.registerTPCE.func1
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tpce.go:96
| main.registerTPCE.func2
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tpce.go:113
| main.(*testRunner).runTest.func2
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_runner.go:767
| runtime.goexit
| /usr/local/go/src/runtime/asm_amd64.s:1371
Wraps: (2) monitor failure
Wraps: (3) unexpected node event: 1: dead
Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *errors.errorString
cluster.go:1667,context.go:89,cluster.go:1656,test_runner.go:848: dead node detection: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod monitor teamcity-3113086-1624515120-79-n4cpu4 --oneshot --ignore-empty-nodes: exit status 1 4: skipped
1: dead
2: 9916
3: 10323
Error: UNCLASSIFIED_PROBLEM: 1: dead
(1) UNCLASSIFIED_PROBLEM
Wraps: (2) attached stack trace
-- stack trace:
| main.glob..func14
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/main.go:1147
| main.wrap.func1
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/main.go:271
| github.com/spf13/cobra.(*Command).execute
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra/command.go:830
| github.com/spf13/cobra.(*Command).ExecuteC
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra/command.go:914
| github.com/spf13/cobra.(*Command).Execute
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra/command.go:864
| main.main
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/main.go:1852
| runtime.main
| /usr/local/go/src/runtime/proc.go:225
| runtime.goexit
| /usr/local/go/src/runtime/asm_amd64.s:1371
Wraps: (3) 1: dead
Error types: (1) errors.Unclassified (2) *withstack.withStack (3) *errutil.leafError
Reproduce
To reproduce, try:
# From https://go.crdb.dev/p/roachstress, perhaps edited lightly.
caffeinate ./roachstress.sh tpce/c=5000/nodes=3Same failure on other branches
- roachtest: tpce/c=5000/nodes=3 failed #66719 roachtest: tpce/c=5000/nodes=3 failed [C-bug C-test-failure O-roachtest O-robot branch-master release-blocker]
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
C-test-failureBroken test (automatically or manually discovered).Broken test (automatically or manually discovered).O-roachtestO-robotOriginated from a bot.Originated from a bot.release-blockerIndicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked.Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked.