-
Notifications
You must be signed in to change notification settings - Fork 4.1k
roachtest: schemachange/during/tpcc failed #69086
Copy link
Copy link
Closed
Labels
C-test-failureBroken test (automatically or manually discovered).Broken test (automatically or manually discovered).O-roachtestO-robotOriginated from a bot.Originated from a bot.T-sql-foundationsSQL Foundations Team (formerly SQL Schema + SQL Sessions)SQL Foundations Team (formerly SQL Schema + SQL Sessions)branch-masterFailures and bugs on the master branch.Failures and bugs on the master branch.release-blockerIndicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked.Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked.
Description
roachtest.schemachange/during/tpcc failed with artifacts on master @ 472008d077a4282b7a5b0557ee6b6b5c12f75586:
The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/schemachange/during/tpcc/run_1
monitor.go:128,tpcc.go:279,schemachange.go:429,test_runner.go:777: monitor failure: unexpected node event: 2: dead (exit status 137)
(1) attached stack trace
-- stack trace:
| main.(*monitorImpl).WaitE
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/monitor.go:116
| main.(*monitorImpl).Wait
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/monitor.go:124
| github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.runTPCC
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/tpcc.go:279
| github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.makeSchemaChangeDuringTPCC.func1
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/schemachange.go:429
| main.(*testRunner).runTest.func2
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_runner.go:777
| runtime.goexit
| /usr/local/go/src/runtime/asm_amd64.s:1371
Wraps: (2) monitor failure
Wraps: (3) unexpected node event: 2: dead (exit status 137)
Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *errors.errorString
cluster.go:1249,context.go:89,cluster.go:1237,test_runner.go:866: dead node detection: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod monitor teamcity-3317393-1629267491-66-n5cpu16 --oneshot --ignore-empty-nodes: exit status 1 5: skipped
2: dead (exit status 137)
4: 11681
1: 12120
3: 11716
Error: UNCLASSIFIED_PROBLEM: 2: dead (exit status 137)
(1) UNCLASSIFIED_PROBLEM
Wraps: (2) attached stack trace
-- stack trace:
| main.glob..func14
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/main.go:1173
| main.wrap.func1
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/main.go:281
| github.com/spf13/cobra.(*Command).execute
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra/command.go:856
| github.com/spf13/cobra.(*Command).ExecuteC
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra/command.go:960
| github.com/spf13/cobra.(*Command).Execute
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra/command.go:897
| main.main
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/main.go:2107
| runtime.main
| /usr/local/go/src/runtime/proc.go:225
| runtime.goexit
| /usr/local/go/src/runtime/asm_amd64.s:1371
Wraps: (3) 2: dead (exit status 137)
Error types: (1) errors.Unclassified (2) *withstack.withStack (3) *errutil.leafError
Reproduce
See: roachtest README
See: CI job to stress roachtests
For the CI stress job, click the ellipsis (...) next to the Run button and fill in: * Changes / Build branch: master * Parameters / `env.TESTS`: `^schemachange/during/tpcc$` * Parameters / `env.COUNT`: <number of runs>
Same failure on other branches
- roachtest: schemachange/during/tpcc failed #54316 roachtest: schemachange/during/tpcc failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-20.2]
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
C-test-failureBroken test (automatically or manually discovered).Broken test (automatically or manually discovered).O-roachtestO-robotOriginated from a bot.Originated from a bot.T-sql-foundationsSQL Foundations Team (formerly SQL Schema + SQL Sessions)SQL Foundations Team (formerly SQL Schema + SQL Sessions)branch-masterFailures and bugs on the master branch.Failures and bugs on the master branch.release-blockerIndicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked.Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked.