Skip to content

roachtest: cdc/crdb-chaos failed #67973

@cockroach-teamcity

Description

@cockroach-teamcity

roachtest.cdc/crdb-chaos failed with artifacts on master @ 5e46fd88b11007ddaf0b5350ed28d11b0c3bfdaf:

		  |   135.0s      376            2.0            2.1     41.9     41.9     41.9     41.9 delivery
		  |   135.0s      376           11.0           20.0     19.9     28.3     35.7     35.7 newOrder
		  |   135.0s      376            1.0            2.1      2.9      2.9      2.9      2.9 orderStatus
		  |   135.0s      376           22.0           21.2     10.5     13.6     15.2     15.2 payment
		  |   135.0s      376            4.0            1.9      8.4     13.6     13.6     13.6 stockLevel
		  |   136.0s      396            0.0            2.1      0.0      0.0      0.0      0.0 delivery
		  |   136.0s      396           13.0           20.0     21.0     23.1     27.3     27.3 newOrder
		  |   136.0s      396            2.0            2.1      3.8      4.5      4.5      4.5 orderStatus
		  |   136.0s      396           18.0           21.2     11.0     17.8     18.9     18.9 payment
		  |   136.0s      396            0.0            1.9      0.0      0.0      0.0      0.0 stockLevel
		Wraps: (4) secondary error attachment
		  | signal: killed
		  | (1) signal: killed
		  | Error types: (1) *exec.ExitError
		Wraps: (5) context canceled
		Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *secondary.withSecondaryError (5) *errors.errorString

	monitor.go:128,cdc.go:271,cdc.go:722,test_runner.go:765: monitor failure: monitor task failed: unexpected status: failed
		(1) attached stack trace
		  -- stack trace:
		  | main.(*monitorImpl).WaitE
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/monitor.go:116
		  | main.(*monitorImpl).Wait
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/monitor.go:124
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.cdcBasicTest
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/cdc.go:271
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.registerCDC.func5
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/cdc.go:722
		  | main.(*testRunner).runTest.func2
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_runner.go:765
		Wraps: (2) monitor failure
		Wraps: (3) attached stack trace
		  -- stack trace:
		  | main.(*monitorImpl).wait.func2
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/monitor.go:172
		Wraps: (4) monitor task failed
		Wraps: (5) attached stack trace
		  -- stack trace:
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*latencyVerifier).pollLatency
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/cdc.go:1601
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.cdcBasicTest.func3
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/cdc.go:249
		  | main.(*monitorImpl).Go.func1
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/monitor.go:106
		  | golang.org/x/sync/errgroup.(*Group).Go.func1
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:57
		  | runtime.goexit
		  | 	/usr/local/go/src/runtime/asm_amd64.s:1371
		Wraps: (6) unexpected status: failed
		Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) *withstack.withStack (6) *errutil.leafError
Reproduce

To reproduce, try:

## Simple repro (linux-only):
  $ make cockroachshort bin/worklaod bin/roachprod bin/roachtest
  $ PATH=$PWD/bin:$PATH roachtest run cdc/crdb-chaos --local

## Proper repro probably needs more roachtest flags, or running
## the programs remotely on GCE. For more details, refer to
## pkg/cmd/roachtest/README.md.

/cc @cockroachdb/cdc

This test on roachdash | Improve this report!

Metadata

Metadata

Assignees

Labels

C-test-failureBroken test (automatically or manually discovered).O-roachtestO-robotOriginated from a bot.T-cdcbranch-masterFailures and bugs on the master branch.release-blockerIndicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions