Skip to content

roachtest: cdc/crdb-chaos failed #68047

@cockroach-teamcity

Description

@cockroach-teamcity

roachtest.cdc/crdb-chaos failed with artifacts on release-21.1 @ b69ba4f24cbd604cc01fd471aed7f7262ece781d:

		  |  1268.0s     5101           17.0           20.3     22.0  10200.5  10200.5  10200.5 newOrder
		  |  1268.0s     5101            2.0            2.0      3.9      5.5      5.5      5.5 orderStatus
		  |  1268.0s     5101           16.0           19.7     11.0     14.2     18.9     18.9 payment
		  |  1268.0s     5101            5.0            2.0   1040.2  11811.2  11811.2  11811.2 stockLevel
		  | _elapsed___errors__ops/sec(inst)___ops/sec(cum)__p50(ms)__p95(ms)__p99(ms)_pMax(ms)
		  |  1269.0s     5125            1.0            1.9     41.9     41.9     41.9     41.9 delivery
		  |  1269.0s     5125           24.0           20.3     21.0   4295.0  11274.3  11274.3 newOrder
		  |  1269.0s     5125            2.0            2.0      3.8      5.8      5.8      5.8 orderStatus
		  |  1269.0s     5125           17.0           19.7     12.6     13.6     18.9     18.9 payment
		  |  1269.0s     5125            2.0            2.0     18.9   3623.9   3623.9   3623.9 stockLevel
		Wraps: (4) secondary error attachment
		  | signal: killed
		  | (1) signal: killed
		  | Error types: (1) *exec.ExitError
		Wraps: (5) context canceled
		Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *secondary.withSecondaryError (5) *errors.errorString

	cluster.go:2688,cdc.go:218,cdc.go:645,test_runner.go:733: monitor failure: monitor task failed: unexpected status: failed
		(1) attached stack trace
		  -- stack trace:
		  | main.(*monitor).WaitE
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2676
		  | main.(*monitor).Wait
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2684
		  | main.cdcBasicTest
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cdc.go:218
		  | main.registerCDC.func4
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cdc.go:645
		  | main.(*testRunner).runTest.func2
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_runner.go:733
		Wraps: (2) monitor failure
		Wraps: (3) attached stack trace
		  -- stack trace:
		  | main.(*monitor).wait.func2
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2732
		Wraps: (4) monitor task failed
		Wraps: (5) attached stack trace
		  -- stack trace:
		  | main.(*latencyVerifier).pollLatency
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cdc.go:1461
		  | main.cdcBasicTest.func3
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cdc.go:196
		  | main.(*monitor).Go.func1
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2666
		  | golang.org/x/sync/errgroup.(*Group).Go.func1
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:57
		  | runtime.goexit
		  | 	/usr/local/go/src/runtime/asm_amd64.s:1371
		Wraps: (6) unexpected status: failed
		Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) *withstack.withStack (6) *errutil.leafError
Reproduce

To reproduce, try:

# From https://go.crdb.dev/p/roachstress, perhaps edited lightly.
caffeinate ./roachstress.sh cdc/crdb-chaos

Same failure on other branches

/cc @cockroachdb/cdc

This test on roachdash | Improve this report!

Epic CRDB-9179

Jira issue: CRDB-8846

Metadata

Metadata

Assignees

Labels

C-test-failureBroken test (automatically or manually discovered).O-roachtestO-robotOriginated from a bot.T-cdcbranch-release-21.1Used to mark GA and release blockers, technical advisories, and bugs for 21.1

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions