Skip to content

roachtest: scrub/all-checks/tpcc/w=100 failed #37108

@cockroach-teamcity

Description

@cockroach-teamcity

SHA: https://github.com/cockroachdb/cockroach/commits/7109d291e3b9edfa38264361f832cec14fff66ee

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=scrub/all-checks/tpcc/w=100 PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1259219&tab=buildLog

The test failed on release-19.1:
	scrub.go:83,cluster.go:1761,errgroup.go:57: dial tcp 35.188.39.106:26257: connect: connection refused
	cluster.go:1423,tpcc.go:168,cluster.go:1761,errgroup.go:57: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod run teamcity-1259219-scrub-all-checks-tpcc-w-100:5 -- ./workload run tpcc --warehouses=100 --histograms=logs/stats.json --wait=false --tolerate-errors --ramp=5m0s --duration=30m0s {pgurl:1-4} returned:
		stderr:
		
		stdout:
		l
		_elapsed___errors__ops/sec(inst)___ops/sec(cum)__p50(ms)__p95(ms)__p99(ms)_pMax(ms)
		  10m49s    26544            0.0            0.1      0.0      0.0      0.0      0.0 delivery
		  10m49s    26544            0.0            0.5      0.0      0.0      0.0      0.0 newOrder
		  10m49s    26544            0.0            0.1      0.0      0.0      0.0      0.0 orderStatus
		  10m49s    26544            0.0            0.3      0.0      0.0      0.0      0.0 payment
		  10m49s    26544            0.0            0.0      0.0      0.0      0.0      0.0 stockLevel
		  10m50s    26544            0.0            0.1      0.0      0.0      0.0      0.0 delivery
		  10m50s    26544            0.0            0.5      0.0      0.0      0.0      0.0 newOrder
		  10m50s    26544            0.0            0.1      0.0      0.0      0.0      0.0 orderStatus
		  10m50s    26544            0.0            0.3      0.0      0.0      0.0      0.0 payment
		  10m50s    26544            0.0            0.0      0.0      0.0      0.0      0.0 stockLevel
		: signal: killed
	cluster.go:1782,tpcc.go:178,scrub.go:58,test.go:1245: Goexit() was called
	cluster.go:1016,context.go:89,cluster.go:1005,asm_amd64.s:522,panic.go:397,test.go:790,test.go:776,cluster.go:1782,tpcc.go:178,scrub.go:58,test.go:1245: dead node detection: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod monitor teamcity-1259219-scrub-all-checks-tpcc-w-100 --oneshot --ignore-empty-nodes: exit status 1 5: skipped
		1: dead
		3: 3213
		2: 3189
		4: 3156
		Error:  1: dead

Metadata

Metadata

Assignees

Labels

C-test-failureBroken test (automatically or manually discovered).O-roachtestO-robotOriginated from a bot.

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions