-
Notifications
You must be signed in to change notification settings - Fork 4.1k
roachtest: scrub/index-only/tpcc/w=100 failed #37551
Copy link
Copy link
Closed
Labels
C-test-failureBroken test (automatically or manually discovered).Broken test (automatically or manually discovered).O-roachtestO-robotOriginated from a bot.Originated from a bot.
Milestone
Description
SHA: https://github.com/cockroachdb/cockroach/commits/c8bda1de440cfe90cf23a433119d77795cfa0047
Parameters:
To repro, try:
# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=scrub/index-only/tpcc/w=100 PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log
Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1292152&tab=buildLog
The test failed on branch=release-19.1, cloud=gce:
cluster.go:1474,tpcc.go:168,cluster.go:1812,errgroup.go:57: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod run teamcity-1292152-scrub-index-only-tpcc-w-100:5 -- ./workload run tpcc --warehouses=100 --histograms=logs/stats.json --wait=false --tolerate-errors --ramp=5m0s --duration=30m0s {pgurl:1-4} returned:
stderr:
stdout:
l
17m36s 13800 0.0 0.1 0.0 0.0 0.0 0.0 delivery
17m36s 13800 1.0 0.9 103079.2 103079.2 103079.2 103079.2 newOrder
17m36s 13800 0.0 0.1 0.0 0.0 0.0 0.0 orderStatus
17m36s 13800 2.0 0.8 15569.3 90194.3 90194.3 90194.3 payment
17m36s 13800 0.0 0.0 0.0 0.0 0.0 0.0 stockLevel
_elapsed___errors__ops/sec(inst)___ops/sec(cum)__p50(ms)__p95(ms)__p99(ms)_pMax(ms)
17m37s 13800 0.0 0.1 0.0 0.0 0.0 0.0 delivery
17m37s 13800 1.0 0.9 103079.2 103079.2 103079.2 103079.2 newOrder
17m37s 13800 1.0 0.1 9126.8 9126.8 9126.8 9126.8 orderStatus
17m37s 13800 0.0 0.8 0.0 0.0 0.0 0.0 payment
17m37s 13800 0.0 0.0 0.0 0.0 0.0 0.0 stockLevel
: signal: killed
cluster.go:1833,tpcc.go:178,scrub.go:58,test.go:1251: unexpected node event: 2: dead
cluster.go:1038,context.go:89,cluster.go:1027,asm_amd64.s:522,panic.go:397,test.go:788,test.go:774,cluster.go:1833,tpcc.go:178,scrub.go:58,test.go:1251: dead node detection: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod monitor teamcity-1292152-scrub-index-only-tpcc-w-100 --oneshot --ignore-empty-nodes: exit status 1 5: skipped
2: dead
4: 3818
1: 4338
3: 3892
Error: 2: dead
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
C-test-failureBroken test (automatically or manually discovered).Broken test (automatically or manually discovered).O-roachtestO-robotOriginated from a bot.Originated from a bot.