roachtest: kv/gracefuldraining/nodes=3 failed [skipped] #33501
Description
SHA: https://github.com/cockroachdb/cockroach/commits/d6adf24cae788d7cd967feadae8e9c0388ce5273
Parameters:
To repro, try:
# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=kv/gracefuldraining/nodes=3 PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log
Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1079910&tab=buildLog
The test failed on master:
test.go:701,kv.go:267: QPS of 337.90 at time 2019-01-04 10:08:30 +0000 UTC is below minimum allowable QPS of 900.00; entire timeseries: [{TimestampNanos:1546596500000000000 Value:1 XXX_NoUnkeyedLiteral:{} XXX_sizecache:0} {TimestampNanos:1546596510000000000 Value:337.9 XXX_NoUnkeyedLiteral:{} XXX_sizecache:0} {TimestampNanos:1546596520000000000 Value:999.2 XXX_NoUnkeyedLiteral:{} XXX_sizecache:0} {TimestampNanos:1546596530000000000 Value:999.4000000000001 XXX_NoUnkeyedLiteral:{} XXX_sizecache:0} {TimestampNanos:1546596540000000000 Value:998.1000000000001 XXX_NoUnkeyedLiteral:{} XXX_sizecache:0} {TimestampNanos:1546596550000000000 Value:999.6000000000001 XXX_NoUnkeyedLiteral:{} XXX_sizecache:0} {TimestampNanos:1546596560000000000 Value:999.5 XXX_NoUnkeyedLiteral:{} XXX_sizecache:0} {TimestampNanos:1546596570000000000 Value:995.8000000000001 XXX_NoUnkeyedLiteral:{} XXX_sizecache:0} {TimestampNanos:1546596580000000000 Value:993.8 XXX_NoUnkeyedLiteral:{} XXX_sizecache:0} {TimestampNanos:1546596590000000000 Value:994.8 XXX_NoUnkeyedLiteral:{} XXX_sizecache:0} {TimestampNanos:1546596600000000000 Value:996 XXX_NoUnkeyedLiteral:{} XXX_sizecache:0} {TimestampNanos:1546596610000000000 Value:997.7 XXX_NoUnkeyedLiteral:{} XXX_sizecache:0} {TimestampNanos:1546596620000000000 Value:996.9000000000001 XXX_NoUnkeyedLiteral:{} XXX_sizecache:0} {TimestampNanos:1546596630000000000 Value:994.8000000000001 XXX_NoUnkeyedLiteral:{} XXX_sizecache:0} {TimestampNanos:1546596640000000000 Value:998.9000000000001 XXX_NoUnkeyedLiteral:{} XXX_sizecache:0} {TimestampNanos:1546596650000000000 Value:998.7 XXX_NoUnkeyedLiteral:{} XXX_sizecache:0} {TimestampNanos:1546596660000000000 Value:999.3 XXX_NoUnkeyedLiteral:{} XXX_sizecache:0} {TimestampNanos:1546596670000000000 Value:998.7 XXX_NoUnkeyedLiteral:{} XXX_sizecache:0} {TimestampNanos:1546596680000000000 Value:1000.2 XXX_NoUnkeyedLiteral:{} XXX_sizecache:0} {TimestampNanos:1546596690000000000 Value:999.8 XXX_NoUnkeyedLiteral:{} XXX_sizecache:0} {TimestampNanos:1546596700000000000 Value:997.5 XXX_NoUnkeyedLiteral:{} XXX_sizecache:0} {TimestampNanos:1546596710000000000 Value:995.6000000000001 XXX_NoUnkeyedLiteral:{} XXX_sizecache:0} {TimestampNanos:1546596720000000000 Value:998.1 XXX_NoUnkeyedLiteral:{} XXX_sizecache:0} {TimestampNanos:1546596730000000000 Value:999.6 XXX_NoUnkeyedLiteral:{} XXX_sizecache:0} {TimestampNanos:1546596740000000000 Value:997.9000000000001 XXX_NoUnkeyedLiteral:{} XXX_sizecache:0} {TimestampNanos:1546596750000000000 Value:997.6 XXX_NoUnkeyedLiteral:{} XXX_sizecache:0} {TimestampNanos:1546596760000000000 Value:997.7 XXX_NoUnkeyedLiteral:{} XXX_sizecache:0}]