-
Notifications
You must be signed in to change notification settings - Fork 4.1k
roachtest: kv/gracefuldraining/nodes=3 failed [qps dropped] #59094
Copy link
Copy link
Closed
Labels
C-test-failureBroken test (automatically or manually discovered).Broken test (automatically or manually discovered).O-roachtestO-robotOriginated from a bot.Originated from a bot.S-1High impact: many users impacted, serious risk of high unavailability or data lossHigh impact: many users impacted, serious risk of high unavailability or data lossT-kvKV TeamKV TeamX-stalebranch-masterFailures and bugs on the master branch.Failures and bugs on the master branch.skipped-test
Description
(roachtest).kv/gracefuldraining/nodes=3 failed on master@7b0ccdda99b81613e70f421c9374483c3feddff3:
The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/kv/gracefuldraining/nodes=3/run_1
kv.go:526,cluster.go:2663,errgroup.go:57: QPS of 884.50 at time 2021-01-17 07:08:50 +0000 UTC is below minimum allowable QPS of 900.00; entire timeseries: [{TimestampNanos:1610867280000000000 Value:998.7} {TimestampNanos:1610867290000000000 Value:969.3} {TimestampNanos:1610867300000000000 Value:969.2} {TimestampNanos:1610867310000000000 Value:988.2} {TimestampNanos:1610867320000000000 Value:932.7} {TimestampNanos:1610867330000000000 Value:884.5} {TimestampNanos:1610867340000000000 Value:943.9000000000001} {TimestampNanos:1610867350000000000 Value:975.5} {TimestampNanos:1610867360000000000 Value:996.4000000000001} {TimestampNanos:1610867370000000000 Value:997.9000000000001} {TimestampNanos:1610867380000000000 Value:999.5} {TimestampNanos:1610867390000000000 Value:999.3} {TimestampNanos:1610867400000000000 Value:999.9000000000001} {TimestampNanos:1610867410000000000 Value:993.8} {TimestampNanos:1610867420000000000 Value:991.7} {TimestampNanos:1610867430000000000 Value:994.2} {TimestampNanos:1610867440000000000 Value:995.7} {TimestampNanos:1610867450000000000 Value:997.8} {TimestampNanos:1610867460000000000 Value:997.7}]
cluster.go:2685,kv.go:536,test_runner.go:760: monitor failure: monitor task failed: t.Fatal() was called
(1) attached stack trace
-- stack trace:
| main.(*monitor).WaitE
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2673
| main.(*monitor).Wait
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2681
| main.registerKVGracefulDraining.func1
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/kv.go:536
| main.(*testRunner).runTest.func2
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_runner.go:760
Wraps: (2) monitor failure
Wraps: (3) attached stack trace
-- stack trace:
| main.(*monitor).wait.func2
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2729
Wraps: (4) monitor task failed
Wraps: (5) attached stack trace
-- stack trace:
| main.init
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2643
| runtime.doInit
| /usr/local/go/src/runtime/proc.go:5652
| runtime.main
| /usr/local/go/src/runtime/proc.go:191
| runtime.goexit
| /usr/local/go/src/runtime/asm_amd64.s:1374
Wraps: (6) t.Fatal() was called
Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) *withstack.withStack (6) *errutil.leafError
More
Artifacts: /kv/gracefuldraining/nodes=3
See this test on roachdash
powered by pkg/cmd/internal/issues
Jira issue: CRDB-3333
Epic CRDB-18656
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
C-test-failureBroken test (automatically or manually discovered).Broken test (automatically or manually discovered).O-roachtestO-robotOriginated from a bot.Originated from a bot.S-1High impact: many users impacted, serious risk of high unavailability or data lossHigh impact: many users impacted, serious risk of high unavailability or data lossT-kvKV TeamKV TeamX-stalebranch-masterFailures and bugs on the master branch.Failures and bugs on the master branch.skipped-test