Skip to content

roachtest: kv/gracefuldraining/nodes=3 failed [qps dropped] #59094

@cockroach-teamcity

Description

@cockroach-teamcity

(roachtest).kv/gracefuldraining/nodes=3 failed on master@7b0ccdda99b81613e70f421c9374483c3feddff3:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/kv/gracefuldraining/nodes=3/run_1
	kv.go:526,cluster.go:2663,errgroup.go:57: QPS of 884.50 at time 2021-01-17 07:08:50 +0000 UTC is below minimum allowable QPS of 900.00; entire timeseries: [{TimestampNanos:1610867280000000000 Value:998.7} {TimestampNanos:1610867290000000000 Value:969.3} {TimestampNanos:1610867300000000000 Value:969.2} {TimestampNanos:1610867310000000000 Value:988.2} {TimestampNanos:1610867320000000000 Value:932.7} {TimestampNanos:1610867330000000000 Value:884.5} {TimestampNanos:1610867340000000000 Value:943.9000000000001} {TimestampNanos:1610867350000000000 Value:975.5} {TimestampNanos:1610867360000000000 Value:996.4000000000001} {TimestampNanos:1610867370000000000 Value:997.9000000000001} {TimestampNanos:1610867380000000000 Value:999.5} {TimestampNanos:1610867390000000000 Value:999.3} {TimestampNanos:1610867400000000000 Value:999.9000000000001} {TimestampNanos:1610867410000000000 Value:993.8} {TimestampNanos:1610867420000000000 Value:991.7} {TimestampNanos:1610867430000000000 Value:994.2} {TimestampNanos:1610867440000000000 Value:995.7} {TimestampNanos:1610867450000000000 Value:997.8} {TimestampNanos:1610867460000000000 Value:997.7}]

	cluster.go:2685,kv.go:536,test_runner.go:760: monitor failure: monitor task failed: t.Fatal() was called
		(1) attached stack trace
		  -- stack trace:
		  | main.(*monitor).WaitE
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2673
		  | main.(*monitor).Wait
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2681
		  | main.registerKVGracefulDraining.func1
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/kv.go:536
		  | main.(*testRunner).runTest.func2
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_runner.go:760
		Wraps: (2) monitor failure
		Wraps: (3) attached stack trace
		  -- stack trace:
		  | main.(*monitor).wait.func2
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2729
		Wraps: (4) monitor task failed
		Wraps: (5) attached stack trace
		  -- stack trace:
		  | main.init
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2643
		  | runtime.doInit
		  | 	/usr/local/go/src/runtime/proc.go:5652
		  | runtime.main
		  | 	/usr/local/go/src/runtime/proc.go:191
		  | runtime.goexit
		  | 	/usr/local/go/src/runtime/asm_amd64.s:1374
		Wraps: (6) t.Fatal() was called
		Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) *withstack.withStack (6) *errutil.leafError

More

Artifacts: /kv/gracefuldraining/nodes=3

See this test on roachdash
powered by pkg/cmd/internal/issues

Jira issue: CRDB-3333

Epic CRDB-18656

Metadata

Metadata

Assignees

Labels

C-test-failureBroken test (automatically or manually discovered).O-roachtestO-robotOriginated from a bot.S-1High impact: many users impacted, serious risk of high unavailability or data lossT-kvKV TeamX-stalebranch-masterFailures and bugs on the master branch.skipped-test

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions