Skip to content

teamcity: failed test: TestRestoreReplicas [skipped] #40351

@cockroach-teamcity

Description

@cockroach-teamcity

The following tests appear to have failed on master (testrace): TestRestoreReplicas

You may want to check for open issues.

#1461211:

TestRestoreReplicas
....go:542  quiescing; tasks left:
7      mtc send
I190829 21:57:09.860764 105833 internal/client/txn.go:634  async rollback failed: failed to send RPC: sending to all 2 replicas failed; last error: <nil> failed to send RPC: store is stopped
I190829 21:57:09.867459 106600 util/stop/stopper.go:542  quiescing; tasks left:
6      mtc send
I190829 21:57:09.870267 105918 internal/client/txn.go:634  async rollback failed: failed to send RPC: sending to all 2 replicas failed; last error: <nil> failed to send RPC: store is stopped
I190829 21:57:09.870672 106081 internal/client/txn.go:634  async rollback failed: failed to send RPC: sending to all 2 replicas failed; last error: <nil> failed to send RPC: store is stopped
I190829 21:57:09.870781 106600 util/stop/stopper.go:542  quiescing; tasks left:
2      mtc send
I190829 21:57:09.871656 105718 storage/client_test.go:1359  [txn=65f1a4cb] test clock advanced to: 169.200000311,0
I190829 21:57:09.871806 106432 internal/client/txn.go:634  async rollback failed: failed to send RPC: sending to all 2 replicas failed; last error: <nil> failed to send RPC: store is stopped
W190829 21:57:09.873172 93401 internal/client/txn.go:524  [liveness-hb] failure aborting transaction: node unavailable; try another peer; abort caused by: result is ambiguous (error=failed to send RPC: store is stopped [exhausted])
I190829 21:57:09.873478 93401 storage/node_liveness.go:836  [liveness-hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (error=failed to send RPC: store is stopped [exhausted])
I190829 21:57:09.873719 105718 internal/client/txn.go:634  async rollback failed: failed to send RPC: sending to all 2 replicas failed; last error: <nil> failed to send RPC: store is stopped
I190829 21:57:09.874511 106273 internal/client/txn.go:634  async rollback failed: failed to send RPC: sending to all 2 replicas failed; last error: <nil> failed to send RPC: store is stopped
W190829 21:57:09.875827 93401 internal/client/txn.go:524  [liveness-hb] failure aborting transaction: node unavailable; try another peer; abort caused by: node unavailable; try another peer
W190829 21:57:09.891890 93401 storage/node_liveness.go:484  [liveness-hb] failed node liveness heartbeat: node unavailable; try another peer
I190829 21:57:09.908716 106599 util/stop/stopper.go:542  quiescing; tasks left:
4      rpc heartbeat
I190829 21:57:09.910447 106599 util/stop/stopper.go:542  quiescing; tasks left:
3      rpc heartbeat
W190829 21:57:09.911913 93150 storage/raft_transport.go:620  while processing outgoing Raft queue to node 1: EOF:
W190829 21:57:09.915158 93320 storage/raft_transport.go:620  while processing outgoing Raft queue to node 2: rpc error: code = Canceled desc = grpc: the client connection is closing:
    soon.go:35: condition failed to evaluate within 45s: node not live
        goroutine 92967 [running]:
        runtime/debug.Stack(0x62ec8a0, 0xc0034bee00, 0xc005745be0)
        	/usr/local/go/src/runtime/debug/stack.go:24 +0xab
        github.com/cockroachdb/cockroach/pkg/testutils.SucceedsSoon(0x62ec8a0, 0xc0034bee00, 0xc005745be0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/testutils/soon.go:36 +0x87
        github.com/cockroachdb/cockroach/pkg/storage_test.(*multiTestContext).restartStore(0xc000be6000, 0x0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/storage/client_test.go:1055 +0x149
        github.com/cockroachdb/cockroach/pkg/storage_test.(*multiTestContext).restart(0xc000be6000)
        	/go/src/github.com/cockroachdb/cockroach/pkg/storage/client_test.go:1108 +0xc1
        github.com/cockroachdb/cockroach/pkg/storage_test.TestRestoreReplicas(0xc0034bee00)
        	/go/src/github.com/cockroachdb/cockroach/pkg/storage/client_raft_test.go:373 +0x7ea
        testing.tRunner(0xc0034bee00, 0x53922f0)
        	/usr/local/go/src/testing/testing.go:865 +0x164
        created by testing.(*T).Run
        	/usr/local/go/src/testing/testing.go:916 +0x65b




Please assign, take a look and update the issue accordingly.

Metadata

Metadata

Assignees

Labels

C-test-failureBroken test (automatically or manually discovered).O-robotOriginated from a bot.branch-masterFailures and bugs on the master branch.

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions