-
Notifications
You must be signed in to change notification settings - Fork 4.1k
storage: TestStoreGossipSystemData failed under stress #27908
Copy link
Copy link
Closed
Labels
A-kv-clientRelating to the KV client and the KV interface.Relating to the KV client and the KV interface.C-test-failureBroken test (automatically or manually discovered).Broken test (automatically or manually discovered).O-robotOriginated from a bot.Originated from a bot.
Milestone
Description
SHA: https://github.com/cockroachdb/cockroach/commits/62f3b753d49758b22203c558b8682b394773f732
Parameters:
TAGS=
GOFLAGS=-race
Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=795482&tab=buildLog
=== RUN TestStoreGossipSystemData
I180725 05:46:18.400551 48498 gossip/gossip.go:363 [n1] NodeDescriptor set to node_id:1 address:<network_field:"tcp" address_field:"127.0.0.1:36329" > attrs:<> locality:<> ServerVersion:<major_val:0 minor_val:0 patch:0 unstable:0 >
I180725 05:46:18.416675 48498 storage/replica_command.go:279 [s1,r1/1:/M{in-ax}] initiating a split of this range at key /Table/SystemConfigSpan/Start [r2]
E180725 05:46:18.432138 48413 storage/queue.go:778 [replicate,s1,r1/1:/{Min-Table/System…}] range requires a replication change, but lacks a quorum of live replicas (0/1)
E180725 05:46:18.432829 48414 storage/queue.go:778 [replicate,s1,r2/1:/{Table/System…-Max}] range requires a replication change, but lacks a quorum of live replicas (0/1)
E180725 05:46:18.462170 48945 config/system.go:325 unable to determine largest reserved object ID from system config: descriptor table not found in system config of 0 values
I180725 05:46:18.474621 48879 storage/replica_proposal.go:222 [s1,r2/1:/{Table/System…-Max}] new range lease repl=(n1,s1):1 seq=2 start=0.000000123,101 epo=1 pro=0.000000123,123 following repl=(n1,s1):1 seq=1 start=0.000000000,0 exp=0.900000123,4 pro=0.000000123,5
==================
WARNING: DATA RACE
Write at 0x00c420d93dd0 by goroutine 170:
runtime.mapassign_fast64()
/usr/local/go/src/runtime/hashmap_fast.go:522 +0x0
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).withCancel()
/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:242 +0x166
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).WithCancelOnStop()
/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:230 +0x71
github.com/cockroachdb/cockroach/pkg/storage.(*NodeLiveness).StartHeartbeat.func1()
/go/src/github.com/cockroachdb/cockroach/pkg/storage/node_liveness.go:459 +0xfa
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1()
/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:199 +0x14a
Previous read at 0x00c420d93dd0 by goroutine 184:
runtime.mapiterinit()
/usr/local/go/src/runtime/hashmap.go:691 +0x0
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).Stop()
/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:464 +0x2d6
github.com/cockroachdb/cockroach/pkg/storage_test.(*multiTestContext).Stop.func1()
/go/src/github.com/cockroachdb/cockroach/pkg/storage/client_test.go:341 +0x2c4
Goroutine 170 (running) created at:
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker()
/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:192 +0xba
github.com/cockroachdb/cockroach/pkg/storage.(*NodeLiveness).StartHeartbeat()
/go/src/github.com/cockroachdb/cockroach/pkg/storage/node_liveness.go:456 +0x27a
github.com/cockroachdb/cockroach/pkg/storage_test.(*multiTestContext).restartStoreWithoutHeartbeat()
/go/src/github.com/cockroachdb/cockroach/pkg/storage/client_test.go:929 +0xc98
github.com/cockroachdb/cockroach/pkg/storage_test.(*multiTestContext).restartStore()
/go/src/github.com/cockroachdb/cockroach/pkg/storage/client_test.go:939 +0x46
github.com/cockroachdb/cockroach/pkg/storage_test.TestStoreGossipSystemData()
/go/src/github.com/cockroachdb/cockroach/pkg/storage/client_lease_test.go:222 +0xaa7
testing.tRunner()
/usr/local/go/src/testing/testing.go:777 +0x16d
Goroutine 184 (running) created at:
github.com/cockroachdb/cockroach/pkg/storage_test.(*multiTestContext).Stop()
/go/src/github.com/cockroachdb/cockroach/pkg/storage/client_test.go:312 +0x8c
github.com/cockroachdb/cockroach/pkg/storage_test.TestStoreGossipSystemData()
/go/src/github.com/cockroachdb/cockroach/pkg/storage/client_lease_test.go:232 +0xb4b
testing.tRunner()
/usr/local/go/src/testing/testing.go:777 +0x16d
==================
--- FAIL: TestStoreGossipSystemData (0.15s)
testing.go:730: race detected during execution of test
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
A-kv-clientRelating to the KV client and the KV interface.Relating to the KV client and the KV interface.C-test-failureBroken test (automatically or manually discovered).Broken test (automatically or manually discovered).O-robotOriginated from a bot.Originated from a bot.