stability: repeatedly applying snapshot for the same range #8629
Description
Running block_writer against a local 4 node cluster, I see a snapshot being applied repeatedly for 1 range on one node:
~/Development/go/src/github.com/cockroachdb/cockroach pmattis/raft-logging-redux grep 'applied Raft snapshot for range 7 ' cockroach-data/3/logs/0.stderr | wc -l
73
The only replica changes for this range are:
~/Development/go/src/github.com/cockroachdb/cockroach pmattis/raft-logging-redux grep 'range=7' cockroach-data/1/logs/0.stderr | egrep 'ADD|REMOVE'
I160817 19:53:52.667525 storage/replica.go:1500 store=1:1 range=7 [/Table/51-/Table/51/1/2319145621337716466/"4c3241bc-22e1-4c42-8996-f40f3f3ef665"/10028): proposing ADD_REPLICA {NodeID:4 StoreID:4 ReplicaID:4} for range 7: [{NodeID:1 StoreID:1 ReplicaID:1} {NodeID:3 StoreID:2 ReplicaID:2} {NodeID:2 StoreID:3 ReplicaID:3} {NodeID:4 StoreID:4 ReplicaID:4}]
I160817 19:53:52.772124 storage/replica.go:1500 store=1:1 range=7 [/Table/51-/Table/51/1/2319145621337716466/"4c3241bc-22e1-4c42-8996-f40f3f3ef665"/10028): proposing REMOVE_REPLICA {NodeID:3 StoreID:2 ReplicaID:2} for range 7: [{NodeID:1 StoreID:1 ReplicaID:1} {NodeID:4 StoreID:4 ReplicaID:4} {NodeID:2 StoreID:3 ReplicaID:3}]
I160817 19:54:49.981856 storage/replica.go:1500 store=1:1 range=7 [/Table/51-/Table/51/1/2319145621337716466/"4c3241bc-22e1-4c42-8996-f40f3f3ef665"/10028): proposing ADD_REPLICA {NodeID:3 StoreID:2 ReplicaID:5} for range 7: [{NodeID:1 StoreID:1 ReplicaID:1} {NodeID:4 StoreID:4 ReplicaID:4} {NodeID:2 StoreID:3 ReplicaID:3} {NodeID:3 StoreID:2 ReplicaID:5}]
I160817 19:54:51.671439 storage/replica.go:1500 store=1:1 range=7 [/Table/51-/Table/51/1/2319145621337716466/"4c3241bc-22e1-4c42-8996-f40f3f3ef665"/10028): proposing REMOVE_REPLICA {NodeID:2 StoreID:3 ReplicaID:3} for range 7: [{NodeID:1 StoreID:1 ReplicaID:1} {NodeID:4 StoreID:4 ReplicaID:4} {NodeID:3 StoreID:2 ReplicaID:5}]
Metadata
Metadata
Assignees
Labels
No labels