-
Notifications
You must be signed in to change notification settings - Fork 4.1k
kvserver: update AdminRelocateRange to use explicit swaps of voters and non-voters #62370
Copy link
Copy link
Closed
Labels
A-kv-replicationRelating to Raft, consensus, and coordination.Relating to Raft, consensus, and coordination.A-kv-replication-constraintsGA-blocker
Description
AdminRelocateRange currently doesn’t make use of the ability to promote non-voters to voters, demote voters to non-voters or swap voters and non-voters. Because of this limitation, the mergeQueue has to resort to a fairly complex way of determining which targets to relocate a range’s replicas to during a merge in order to avoid redundant data movement.
There are a few problems with this:
- With the ability to swap replica types, the complexity mentioned above is unnecessary. We can avoid redundant data movement by performing all the promotions / demotions / swaps and then relocating the rest of the misaligned replicas.
- The merge queue relocation code was written before config: introduce
num_votersandvoter_constraints#57184. We’ve since introduced the concept ofvoter_constraintswhich govern the constraints over just the set of voting replicas. This means that the relocation algorithm computes results that may not necessarily meet the {voter_}constraints placed over the RHS. In such cases, the range merge will fail. - It also has this limitation:
cockroach/pkg/kv/kvserver/replica_command.go
Lines 2547 to 2559 in df1c620
if len(finalRightVoters.Descriptors()) == 0 { // TODO(aayush): We can end up in this case for scenarios like the // following (the digits represent StoreIDs): // // LHS-> voters: {1, 2, 3}, non-voters: {} // RHS-> voters: {4}, non-voters: {1, 2, 3} // // Remove this error path once we support swapping voters and non-voters. return nil, nil, errors.UnimplementedErrorf(errors.IssueLink{IssueURL: build.MakeIssueURL(58499)}, "unsupported configuration of RHS(%s) and LHS(%s) as it requires an atomic swap of a"+ " voter and non-voter", rightRepls, leftRepls) }
We need to update AdminRelocateRange to use the aforementioned voter<->non-voter swaps and simply have the merge queue directly call into it, instead of having it perform its own calculation of where to move the {voter,non-voter} replicas.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
A-kv-replicationRelating to Raft, consensus, and coordination.Relating to Raft, consensus, and coordination.A-kv-replication-constraintsGA-blocker