Skip to content

storage: Balance leaseholders based on load #21419

@a-robinson

Description

@a-robinson

Currently we only rebalance leaseholders based on the number of leases per store, not by how much load is on each. We have an experimental form of rebalancing replicas themselves based on load (RFC), but never did any similar work for leases despite having a lower cost (transferring a lease is much cheaper than creating a new replica). This can lead to situations where all the hot ranges in a cluster are on the same node, overloading that node while others are hardly utilized at all.

We already have stats for the number of requests being processed by each range and store, so the data is already in place. The work would primarily be to avoid over-correcting for transient changes in load and to handle the conflicting demands of balancing the load on each node vs balancing moving leases closer to load when running in a distributed cluster.

Metadata

Metadata

Assignees

Labels

A-kv-distributionRelating to rebalancing and leasing.C-enhancementSolution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions