-
Notifications
You must be signed in to change notification settings - Fork 4.1k
storage: Balance leaseholders based on load #21419
Description
Currently we only rebalance leaseholders based on the number of leases per store, not by how much load is on each. We have an experimental form of rebalancing replicas themselves based on load (RFC), but never did any similar work for leases despite having a lower cost (transferring a lease is much cheaper than creating a new replica). This can lead to situations where all the hot ranges in a cluster are on the same node, overloading that node while others are hardly utilized at all.
We already have stats for the number of requests being processed by each range and store, so the data is already in place. The work would primarily be to avoid over-correcting for transient changes in load and to handle the conflicting demands of balancing the load on each node vs balancing moving leases closer to load when running in a distributed cluster.