-
Notifications
You must be signed in to change notification settings - Fork 4.1k
kvserver: load based rebalancing within cold regions in heterogenous workloads #69901
Description
With #65379, the StoreRebalancer is able to perform QPS-based lease and replica rebalancing in the "hot" regions in a cluster experiencing heterogeneous load across its regions.
However, as of the time of writing this, the StoreRebalancer is only activated on stores serving QPS sufficiently above the average QPS across all stores in the cluster. This means that, in a heterogeneously loaded cluster, the colder regions won't rebalance within themselves until the degree of imbalance gets high enough that one of the stores is above the "overfullness threshold".
The caveat against lifting this restriction is that we might be too aggressive in performing load based {lease,replica} rebalancing on stores that have no need for it, and end up actually hurting QPS overall due to the overheads associated with lease transfers and replica rebalances.
We should investigate the impact of allowing load based rebalances on colder stores, and / or predicate the activation of the StoreRebalancer on a tunable QPS threshold.
Jira issue: CRDB-9853