The StoreRebalancer tries to find replacement stores for all replicas of a hot range on "overfull" stores.

Currently, if it cannot find a replacement for all replicas on overfull stores, it will decide to ignore the range and try the next hottest range. This renders our load based rebalancing less effective than it could be, since it won't act on a range unless it can move all replicas on overfull stores to stores that are not overfull. The StoreRebalancer's lack of constraints awareness (as explained in #61883), further exacerbates this problem under cluster topologies where different regions have different load profiles.

We should consider re-organizing some of the logic here to make it such that our load based rebalancing logic will replace the subset of replicas that it can, even if it cannot replace all of them. In other words, if the StoreRebalancer can move a range from a configuration that has 5 replicas on overfull stores to a configuration that only has 4 replicas on overfull stores, it should be able to do so.

Additional context:

• This issue is somewhat related to (but not the same as) storage: Load-based rebalancing should be more willing to make certain incremental improvements #31135
• Some discussion over at: kvserver: make the StoreRebalancer aware of non-voters #61600 (review)

gz#7724