-
Notifications
You must be signed in to change notification settings - Fork 4.1k
storage: re-enable stats-based rebalancing #17979
Copy link
Copy link
Closed
Labels
A-kv-distributionRelating to rebalancing and leasing.Relating to rebalancing and leasing.C-enhancementSolution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)
Milestone
Description
#17968 disabled stats-based rebalancing because of unresolved issues under certain situations.
Re-enabling it wil involve:
- Fix behavior under splits without load (found while testing
sky) storage: consider disabling kv.allocator.stat_based_rebalancing.enabled by default #17645 - Fixing storage: stats-based rebalancing can't handle large numbers of splits and scatters storage: stats-based rebalancing can't handle large numbers of splits and scatters #17671
- Fixing storage: Delayed update of per-store write stats can cause rebalance thrashing storage: Delayed update of per-store write stats can cause rebalance thrashing #17970
- Fixing storage: Consider which replica will be removed when adding a replica to improve balance storage: Consider which replica will be removed when adding a replica to improve balance #17971
- Fix the success criteria for the nightly allocator tests. Stats-based rebalancing means that we're no longer balancing just range counts, so the nightly allocator tests need a different set of metrics to measure. acceptance: stats-based rebalancing makes Allocator Test 3 to 5 10G flaky #17685
- Ideally, introducing an allocator acceptance test (a CI acceptance test rather than a nightly) that tests the allocator's behavior in various situations.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
A-kv-distributionRelating to rebalancing and leasing.Relating to rebalancing and leasing.C-enhancementSolution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)