Conversation
|
Current dependencies on/for this PR:
This comment was auto-generated by Graphite. |
|
You have successfully added a new CodeQL configuration |
|
You have successfully added a new CodeQL configuration |
Codecov ReportPatch coverage:
📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more Additional details and impacted files@@ Coverage Diff @@
## random_replica_selector #396 +/- ##
===========================================================
+ Coverage 63.61% 63.68% +0.07%
===========================================================
Files 131 131
Lines 17772 17793 +21
===========================================================
+ Hits 11306 11332 +26
+ Misses 5925 5922 -3
+ Partials 541 539 -2
... and 8 files with indirect coverage changes Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. ☔ View full report in Codecov by Sentry. |
hungryjang
left a comment
There was a problem hiding this comment.
It seems that there will be an issue with the primary being crowded. The selection algorithm is likely to change several times.
@hungryjang, you are right. Simply picking a storage node cannot balance the number of primary replicas among the nodes, and LFU selection selects the best node possible. We can add new strategies with new names. |
bbfd462 to
93b86ff
Compare
354f35c to
690b414
Compare
This PR adds a new replica selector based on the Least Frequently Used (LFU) algorithm. `lfuReplicaSelector` selects each replica's storage node and data path, giving preference to those with fewer assigned replicas. All replica selectors, including `lfuReplicaSelector`, are stateless, meaning they don't keep existing log stream topology. When `Select` is invoked, `lfuReplicaSelector` collects used counters for each storage node and path. Although it seems to be inefficient, it is simple as well as fault-tolerant. To select the least used storage nodes and paths, `lfuReplicaSelector` runs as follows: - Fetch cluster metadata from the `ClusterMetadataView`. - Increase the used counters for each storage node and path assigned to each log stream. - Sort counters. - Choose the least used storage nodes and paths. Updates #393
93b86ff to
ad1226e
Compare
690b414 to
0e08901
Compare
What this PR does
This PR adds a new replica selector based on the Least Frequently Used (LFU) algorithm.
lfuReplicaSelectorselects each replica's storage node and data path, giving preference to thosewith fewer assigned replicas.
All replica selectors, including
lfuReplicaSelector, are stateless, meaning they don't keepexisting log stream topology. When
Selectis invoked,lfuReplicaSelectorcollects used countersfor each storage node and path. Although it seems to be inefficient, it is simple as well as
fault-tolerant.
To select the least used storage nodes and paths,
lfuReplicaSelectorruns as follows:ClusterMetadataView.Which issue(s) this PR resolves
Updates #393