storage: filter unusable stores from candidate count#12565
storage: filter unusable stores from candidate count#12565petermattis merged 1 commit intocockroachdb:masterfrom
Conversation
|
Reviewed 2 of 2 files at r1. pkg/storage/allocator.go, line 449 at r1 (raw file):
Can't this be simplified to get rid of the exclude variable by using a label? Not sure if we have a policy against labels, but this is an easy case for one. On the other hand, is it worth creating a map to not have to do the inner loop at all? I know it's going to normally be 3 items, but we do this a lot throughout the allocator and balancer, perhaps we should create that map once at the start. pkg/storage/allocator_test.go, line 1032 at r1 (raw file):
Since existing is always the same, why keep it in the struct? pkg/storage/allocator_test.go, line 1042 at r1 (raw file):
Shouldn't the subtest have a name? Just the leaseholder should be fine. Comments from Reviewable |
Stores that are on the same node as existing replicas are not usable targets for rebalancing which can foul up the lease rebalancing heuristics. In particular, in a 3-node cluster with 2 stores per node a 3 replica range will see half the stores as being unusable for rebalancing. Not being able to rebalance replicas to those stores then fouls up the lease rebalancing heuristics which tries to keep the number of leases per store close to the average. Now we only consider stores which are usable rebalance targets in compute the average lease per store. Fixes cockroachdb#12322
4866eda to
2bd3ce4
Compare
|
Review status: 1 of 2 files reviewed at latest revision, 3 unresolved discussions. pkg/storage/allocator.go, line 449 at r1 (raw file): Previously, BramGruneir (Bram Gruneir) wrote…
Yes, this can be changed as you suggest, though the benefit is small given that this code will be going away when we switch to the new allocator. I'd prefer to leave as-is unless you feel really strongly about it. pkg/storage/allocator_test.go, line 1032 at r1 (raw file): Previously, BramGruneir (Bram Gruneir) wrote…
Because pkg/storage/allocator_test.go, line 1042 at r1 (raw file): Previously, BramGruneir (Bram Gruneir) wrote…
We don't have any requirement about giving subtests names. So far, I find doing so onerous and prefer to use the automatically generated names (i.e. Comments from Reviewable |
|
Reviewed 1 of 1 files at r2. pkg/storage/allocator.go, line 449 at r1 (raw file): Previously, petermattis (Peter Mattis) wrote…
Is this code going away soon? This is part of lease transferring and I haven't ported any lease stuff over yet. I'd like to port if over, but I think there are more important parts of replica rebalancing that need to be addressed first. That being said, I don't feel strongly about it, I find labels to be almost like gotos and I'm not a big fan. pkg/storage/allocator_test.go, line 1042 at r1 (raw file): Previously, petermattis (Peter Mattis) wrote…
I find it easier to pinpoint a test case when there are a large number of them. But with such a small number, this is fine. Comments from Reviewable |
|
Review status: all files reviewed at latest revision, 2 unresolved discussions, all commit checks successful. pkg/storage/allocator.go, line 449 at r1 (raw file): Previously, BramGruneir (Bram Gruneir) wrote…
This code needs to be adapted to the rule-allocator very soon. Any significant usage of diversity or constraints will foul up lease rebalancing. Comments from Reviewable |
In cockroachdb#51567 we added the ability to rebalance replicas between stores on the same node. This PR reverts the changes introduced in cockroachdb#12565, since we no longer need to special case multiple stores per node. We also change the filter condition for eligible stores to use zone.VoterConstraints, since leases can only move to voters right now. Release note: None
In cockroachdb#51567 we added the ability to rebalance replicas between stores on the same node. This PR reverts the changes introduced in cockroachdb#12565, since we no longer need to special case multiple stores per node. Release note: None
63390: opt: remove view deps user defined types used by dependent table r=rytaft,mgartner a=the-ericwang35 Fixes #63191. Previously, when a view depended on a table that had a user defined type column, we would create a dependency between the view and the type, even if the view did not directly reference that column. This is because the table referencing the UDT has an implicit CHECK constraint on the type. For example, ``` CREATE TYPE typ AS ENUM('a', 'b', 'c'); CREATE TABLE t (i INT, j typ); # Has check constraint (j IN (x'40':::@100053, x'80':::@100053, x'c0':::@100053)) CREATE VIEW v AS (SELECT i FROM t); # Has a dependency on typ. ``` When we create `v`, it calls `buildScalar` on `t`'s check constraint and then `maybeTrackUserDefinedTypeDepsForViews`, which adds dependencies to the types even if the view does not directly use them. This patch addresses this by not capturing this dependency inside `maybeTrackUserDefinedTypeDepsForViews`. Instead, it will only capture dependencies explicitly used in the view. Then, in `ConstructCreateView`, we will look for column dependencies, and see if any columns are a UDT. Release note: None 63489: kvserver: remove same node check from TransferLeaseTarget r=lunevalex a=lunevalex In #51567 we added the ability to rebalance replicas between stores on the same node. This PR reverts the changes introduced in #12565, since we no longer need to special case multiple stores per node. Release note: None Co-authored-by: Eric Wang <ericw@cockroachlabs.com> Co-authored-by: Alex Lunev <alexl@cockroachlabs.com>
Stores that are on the same node as existing replicas are not usable
targets for rebalancing which can foul up the lease rebalancing
heuristics. In particular, in a 3-node cluster with 2 stores per node a
3 replica range will see half the stores as being unusable for
rebalancing. Not being able to rebalance replicas to those stores then
fouls up the lease rebalancing heuristics which tries to keep the number
of leases per store close to the average. Now we only consider stores
which are usable rebalance targets in compute the average lease per
store.
Fixes #12322
This change is