Skip to content

storage: filter unusable stores from candidate count#12565

Merged
petermattis merged 1 commit intocockroachdb:masterfrom
petermattis:pmattis/transfer-lease-target
Jan 2, 2017
Merged

storage: filter unusable stores from candidate count#12565
petermattis merged 1 commit intocockroachdb:masterfrom
petermattis:pmattis/transfer-lease-target

Conversation

@petermattis
Copy link
Copy Markdown
Collaborator

@petermattis petermattis commented Dec 23, 2016

Stores that are on the same node as existing replicas are not usable
targets for rebalancing which can foul up the lease rebalancing
heuristics. In particular, in a 3-node cluster with 2 stores per node a
3 replica range will see half the stores as being unusable for
rebalancing. Not being able to rebalance replicas to those stores then
fouls up the lease rebalancing heuristics which tries to keep the number
of leases per store close to the average. Now we only consider stores
which are usable rebalance targets in compute the average lease per
store.

Fixes #12322


This change is Reviewable

@BramGruneir
Copy link
Copy Markdown
Member

Reviewed 2 of 2 files at r1.
Review status: all files reviewed at latest revision, 3 unresolved discussions.


pkg/storage/allocator.go, line 449 at r1 (raw file):

	// stores the replicas are on.
	filteredDescs := make([]roachpb.StoreDescriptor, 0, len(sl.stores))
	for _, s := range sl.stores {

Can't this be simplified to get rid of the exclude variable by using a label? Not sure if we have a policy against labels, but this is an easy case for one.

OUTER:
for _, s := range sl.stores {
  for _, r := range existing {
    if r.NodeID == s.Node.NodeID && r.StoreID != s.StoreID {
      continue OUTER 
    }
  }
  filteredDesc = append(filteredDescs, s)
}

On the other hand, is it worth creating a map to not have to do the inner loop at all? I know it's going to normally be 3 items, but we do this a lot throughout the allocator and balancer, perhaps we should create that map once at the start.


pkg/storage/allocator_test.go, line 1032 at r1 (raw file):

	testCases := []struct {
		existing    []roachpb.ReplicaDescriptor

Since existing is always the same, why keep it in the struct?


pkg/storage/allocator_test.go, line 1042 at r1 (raw file):

	}
	for _, c := range testCases {
		t.Run("", func(t *testing.T) {

Shouldn't the subtest have a name? Just the leaseholder should be fine.


Comments from Reviewable

Stores that are on the same node as existing replicas are not usable
targets for rebalancing which can foul up the lease rebalancing
heuristics. In particular, in a 3-node cluster with 2 stores per node a
3 replica range will see half the stores as being unusable for
rebalancing. Not being able to rebalance replicas to those stores then
fouls up the lease rebalancing heuristics which tries to keep the number
of leases per store close to the average. Now we only consider stores
which are usable rebalance targets in compute the average lease per
store.

Fixes cockroachdb#12322
@petermattis petermattis force-pushed the pmattis/transfer-lease-target branch from 4866eda to 2bd3ce4 Compare December 24, 2016 18:58
@petermattis
Copy link
Copy Markdown
Collaborator Author

Review status: 1 of 2 files reviewed at latest revision, 3 unresolved discussions.


pkg/storage/allocator.go, line 449 at r1 (raw file):

Previously, BramGruneir (Bram Gruneir) wrote…

Can't this be simplified to get rid of the exclude variable by using a label? Not sure if we have a policy against labels, but this is an easy case for one.

OUTER:
for _, s := range sl.stores {
  for _, r := range existing {
    if r.NodeID == s.Node.NodeID && r.StoreID != s.StoreID {
      continue OUTER 
    }
  }
  filteredDesc = append(filteredDescs, s)
}

On the other hand, is it worth creating a map to not have to do the inner loop at all? I know it's going to normally be 3 items, but we do this a lot throughout the allocator and balancer, perhaps we should create that map once at the start.

Yes, this can be changed as you suggest, though the benefit is small given that this code will be going away when we switch to the new allocator. I'd prefer to leave as-is unless you feel really strongly about it.


pkg/storage/allocator_test.go, line 1032 at r1 (raw file):

Previously, BramGruneir (Bram Gruneir) wrote…

Since existing is always the same, why keep it in the struct?

Because TestAllocatorTransferLeaseTarget does that. Changed.


pkg/storage/allocator_test.go, line 1042 at r1 (raw file):

Previously, BramGruneir (Bram Gruneir) wrote…

Shouldn't the subtest have a name? Just the leaseholder should be fine.

We don't have any requirement about giving subtests names. So far, I find doing so onerous and prefer to use the automatically generated names (i.e. #00, #01, etc).


Comments from Reviewable

@BramGruneir
Copy link
Copy Markdown
Member

:lgtm:


Reviewed 1 of 1 files at r2.
Review status: all files reviewed at latest revision, 2 unresolved discussions, all commit checks successful.


pkg/storage/allocator.go, line 449 at r1 (raw file):

Previously, petermattis (Peter Mattis) wrote…

Yes, this can be changed as you suggest, though the benefit is small given that this code will be going away when we switch to the new allocator. I'd prefer to leave as-is unless you feel really strongly about it.

Is this code going away soon? This is part of lease transferring and I haven't ported any lease stuff over yet. I'd like to port if over, but I think there are more important parts of replica rebalancing that need to be addressed first.

That being said, I don't feel strongly about it, I find labels to be almost like gotos and I'm not a big fan.


pkg/storage/allocator_test.go, line 1042 at r1 (raw file):

Previously, petermattis (Peter Mattis) wrote…

We don't have any requirement about giving subtests names. So far, I find doing so onerous and prefer to use the automatically generated names (i.e. #00, #01, etc).

I find it easier to pinpoint a test case when there are a large number of them. But with such a small number, this is fine.


Comments from Reviewable

@petermattis
Copy link
Copy Markdown
Collaborator Author

Review status: all files reviewed at latest revision, 2 unresolved discussions, all commit checks successful.


pkg/storage/allocator.go, line 449 at r1 (raw file):

Previously, BramGruneir (Bram Gruneir) wrote…

Is this code going away soon? This is part of lease transferring and I haven't ported any lease stuff over yet. I'd like to port if over, but I think there are more important parts of replica rebalancing that need to be addressed first.

That being said, I don't feel strongly about it, I find labels to be almost like gotos and I'm not a big fan.

This code needs to be adapted to the rule-allocator very soon. Any significant usage of diversity or constraints will foul up lease rebalancing.


Comments from Reviewable

@petermattis petermattis merged commit 28e2034 into cockroachdb:master Jan 2, 2017
@petermattis petermattis deleted the pmattis/transfer-lease-target branch January 2, 2017 13:37
lunevalex pushed a commit to lunevalex/cockroach that referenced this pull request Apr 12, 2021
In cockroachdb#51567 we added the ability to rebalance replicas between
stores on the same node. This PR reverts the changes
introduced in cockroachdb#12565, since we no longer need to special case
multiple stores per node.

We also change the filter condition for eligible stores to
use zone.VoterConstraints, since leases can only move to
voters right now.

Release note: None
lunevalex pushed a commit to lunevalex/cockroach that referenced this pull request Apr 12, 2021
In cockroachdb#51567 we added the ability to rebalance replicas between
stores on the same node. This PR reverts the changes
introduced in cockroachdb#12565, since we no longer need to special case
multiple stores per node.

Release note: None
craig bot pushed a commit that referenced this pull request Apr 14, 2021
63390: opt: remove view deps user defined types used by dependent table r=rytaft,mgartner a=the-ericwang35

Fixes #63191.

Previously, when a view depended on a table that had
a user defined type column, we would create a
dependency between the view and the type, even if
the view did not directly reference that column.
This is because the table referencing the UDT 
has an implicit CHECK constraint on the type.

For example,
```
CREATE TYPE typ AS ENUM('a', 'b', 'c');
CREATE TABLE t (i INT, j typ); # Has check constraint (j IN (x'40':::@100053, x'80':::@100053, x'c0':::@100053))
CREATE VIEW v AS (SELECT i FROM t); # Has a dependency on typ.
```

When we create `v`, it calls `buildScalar` on `t`'s check constraint
and then `maybeTrackUserDefinedTypeDepsForViews`,  which
adds dependencies to the types even if the view does
not directly use them. 
 
This patch addresses this by not capturing this dependency
inside `maybeTrackUserDefinedTypeDepsForViews`.
Instead, it will only capture dependencies explicitly used in
the view. Then, in `ConstructCreateView`, we will look for
column dependencies, and see if any columns are a UDT.

Release note: None

63489: kvserver: remove same node check from TransferLeaseTarget r=lunevalex a=lunevalex

In #51567 we added the ability to rebalance replicas between
stores on the same node. This PR reverts the changes
introduced in #12565, since we no longer need to special case
multiple stores per node.

Release note: None

Co-authored-by: Eric Wang <ericw@cockroachlabs.com>
Co-authored-by: Alex Lunev <alexl@cockroachlabs.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

storage: 3-node cluster not balancing leases

2 participants