Skip to content

keys: Clean up MakeRowSentinelKey and EnsureSafeSplitKey#16649

Merged
bdarnell merged 2 commits intocockroachdb:masterfrom
a6802739:Clean_up_split
Jul 15, 2017
Merged

keys: Clean up MakeRowSentinelKey and EnsureSafeSplitKey#16649
bdarnell merged 2 commits intocockroachdb:masterfrom
a6802739:Clean_up_split

Conversation

@a6802739
Copy link
Copy Markdown
Contributor

Fixes #16344. @bdarnell

@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

@a6802739 a6802739 requested a review from bdarnell June 22, 2017 03:12
@bdarnell
Copy link
Copy Markdown
Contributor

This looks good, but the tests are all failing. I think you also need to remove the call to MakeRowSentinelKey in keys/constants.go:247: SystemConfigSplitKey = MakeRowSentinelKey(TableDataMin)


Reviewed 5 of 5 files at r1.
Review status: all files reviewed at latest revision, 3 unresolved discussions, some commit checks failed.


pkg/ccl/sqlccl/restore.go, line 536 at r1 (raw file):

		// Pick the index such that it's 0 if len(splitPoints) == 1.
		splitIdx := len(splitPoints) / 2
		// AdminSplit requires that the key be a valid table key, which means

Now you can remove this comment too. splitKey is just splitPoints[splitIdx] and the same value can be used for both arguments to AdminSplit.


pkg/storage/replica_command.go, line 2642 at r1 (raw file):

		}

		// EnsureSafeSplitKey could have changed the key, so we must

Remove this section too.


pkg/storage/engine/mvcc.go, line 2313 at r1 (raw file):

	}

	// The key has been removed family ID now.

"The family ID has been removed from this key, making it a valid split point."


Comments from Reviewable

@tamird
Copy link
Copy Markdown
Contributor

tamird commented Jun 22, 2017

Review status: all files reviewed at latest revision, 4 unresolved discussions, some commit checks failed.


pkg/storage/engine/mvcc.go, line 2308 at r1 (raw file):

	}

	splitKey, err := keys.EnsureSafeSplitKey(bestSplitKey.Key)

I think this new code is the same as return keys.EnsureSafeSplitKey(bestSplitKey.Key).


Comments from Reviewable

@a6802739
Copy link
Copy Markdown
Contributor Author

a6802739 commented Jul 6, 2017

@bdarnell I found a lot of error logs like

E170706 19:41:38.251592 49696 storage/queue.go:657  [split,s1,r4/1:/{System/tse-Max}] unable to split [n1,s1,r4/1:/{System/tse-
Max}] at key "/Table/0": storage/replica_command.go:2658: cannot split range at key /Table/0

if we remove MakeRowSentinelKey for SystemConfigSplitKey, will the split be invalid?

And all the test just panic: test timed out after 4m0s.

@bdarnell
Copy link
Copy Markdown
Contributor

bdarnell commented Jul 6, 2017

I think this line in IsValidSplitKey needs to be changed to use > instead of >=. We want to allow splits at either end of the NoSplitSpans, just not in the middle.

@a6802739
Copy link
Copy Markdown
Contributor Author

a6802739 commented Jul 7, 2017

@bdarnell, I found it will report had 4 ranges at startup, expected 10 for tests in test_server_shim.go, I don't know what does this mean.

Need some help here. :)

@bdarnell
Copy link
Copy Markdown
Contributor

The remaining issues are test-only problems. Two tests were using MakeRowSentinelKey where it is no longer necessary, and an assertion in sql/main_test.go was incorrectly reporting errors in many tests because it was using keys.Addr when it should not. I've added a commit that should fix these.

@bdarnell
Copy link
Copy Markdown
Contributor

I had to rebase to pick up some build changes, and in the meantime another test was added that needed to be updated. This is ready to go now.

@tamird
Copy link
Copy Markdown
Contributor

tamird commented Jul 14, 2017

Could you expand the commit message to (1) include the issue reference and (2) describe the actual change?


Reviewed 1 of 5 files at r1, 12 of 12 files at r2.
Review status: all files reviewed at latest revision, 7 unresolved discussions, all commit checks successful.


pkg/ccl/sqlccl/restore.go, line 536 at r2 (raw file):

		// Pick the index such that it's 0 if len(splitPoints) == 1.
		splitIdx := len(splitPoints) / 2
		splitKey := append([]byte(nil), splitPoints[splitIdx]...)

why is this copy needed?


pkg/keys/constants.go, line 247 at r2 (raw file):

	// SystemConfigSplitKey is the key to split at immediately prior to the
	// system config span. NB: Split keys need to be valid column keys.
	SystemConfigSplitKey = []byte(TableDataMin)

why the different type?


pkg/storage/client_split_test.go, line 165 at r2 (raw file):

	rowKey := roachpb.Key(encoding.EncodeVarintAscending(append([]byte(nil), tableKey...), 1))
	rowKey = encoding.EncodeStringAscending(encoding.EncodeVarintAscending(rowKey, 1), "a")
	col1Key, _ := keys.EnsureSafeSplitKey(keys.MakeFamilyKey(append([]byte(nil), rowKey...), 1))

why ignore these errors?


pkg/storage/client_split_test.go, line 306 at r2 (raw file):

func TestEnsureSafeSplitKeyOutOfBounds(t *testing.T) {
	defer leaktest.AfterTest(t)()
	t.Skip()

did you mean to leave this in?


Comments from Reviewable

@bdarnell
Copy link
Copy Markdown
Contributor

Review status: 10 of 14 files reviewed at latest revision, 7 unresolved discussions.


pkg/ccl/sqlccl/restore.go, line 536 at r2 (raw file):

Previously, tamird (Tamir Duberstein) wrote…

why is this copy needed?

I don't think it is any more. Deleted.


pkg/keys/constants.go, line 247 at r2 (raw file):

Previously, tamird (Tamir Duberstein) wrote…

why the different type?

MakeRowSentinelKey returns a []byte, so this is the same as it was before this PR. It looks like it should be either a roachpb.Key or roachpb.RKey, but it's used inconsistently today. Added a TODO here and on MakeRowSentinelKey to change to a more precise type.


pkg/storage/client_split_test.go, line 165 at r2 (raw file):

Previously, tamird (Tamir Duberstein) wrote…

why ignore these errors?

No good reason; added checks.


pkg/storage/client_split_test.go, line 306 at r2 (raw file):

Previously, tamird (Tamir Duberstein) wrote…

did you mean to leave this in?

No, I meant to remove this whole test but forgot to push my last commit.


Comments from Reviewable

@tamird
Copy link
Copy Markdown
Contributor

tamird commented Jul 14, 2017

Reviewed 4 of 4 files at r3.
Review status: all files reviewed at latest revision, 7 unresolved discussions, some commit checks pending.


Comments from Reviewable

@tamird
Copy link
Copy Markdown
Contributor

tamird commented Jul 14, 2017

LGTM modulo commit message changes.


Review status: all files reviewed at latest revision, 7 unresolved discussions, all commit checks successful.


Comments from Reviewable

@bdarnell
Copy link
Copy Markdown
Contributor

Rewrote commit message and added another commit which removes MakeRowSentinelKey entirely.

@tamird
Copy link
Copy Markdown
Contributor

tamird commented Jul 14, 2017

:lgtm: mod typo in first commit message:

MVCCFIndSplitKey

Should be MVCCFindSplitKey


Reviewed 15 of 15 files at r4.
Review status: all files reviewed at latest revision, 10 unresolved discussions.


pkg/keys/keys.go, line 571 at r3 (raw file):

// SentinelFamilyID indicates that MakeFamilyKey should make a sentinel key.
const SentinelFamilyID = 0

since there are at least two comments that reference special behaviour associated with this value, perhaps keep this constant is worthwhile, if only to have a place to anchor comments.


pkg/sql/sqlbase/rowfetcher.go, line 399 at r4 (raw file):

	prettyKey = prettyKeyPrefix

	// If this is the row sentinel (in the legacy pre-family format),

isn't this also the case for column family zero, which is encoded without this value?


pkg/sql/sqlbase/table.go, line 1351 at r4 (raw file):

	}

	// Index keys are considered "sentinel" keys in that they do not have a

this comment is somewhat incongruent now


Comments from Reviewable

a6802739 and others added 2 commits July 14, 2017 20:13
EnsureSafeSplitKey transforms the requested split key in a way that is
only appropriate in some contexts. Since this was called in AdminSplit
itself, many call sites had to manually reverse this
transformation (by calling MakeRowSentinelKey) in order to split at
the desired location. This was easy to get wrong (see #16008).

This commit removes EnsureSafeSplitKey from AdminSplit, calling it
instead in MVCCFindSplitKey. Callers are now responsible for providing
valid split keys directly, instead of a key which will be a valid
split key after being transformed by EnsureSafeSplitKey.

Fixes #16344
The concept of "row sentinel key" has been obsolete since the
introduction of column families. Now that splits do not require a
column family suffix, most calls to this method can be removed, and
the few remaining cases where it is meaningful are replaced with an
explicit use of column family 0.
@bdarnell
Copy link
Copy Markdown
Contributor

Review status: 10 of 25 files reviewed at latest revision, 10 unresolved discussions.


pkg/keys/keys.go, line 571 at r3 (raw file):

Previously, tamird (Tamir Duberstein) wrote…

since there are at least two comments that reference special behaviour associated with this value, perhaps keep this constant is worthwhile, if only to have a place to anchor comments.

But the two places that referred to it were doing so for two separate reasons: column zero was a special sentinel in the pre-family encoding. With column families it's just a normal family with a small optimization in its key encoding.


pkg/sql/sqlbase/rowfetcher.go, line 399 at r4 (raw file):

Previously, tamird (Tamir Duberstein) wrote…

isn't this also the case for column family zero, which is encoded without this value?

No, family zero has values (including the entire table by default). The family-aware format doesn't go through processValueSingle (it uses processValueTuple instead).


Comments from Reviewable

@tamird
Copy link
Copy Markdown
Contributor

tamird commented Jul 15, 2017 via email

// /System/"tse"-"ranges3" [4]
// 0: node-id=1 store-id=1
// "ranges3"-/Table/0 [11]
// "ranges3"-/Table/SystemConfigSpan/Start [11]
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bdarnell, Why we should change here?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a special case in the key pretty-printer for the start of the system config span (which would otherwise look like /Table/0/0). We try to split at that key, but before this change, EnsureSafeSplitKey changed the split key to /Table/0. Now the split point correctly shows that it is /Table/0/0 or its alias /Table/SystemConfigSpan/Start.

}
if bytes.Compare(keyAddr, keys.SystemConfigSpan.Key) >= 0 &&
bytes.Compare(keyAddr, keys.SystemConfigSpan.EndKey) < 0 {
if bytes.Compare(span.Key, keys.SystemConfigSpan.Key) >= 0 &&
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bdarnell , I don't understand why we couldn't use keys.Addr here. Could you please explain here?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We want to set the system config trigger for any data records in the system config span. We can determine that by looking at the keys alone. Using keys.Addr finds any records that live on the same range as the system config span, including range descriptors and transaction records. This bug was revealed when the split points changed so that range descriptors fall into this range.

@bdarnell bdarnell merged commit b23fbc0 into cockroachdb:master Jul 15, 2017
nvb added a commit to nvb/cockroach that referenced this pull request Jun 5, 2020
This argument was added in 9234460 as a fix for cockroachdb#16008 and a short-term
workaround for the cleanup in cockroachdb#16344. That cleanup was addressed in cockroachdb#16649,
which removed the one place where spanKey and splitKey diverged
(https://github.com/cockroachdb/cockroach/pull/16649/files#diff-6955fb0c3216b1d7d4292ccdf2e18dcf).
They have been identical in all uses of AdminSplit ever since.
nvb added a commit to nvb/cockroach that referenced this pull request Jun 9, 2020
This argument was added in 9234460 as a fix for cockroachdb#16008 and a short-term
workaround for the cleanup in cockroachdb#16344. That cleanup was addressed in cockroachdb#16649,
which removed the one place where spanKey and splitKey diverged
(https://github.com/cockroachdb/cockroach/pull/16649/files#diff-6955fb0c3216b1d7d4292ccdf2e18dcf).
They have been identical in all uses of AdminSplit ever since.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants