keys: Clean up MakeRowSentinelKey and EnsureSafeSplitKey#16649
keys: Clean up MakeRowSentinelKey and EnsureSafeSplitKey#16649bdarnell merged 2 commits intocockroachdb:masterfrom a6802739:Clean_up_split
MakeRowSentinelKey and EnsureSafeSplitKey#16649Conversation
|
This looks good, but the tests are all failing. I think you also need to remove the call to MakeRowSentinelKey in Reviewed 5 of 5 files at r1. pkg/ccl/sqlccl/restore.go, line 536 at r1 (raw file):
Now you can remove this comment too. pkg/storage/replica_command.go, line 2642 at r1 (raw file):
Remove this section too. pkg/storage/engine/mvcc.go, line 2313 at r1 (raw file):
"The family ID has been removed from this key, making it a valid split point." Comments from Reviewable |
|
Review status: all files reviewed at latest revision, 4 unresolved discussions, some commit checks failed. pkg/storage/engine/mvcc.go, line 2308 at r1 (raw file):
I think this new code is the same as Comments from Reviewable |
|
@bdarnell I found a lot of error logs like if we remove And all the test just |
|
I think this line in IsValidSplitKey needs to be changed to use |
|
@bdarnell, I found it will report Need some help here. :) |
|
The remaining issues are test-only problems. Two tests were using MakeRowSentinelKey where it is no longer necessary, and an assertion in sql/main_test.go was incorrectly reporting errors in many tests because it was using keys.Addr when it should not. I've added a commit that should fix these. |
|
I had to rebase to pick up some build changes, and in the meantime another test was added that needed to be updated. This is ready to go now. |
|
Could you expand the commit message to (1) include the issue reference and (2) describe the actual change? Reviewed 1 of 5 files at r1, 12 of 12 files at r2. pkg/ccl/sqlccl/restore.go, line 536 at r2 (raw file):
why is this copy needed? pkg/keys/constants.go, line 247 at r2 (raw file):
why the different type? pkg/storage/client_split_test.go, line 165 at r2 (raw file):
why ignore these errors? pkg/storage/client_split_test.go, line 306 at r2 (raw file):
did you mean to leave this in? Comments from Reviewable |
|
Review status: 10 of 14 files reviewed at latest revision, 7 unresolved discussions. pkg/ccl/sqlccl/restore.go, line 536 at r2 (raw file): Previously, tamird (Tamir Duberstein) wrote…
I don't think it is any more. Deleted. pkg/keys/constants.go, line 247 at r2 (raw file): Previously, tamird (Tamir Duberstein) wrote…
MakeRowSentinelKey returns a pkg/storage/client_split_test.go, line 165 at r2 (raw file): Previously, tamird (Tamir Duberstein) wrote…
No good reason; added checks. pkg/storage/client_split_test.go, line 306 at r2 (raw file): Previously, tamird (Tamir Duberstein) wrote…
No, I meant to remove this whole test but forgot to push my last commit. Comments from Reviewable |
|
Reviewed 4 of 4 files at r3. Comments from Reviewable |
|
LGTM modulo commit message changes. Review status: all files reviewed at latest revision, 7 unresolved discussions, all commit checks successful. Comments from Reviewable |
|
Rewrote commit message and added another commit which removes MakeRowSentinelKey entirely. |
|
Should be Reviewed 15 of 15 files at r4. pkg/keys/keys.go, line 571 at r3 (raw file):
since there are at least two comments that reference special behaviour associated with this value, perhaps keep this constant is worthwhile, if only to have a place to anchor comments. pkg/sql/sqlbase/rowfetcher.go, line 399 at r4 (raw file):
isn't this also the case for column family zero, which is encoded without this value? pkg/sql/sqlbase/table.go, line 1351 at r4 (raw file):
this comment is somewhat incongruent now Comments from Reviewable |
EnsureSafeSplitKey transforms the requested split key in a way that is only appropriate in some contexts. Since this was called in AdminSplit itself, many call sites had to manually reverse this transformation (by calling MakeRowSentinelKey) in order to split at the desired location. This was easy to get wrong (see #16008). This commit removes EnsureSafeSplitKey from AdminSplit, calling it instead in MVCCFindSplitKey. Callers are now responsible for providing valid split keys directly, instead of a key which will be a valid split key after being transformed by EnsureSafeSplitKey. Fixes #16344
The concept of "row sentinel key" has been obsolete since the introduction of column families. Now that splits do not require a column family suffix, most calls to this method can be removed, and the few remaining cases where it is meaningful are replaced with an explicit use of column family 0.
|
Review status: 10 of 25 files reviewed at latest revision, 10 unresolved discussions. pkg/keys/keys.go, line 571 at r3 (raw file): Previously, tamird (Tamir Duberstein) wrote…
But the two places that referred to it were doing so for two separate reasons: column zero was a special sentinel in the pre-family encoding. With column families it's just a normal family with a small optimization in its key encoding. pkg/sql/sqlbase/rowfetcher.go, line 399 at r4 (raw file): Previously, tamird (Tamir Duberstein) wrote…
No, family zero has values (including the entire table by default). The family-aware format doesn't go through Comments from Reviewable |
|
Thanks, LGTM.
On Jul 14, 2017 20:21, "Ben Darnell" <notifications@github.com> wrote:
Review status: 10 of 25 files reviewed at latest revision, 10 unresolved
discussions.
------------------------------
*pkg/keys/keys.go, line 571 at r3
<https://reviewable.io:443/reviews/cockroachdb/cockroach/16649#-Kp2MKEqET_TJHHDxngZ:-Kp2aiNOvbsUOIRX7S-U:b-cw9787>
(raw file
<https://github.com/cockroachdb/cockroach/blob/cf94595c660c97786d3f34c1dbdf30b90306c9c9/pkg/keys/keys.go#L571>):*
*Previously, tamird (Tamir Duberstein) wrote…*
since there are at least two comments that reference special behaviour
associated with this value, perhaps keep this constant is worthwhile, if
only to have a place to anchor comments.
But the two places that referred to it were doing so for two separate
reasons: column zero was a special sentinel in the pre-family encoding.
With column families it's just a normal family with a small optimization in
its key encoding.
------------------------------
*pkg/sql/sqlbase/rowfetcher.go, line 399 at r4
<https://reviewable.io:443/reviews/cockroachdb/cockroach/16649#-Kp2LdkDs5rOsWHq88d5:-Kp2aLbPtVqDVKr05zT2:b-tmfbhq>
(raw file
<https://github.com/cockroachdb/cockroach/blob/e12446ef2534db864ceccf7c88e54c71d6c95d3d/pkg/sql/sqlbase/rowfetcher.go#L399>):*
*Previously, tamird (Tamir Duberstein) wrote…*
isn't this also the case for column family zero, which is encoded without
this value?
No, family zero has values (including the entire table by default). The
family-aware format doesn't go through processValueSingle (it uses
processValueTuple instead).
------------------------------
*Comments from Reviewable
<https://reviewable.io:443/reviews/cockroachdb/cockroach/16649>*
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#16649 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ABdsPL0FUyihZFRwPw3DgdC_5SQ5nP0Uks5sOAYVgaJpZM4OAxzO>
.
|
| // /System/"tse"-"ranges3" [4] | ||
| // 0: node-id=1 store-id=1 | ||
| // "ranges3"-/Table/0 [11] | ||
| // "ranges3"-/Table/SystemConfigSpan/Start [11] |
There was a problem hiding this comment.
There's a special case in the key pretty-printer for the start of the system config span (which would otherwise look like /Table/0/0). We try to split at that key, but before this change, EnsureSafeSplitKey changed the split key to /Table/0. Now the split point correctly shows that it is /Table/0/0 or its alias /Table/SystemConfigSpan/Start.
| } | ||
| if bytes.Compare(keyAddr, keys.SystemConfigSpan.Key) >= 0 && | ||
| bytes.Compare(keyAddr, keys.SystemConfigSpan.EndKey) < 0 { | ||
| if bytes.Compare(span.Key, keys.SystemConfigSpan.Key) >= 0 && |
There was a problem hiding this comment.
@bdarnell , I don't understand why we couldn't use keys.Addr here. Could you please explain here?
There was a problem hiding this comment.
We want to set the system config trigger for any data records in the system config span. We can determine that by looking at the keys alone. Using keys.Addr finds any records that live on the same range as the system config span, including range descriptors and transaction records. This bug was revealed when the split points changed so that range descriptors fall into this range.
This argument was added in 9234460 as a fix for cockroachdb#16008 and a short-term workaround for the cleanup in cockroachdb#16344. That cleanup was addressed in cockroachdb#16649, which removed the one place where spanKey and splitKey diverged (https://github.com/cockroachdb/cockroach/pull/16649/files#diff-6955fb0c3216b1d7d4292ccdf2e18dcf). They have been identical in all uses of AdminSplit ever since.
This argument was added in 9234460 as a fix for cockroachdb#16008 and a short-term workaround for the cleanup in cockroachdb#16344. That cleanup was addressed in cockroachdb#16649, which removed the one place where spanKey and splitKey diverged (https://github.com/cockroachdb/cockroach/pull/16649/files#diff-6955fb0c3216b1d7d4292ccdf2e18dcf). They have been identical in all uses of AdminSplit ever since.
Fixes #16344. @bdarnell