Skip to content

roachpb: enforce checksum optionality, add Value.EqualData method#24126

Merged
nvb merged 1 commit intocockroachdb:masterfrom
nvb:nvanbenschoten/ignoreChecksum
Mar 21, 2018
Merged

roachpb: enforce checksum optionality, add Value.EqualData method#24126
nvb merged 1 commit intocockroachdb:masterfrom
nvb:nvanbenschoten/ignoreChecksum

Conversation

@nvb
Copy link
Copy Markdown
Contributor

@nvb nvb commented Mar 21, 2018

Fixes #23984.
Fixes #22636.

In #22636 we noticed that conditional puts were not respecting the fact
that Value checksums are meant to be optional. This is because they include
checksums when performing the equality check between expected and existing
values. This caused serious issues and forced us to roll back 75cbeb8.
This was unfortunate, but the damage was contained. It demonstrated that
we need to be very careful about changing checksums on values.

Later, we found in #23984 that a path somewhere in IMPORT was forgetting
to set checksums on Values. This was causing SQL operations at a much later
time to fail for the same reason that 75cbeb8 caused issues. This bug will
be fixed in a different change, but the fact that it exists shows how
fragile the current approach is because despite the intention, checksums
are absolutely not optional.

This change addresses both of these problems by making checksums optional,
as they were intended to be. CPut and InitPut no longer compare the checksums
of the expected and existing values. Instead, they call into a new EqualData
method, which ignores the checksum header when comparing byte values.

This is a bit higher risk than most of the changes we've been accepting at
this stage of the release cycle, but it would be really nice to get this
change in to 2.0 so that we can stay on track with the timeline proposed
in #22636 (comment)
(although I'm not positive that the last step of the timeline is correct
because I don't think we can rely on the "baked in" protection with such
a low-level change). It will also save anyone who has used IMPORT already
and hit #22636.

This change should be safe from beneath Raft inconsistency for the same
reason that 75cbeb8 was safe. All CPuts beneath Raft are still careful
to set checksums, so we should not see any divergence in Raft state
because of this error handling change.

After this change, the only uses of bytes.Equal(v1.RawBytes, v2.RawBytes)
are in tests and the value.Equal method. There may be an argument for
making even value.Equal ignore the checksum header, although it isn't
used outside of other proto.Equal methods.

Nathans-MacBook-Pro:cockroach$ grep -lRE 'bytes\.Equal\(.*\.RawBytes,*,.*\.RawBytes,*\)' pkg
pkg/roachpb/data.pb.go
pkg/storage/engine/mvcc_test.go

Release note: None

Fixes cockroachdb#23984.
Fixes cockroachdb#22636.

In cockroachdb#22636 we noticed that conditional puts were not respecting the fact
that Value checksums are meant to be optional. This is because they include
checksums when performing the equality check between expected and existing
values. This caused serious issues and forced us to roll back 75cbeb8.
This was unfortunate, but the damage was contained. It demonstrated that
we need to be very careful about changing checksums on values.

Later, we found in cockroachdb#23984 that a path somewhere in IMPORT was forgetting
to set checksums on Values. This was causing SQL operations at a much later
time to fail for the same reason that 75cbeb8 caused issues. This bug will
be fixed in a different change, but the fact that it exists shows how
fragile the current approach is because despite the intention, checksums
are absolutely not optional.

This change addresses both of these problems by making checksums optional,
as they were intended to be. CPut and InitPut no longer compare the checksums
of the expected and existing values. Instead, they call into a new `EqualData`
method, which ignores the checksum header when comparing byte values.

This is a bit higher risk than most of the changes we've been accepting at
this stage of the release cycle, but it would be really nice to get this
change in to 2.0 so that we can stay on track with the timeline proposed
in cockroachdb#22636 (comment)
(although I'm not positive that the last step of the timeline is correct
because I don't think we can rely on the "baked in" protection with such
a low-level change). It will also save anyone who has used IMPORT already
and hit cockroachdb#22636.

This change should be safe from beneath Raft inconsistency for the same
reason that 75cbeb8 was safe. All CPuts beneath Raft are still careful
to set checksums, so we should not see any divergence in Raft state
because of this error handling change.

After this change, the only uses of `bytes.Equal(v1.RawBytes, v2.RawBytes)`
are in tests and the `value.Equal` method. There may be an argument for
making even `value.Equal` ignore the checksum header, although it isn't
used outside of other `proto.Equal` methods.
```
Nathans-MacBook-Pro:cockroach$ grep -lRE 'bytes\.Equal\(.*\.RawBytes,*,.*\.RawBytes,*\)' pkg
pkg/roachpb/data.pb.go
pkg/storage/engine/mvcc_test.go
```

Release note: None
@nvb nvb requested review from a team and bdarnell March 21, 2018 19:59
@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

@nvb
Copy link
Copy Markdown
Contributor Author

nvb commented Mar 21, 2018

I'm going to spend some time running this with the version roachtest and with the TestVersionUpgrade acceptance test.

@bdarnell
Copy link
Copy Markdown
Contributor

:lgtm:


Review status: 0 of 7 files reviewed at latest revision, all discussions resolved, some commit checks pending.


Comments from Reviewable

@nvb
Copy link
Copy Markdown
Contributor Author

nvb commented Mar 21, 2018

The version roachtest finally passed! TFTR @bdarnell.

@nvb nvb merged commit 5029c38 into cockroachdb:master Mar 21, 2018
@nvb nvb deleted the nvanbenschoten/ignoreChecksum branch March 21, 2018 23:48
@madelynnblue
Copy link
Copy Markdown
Contributor

Confirmed that this fixes the original issue (without the fix to IMPORT).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

sql: duplicate key value on non-unique index storage: conditional puts do not respect the optional nature of value checksums

4 participants