Skip to content

kv: declare write access to AbortSpan on all aborting EndTxn reqs#48245

Merged
craig[bot] merged 1 commit intocockroachdb:masterfrom
nvb:nvanbenschoten/fix48046
Apr 30, 2020
Merged

kv: declare write access to AbortSpan on all aborting EndTxn reqs#48245
craig[bot] merged 1 commit intocockroachdb:masterfrom
nvb:nvanbenschoten/fix48046

Conversation

@nvb
Copy link
Copy Markdown
Contributor

@nvb nvb commented Apr 30, 2020

Fixes #43707.
Fixes #48046.
Fixes #48189.

Part of the change made by #42765 was to clear AbortSpan entries on
non-poisoning, aborting EndTxn requests. Specifically, this change was
made in 1328787. The change forgot to update the corresponding span
declaration logic to reflect the fact that we were now writing to the
AbortSpan in cases where we previously weren't.

This was triggering an assertion in race builds that tried to catch this
kind of undeclared span access. The assertion failure was very rare
because it required the following conditions to all be met:

  1. running a test with the race detector enabled
  2. a txn (A) must have been aborted by another txn (B)
  3. txn B must have cleared an intent on txn A's transaction record range
  4. txn A must have noticed and issued a non-poisoning EndTxn(ABORT)

We should backport this when we get a change (once v20.1.0 has
stabilized), but I don't expect that this could actually cause any
issues. The AbortSpan update was strictly a matter of performance and we
should never be racing with another request that is trying to read the
same AbortSpan entry.

Fixes cockroachdb#43707.
Fixes cockroachdb#48046.
Fixes cockroachdb#48189.

Part of the change made by cockroachdb#42765 was to clear AbortSpan entries on
non-poisoning, aborting EndTxn requests. Specifically, this change was
made in 1328787. The change forgot to update the corresponding span
declaration logic to reflect the fact that we were now writing to the
AbortSpan in cases where we previously weren't.

This was triggering an assertion in race builds that tried to catch this
kind of undeclared span access. The assertion failure was very rare
because it required the following conditions to all be met:
1. running a test with the race detector enabled
2. a txn (A) must have been aborted by another txn (B)
3. txn B must have cleared an intent on txn A's transaction record range
4. txn A must have noticed and issued a non-poisoning EndTxn(ABORT)

We should backport this when we get a change (once v20.1.0 has
stabilized), but I don't expect that this could actually cause any
issues. The AbortSpan update was strictly a matter of performance and we
should never be racing with another request that is trying to read the
same AbortSpan entry.
@nvb nvb requested a review from andreimatei April 30, 2020 18:26
@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

@cockroachdb cockroachdb deleted a comment from blathers-crl bot Apr 30, 2020
Copy link
Copy Markdown
Contributor

@andreimatei andreimatei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Also fixes #47958

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @andreimatei)

@nvb
Copy link
Copy Markdown
Contributor Author

nvb commented Apr 30, 2020

bors r+

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Apr 30, 2020

Build succeeded

@craig craig bot merged commit 237ea24 into cockroachdb:master Apr 30, 2020
andreimatei added a commit to andreimatei/cockroach that referenced this pull request May 1, 2020
We had an assertion that only a TransactionAbortedError can carry an
ABORTED transaction. This assertion seems pretty dubious though - we've
just seen failed rollbacks trigger it. While that was a bug - and in
fact we caught it due to this assertion (see cockroachdb#48245), it seems hard to
ensure that rollbacks will not fail in the future.

Release note: None
andreimatei added a commit to andreimatei/cockroach that referenced this pull request May 4, 2020
We had an assertion that only a TransactionAbortedError can carry an
ABORTED transaction. This assertion seems pretty dubious though - we've
just seen failed rollbacks trigger it. While that was a bug - and in
fact we caught it due to this assertion (see cockroachdb#48245), it seems hard to
ensure that rollbacks will not fail in the future.

Release note: None
@nvb nvb deleted the nvanbenschoten/fix48046 branch May 6, 2020 18:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ccl/importccl: TestImportPgDump failed kv/kvnemesis: TestKVNemesisSingleNode failed ccl/partitionccl: TestRepartitioning failed under stress

3 participants