Skip to content

kvserver/concurrency: allow lock acquisition out of seq num order#46019

Merged
craig[bot] merged 1 commit intocockroachdb:masterfrom
nvb:nvanbenschoten/insertSeqNum
Mar 13, 2020
Merged

kvserver/concurrency: allow lock acquisition out of seq num order#46019
craig[bot] merged 1 commit intocockroachdb:masterfrom
nvb:nvanbenschoten/insertSeqNum

Conversation

@nvb
Copy link
Copy Markdown
Contributor

@nvb nvb commented Mar 12, 2020

This commit updates the lock-table to allow lock acquisition at
a sequence number below but not contained within an existing
lock's sequence number history. This is not possible to exercise
when locks are only added to the lock-table using AcquireLock,
but is possible when a later sequence number is added to the
lock=table through AddDiscoveredLock before the lock-table is
notified of the corresponding sequence of lock acquisitions.

This is possible because we do not immediately add replicated
locks to the lock-table. This means that a txn might acquire a
lock at sequence 2 and 3 with neither being added to the lock-table.
A second txn might then "discover" the lock and add it at seq 3
to the lock-table. If the first txn was to then replay its batch
with sequence 2 and 3, it would have previously hit the assertion.
I saw this while stressing kvnemesis.

Release justification: low risk bug fix that prevents a fatal server crash

This commit updates the lock-table to allow lock acquisition at
a sequence number below but not contained within an existing
lock's sequence number history. This is not possible to exercise
when locks are only added to the lock-table using AcquireLock,
but is possible when a later sequence number is added to the
lock=table through AddDiscoveredLock before the lock-table is
notified of the corresponding sequence of lock acquisitions.

This is possible because we do not immediately add replicated
locks to the lock-table. This means that a txn might acquire a
lock at sequence 2 and 3 with neither being added to the lock-table.
A second txn might then "discover" the lock and add it at seq 3
to the lock-table. If the first txn was to then replay its batch
with sequence 2 and 3, it would have previously hit the assertion.
I saw this while stressing kvnemesis.

Release justification: low risk bug fix
@nvb nvb requested a review from sumeerbhola March 12, 2020 03:04
@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

Copy link
Copy Markdown
Contributor

@ajwerner ajwerner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 2 of 2 files at r1.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @sumeerbhola)

@nvb
Copy link
Copy Markdown
Contributor Author

nvb commented Mar 13, 2020

TFTR!

bors r+

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Mar 13, 2020

Build failed

Copy link
Copy Markdown
Collaborator

@sumeerbhola sumeerbhola left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This means that a txn might acquire a
lock at sequence 2 and 3 with neither being added to the lock-table.

Is this also triggered if 2 was replicated and 3 was unreplicated, so due to the lack of contention only 3 was known to the lockTable and then the replay added 2?

:lgtm:

Reviewed 2 of 2 files at r1.
Reviewable status: :shipit: complete! 2 of 0 LGTMs obtained

@nvb
Copy link
Copy Markdown
Contributor Author

nvb commented Mar 13, 2020

Is this also triggered if 2 was replicated and 3 was unreplicated, so due to the lack of contention only 3 was known to the lockTable and then the replay added 2?

I don't think so, but only because of how we store replicated and unreplicated locks with their own sequence histories each.

Lint failed with:

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory

bors r+

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Mar 13, 2020

Build succeeded

@craig craig bot merged commit f3b2175 into cockroachdb:master Mar 13, 2020
@nvb nvb deleted the nvanbenschoten/insertSeqNum branch March 16, 2020 16:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants