Skip to content

kvserver: take out additional readOnlyCmdMu locks#64378

Merged
craig[bot] merged 1 commit intocockroachdb:masterfrom
erikgrinaker:replica-removal-readonlymu
Apr 30, 2021
Merged

kvserver: take out additional readOnlyCmdMu locks#64378
craig[bot] merged 1 commit intocockroachdb:masterfrom
erikgrinaker:replica-removal-readonlymu

Conversation

@erikgrinaker
Copy link
Copy Markdown
Contributor

@erikgrinaker erikgrinaker commented Apr 29, 2021

Follow-up for #64324 that takes out readOnlyCmdMu to synchronize
read-only requests with replica removals in a couple of additional
spots. We now take them out at all destroyStatus.Set() sites, even
though they may not all strictly need them.

Also fixes lock ordering wrt. Store.mu in tryGetOrCreateReplica.

This change has already been included in backports for #64324.

Release note: None

@erikgrinaker erikgrinaker requested review from nvb and tbg April 29, 2021 12:46
@erikgrinaker erikgrinaker self-assigned this Apr 29, 2021
@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

Copy link
Copy Markdown
Member

@tbg tbg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 2 of 2 files at r1.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @erikgrinaker and @nvanbenschoten)


pkg/kv/kvserver/replica.go, line 1715 at r1 (raw file):

		}
		r.raftMu.Lock()
		r.readOnlyCmdMu.Lock()

This is the order described in the "lock order" comment on Store.mu, right?

Copy link
Copy Markdown
Contributor Author

@erikgrinaker erikgrinaker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @nvanbenschoten and @tbg)


pkg/kv/kvserver/replica.go, line 1715 at r1 (raw file):
Yes, but we may not be using the correct lock order elsewhere, so good looking out. It says:

Replica.raftMu < Replica.readOnlyCmdMu < Store.mu < Replica.mu

but I believe there are places where we take out Store.mu before readOnlyCmdMu, as I wasn't aware of the store ordering here. Let me do another pass for this.

Follow-up for cockroachdb#64324 that takes out `readOnlyCmdMu` to synchronize
read-only requests with replica removals in a couple of additional
spots. We now take them out at all `destroyStatus.Set()` sites, even
though they may not all strictly need them.

Also fixes lock ordering wrt. `Store.mu` in `tryGetOrCreateReplica`.

This change has already been included in backports for cockroachdb#64324.

Release note: None
@erikgrinaker erikgrinaker force-pushed the replica-removal-readonlymu branch from ed9ace4 to a86277d Compare April 29, 2021 16:59
Copy link
Copy Markdown
Contributor Author

@erikgrinaker erikgrinaker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @nvanbenschoten and @tbg)


pkg/kv/kvserver/replica.go, line 1715 at r1 (raw file):

Previously, erikgrinaker (Erik Grinaker) wrote…

Yes, but we may not be using the correct lock order elsewhere, so good looking out. It says:

Replica.raftMu < Replica.readOnlyCmdMu < Store.mu < Replica.mu

but I believe there are places where we take out Store.mu before readOnlyCmdMu, as I wasn't aware of the store ordering here. Let me do another pass for this.

Fixed lock ordering wrt. Store.mu during tryGetOrCreateReplica as well, PTAL. Couldn't see any other relevant sites. Note that this already unlocks Store.mu out of order, I'm assuming this is for good reason:

s.mu.Unlock() // NB: unlocking out of order

Copy link
Copy Markdown
Contributor

@nvb nvb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 2 of 2 files at r1, 1 of 1 files at r2.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @tbg)

@erikgrinaker
Copy link
Copy Markdown
Contributor Author

bors r=tbg,nvanbenschoten

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Apr 30, 2021

Build succeeded:

@craig craig bot merged commit 42989d0 into cockroachdb:master Apr 30, 2021
@erikgrinaker erikgrinaker deleted the replica-removal-readonlymu branch April 30, 2021 10:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants