storage: adopt util/quotapool#39119
Conversation
c3282ae to
af75d11
Compare
dd4e8c8 to
7e65960
Compare
|
@andreimatei hold off on reviewing this guy. Nathan has more context and we just talked about another change we want to make here. |
7e65960 to
42f7d15
Compare
ajwerner
left a comment
There was a problem hiding this comment.
Reviewable status:
complete! 0 of 0 LGTMs obtained (waiting on @andreimatei and @nvanbenschoten)
pkg/storage/replica.go, line 295 at r2 (raw file):
// // The *ProposalData in the map are "owned" by it. Elements from the // map must only be referenced while the Replica.raftMu is held, except
@nvanbenschoten and I spoke offline about this. It intended to say the Replica.mu but we added further synchronization on the ctx field in #38343. I added some more comments but I'm not sure they're super clear. I'm going to work on moving the proposals map down below the raftMu instead.
0fd10eb to
6a7bd43
Compare
|
This is RFAL |
d08ec0a to
8e7bb11
Compare
nvb
left a comment
There was a problem hiding this comment.
Reviewed 3 of 12 files at r1, 10 of 10 files at r3.
Reviewable status:complete! 1 of 0 LGTMs obtained (waiting on @ajwerner and @andreimatei)
pkg/storage/replica_destroy.go, line 157 at r3 (raw file):
// It requires that both mu and raftMu are held. func (r *Replica) cancelPendingCommandsLocked() { r.raftMu.AssertHeld()
nit: assert in the same order that you do above.
pkg/storage/replica_proposal_quota.go, line 244 at r3 (raw file):
var toRelease *quotapool.IntAlloc for _, rel := range r.mu.quotaReleaseQueue[:numReleases] { if rel == nil || !rel.From(r.mu.proposalQuota) {
Add a comment about when !rel.From(r.mu.proposalQuota) would evaluate to true.
pkg/storage/replica_proposal_quota.go, line 250 at r3 (raw file):
toRelease = rel } else { toRelease.Merge(rel)
Is this more efficient than releasing each IntAlloc one-by-one?
8e7bb11 to
af42a40
Compare
ajwerner
left a comment
There was a problem hiding this comment.
TFTR! I reworked the release logic into the quotapool package.
Reviewable status:
complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @andreimatei and @nvanbenschoten)
pkg/storage/replica_destroy.go, line 157 at r3 (raw file):
Previously, nvanbenschoten (Nathan VanBenschoten) wrote…
nit: assert in the same order that you do above.
Switched the other one.
pkg/storage/replica_proposal_quota.go, line 244 at r3 (raw file):
Previously, nvanbenschoten (Nathan VanBenschoten) wrote…
Add a comment about when
!rel.From(r.mu.proposalQuota)would evaluate to true.
Added an NB but also moved this logic into the quotapool package as a separate commit.
pkg/storage/replica_proposal_quota.go, line 250 at r3 (raw file):
Previously, nvanbenschoten (Nathan VanBenschoten) wrote…
Is this more efficient than releasing each
IntAllocone-by-one?
Yes. Merge updates the value in toRelease and puts rel back into the underlying sync.Pool without any synchronization. Release needs to synchronize on the Pool to check if somebody needs to get notified.
pkg/storage/replica_proposal_quota.go, line 141 at r5 (raw file):
// (if any) and release the quotaReleaseQueue so its allocs are pooled. r.mu.proposalQuota.Close("leader change") r.mu.proposalQuota.Release(r.mu.quotaReleaseQueue...)
I had forgotten to release this before.
af42a40 to
b2506df
Compare
nvb
left a comment
There was a problem hiding this comment.
Reviewed 1 of 13 files at r4, 12 of 12 files at r6, 11 of 11 files at r7.
Reviewable status:complete! 1 of 0 LGTMs obtained (waiting on @ajwerner and @andreimatei)
pkg/storage/replica_proposal_quota.go, line 250 at r3 (raw file):
Previously, ajwerner wrote…
Yes. Merge updates the value in toRelease and puts rel back into the underlying sync.Pool without any synchronization. Release needs to synchronize on the
Poolto check if somebody needs to get notified.
👍 thanks for encapsulating this concern.
pkg/storage/replica_proposal_quota.go, line 141 at r5 (raw file):
Previously, ajwerner wrote…
I had forgotten to release this before.
Good catch. Did you hit an assertion or anything?
pkg/util/quotapool/intpool_test.go, line 473 at r6 (raw file):
} // TestIntpoolRelease tests the Release method of intpool to ensure that it releases
Is there more to this or is it just missing a period?
pkg/util/quotapool/intpool_test.go, line 507 at r6 (raw file):
pools := makePools() allocs := make([]*quotapool.IntAlloc, len(c.toAcquire)) for i, acq := range c.toAcquire {
Let's require that the fields in acq are within their numPools and capacity limits.
util/quotapool is a generalization and optimization of the excised quotaPool. Its somewhat different interface requires some code changes, specifically around synchronization of modifications to the `ProposalData` struct. Fortunately these changes combined with the logic changes in cockroachdb#39135 leave us with proposal quota tracking which never double frees and exactly tracks outstanding proposals (modulo proposals which commit but are re-proposed). Release note: None
b2506df to
0013508
Compare
ajwerner
left a comment
There was a problem hiding this comment.
TFTR
bors r=nvanbenschoten
Reviewable status:
complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @andreimatei and @nvanbenschoten)
pkg/storage/replica_proposal_quota.go, line 141 at r5 (raw file):
Previously, nvanbenschoten (Nathan VanBenschoten) wrote…
Good catch. Did you hit an assertion or anything?
No. It wasn't a correctness issue, just weren't pooling.
pkg/util/quotapool/intpool_test.go, line 473 at r6 (raw file):
Previously, nvanbenschoten (Nathan VanBenschoten) wrote…
Is there more to this or is it just missing a period?
Finished the sentence.
pkg/util/quotapool/intpool_test.go, line 507 at r6 (raw file):
Previously, nvanbenschoten (Nathan VanBenschoten) wrote…
Let's require that the fields in
acqare within theirnumPoolsandcapacitylimits.
It will panic if it's outside of numPools. Requests above capacity are truncated so it's not technically a problem but added an assertion anyway.
nvb
left a comment
There was a problem hiding this comment.
Reviewed 1 of 12 files at r8, 11 of 11 files at r9.
Reviewable status:complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @andreimatei)
Build succeeded |
39609: syncutil: add AssertRHeld methods to RWMutex implementations r=nvanbenschoten a=nvanbenschoten This mirrors the AssertHeld methods we have on these mutexes and extends them for read locks. These serve as both a useful assertion and good documentation (because they compile away for real builds). #39119 reminded me that I had been sitting on this commit since June as part of a larger change that won't be making it in any time soon. Release note: None Co-authored-by: Nathan VanBenschoten <nvanbenschoten@gmail.com>
Release note: None