Skip to content

storage: adopt util/quotapool#39119

Merged
craig[bot] merged 2 commits intocockroachdb:masterfrom
ajwerner:ajwerner/quota-pool-adoption
Aug 13, 2019
Merged

storage: adopt util/quotapool#39119
craig[bot] merged 2 commits intocockroachdb:masterfrom
ajwerner:ajwerner/quota-pool-adoption

Conversation

@ajwerner
Copy link
Copy Markdown
Contributor

Release note: None

@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

@ajwerner ajwerner force-pushed the ajwerner/quota-pool-adoption branch from c3282ae to af75d11 Compare August 2, 2019 21:31
@ajwerner ajwerner requested a review from andreimatei August 2, 2019 21:31
@ajwerner ajwerner marked this pull request as ready for review August 2, 2019 21:32
@ajwerner ajwerner requested a review from a team August 2, 2019 21:32
@ajwerner ajwerner force-pushed the ajwerner/quota-pool-adoption branch 6 times, most recently from dd4e8c8 to 7e65960 Compare August 6, 2019 16:13
@ajwerner ajwerner requested a review from nvb August 6, 2019 16:37
@ajwerner
Copy link
Copy Markdown
Contributor Author

ajwerner commented Aug 6, 2019

@andreimatei hold off on reviewing this guy. Nathan has more context and we just talked about another change we want to make here.

@ajwerner ajwerner force-pushed the ajwerner/quota-pool-adoption branch from 7e65960 to 42f7d15 Compare August 6, 2019 17:09
Copy link
Copy Markdown
Contributor Author

@ajwerner ajwerner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @andreimatei and @nvanbenschoten)


pkg/storage/replica.go, line 295 at r2 (raw file):

		//
		// The *ProposalData in the map are "owned" by it. Elements from the
		// map must only be referenced while the Replica.raftMu is held, except

@nvanbenschoten and I spoke offline about this. It intended to say the Replica.mu but we added further synchronization on the ctx field in #38343. I added some more comments but I'm not sure they're super clear. I'm going to work on moving the proposals map down below the raftMu instead.

@ajwerner ajwerner force-pushed the ajwerner/quota-pool-adoption branch 4 times, most recently from 0fd10eb to 6a7bd43 Compare August 7, 2019 16:04
@ajwerner
Copy link
Copy Markdown
Contributor Author

ajwerner commented Aug 7, 2019

This is RFAL

@ajwerner ajwerner force-pushed the ajwerner/quota-pool-adoption branch 2 times, most recently from d08ec0a to 8e7bb11 Compare August 8, 2019 13:52
Copy link
Copy Markdown
Contributor

@nvb nvb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm: very nice!

Reviewed 3 of 12 files at r1, 10 of 10 files at r3.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @ajwerner and @andreimatei)


pkg/storage/replica_destroy.go, line 157 at r3 (raw file):

// It requires that both mu and raftMu are held.
func (r *Replica) cancelPendingCommandsLocked() {
	r.raftMu.AssertHeld()

nit: assert in the same order that you do above.


pkg/storage/replica_proposal_quota.go, line 244 at r3 (raw file):

		var toRelease *quotapool.IntAlloc
		for _, rel := range r.mu.quotaReleaseQueue[:numReleases] {
			if rel == nil || !rel.From(r.mu.proposalQuota) {

Add a comment about when !rel.From(r.mu.proposalQuota) would evaluate to true.


pkg/storage/replica_proposal_quota.go, line 250 at r3 (raw file):

				toRelease = rel
			} else {
				toRelease.Merge(rel)

Is this more efficient than releasing each IntAlloc one-by-one?

Copy link
Copy Markdown
Contributor Author

@ajwerner ajwerner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TFTR! I reworked the release logic into the quotapool package.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @andreimatei and @nvanbenschoten)


pkg/storage/replica_destroy.go, line 157 at r3 (raw file):

Previously, nvanbenschoten (Nathan VanBenschoten) wrote…

nit: assert in the same order that you do above.

Switched the other one.


pkg/storage/replica_proposal_quota.go, line 244 at r3 (raw file):

Previously, nvanbenschoten (Nathan VanBenschoten) wrote…

Add a comment about when !rel.From(r.mu.proposalQuota) would evaluate to true.

Added an NB but also moved this logic into the quotapool package as a separate commit.


pkg/storage/replica_proposal_quota.go, line 250 at r3 (raw file):

Previously, nvanbenschoten (Nathan VanBenschoten) wrote…

Is this more efficient than releasing each IntAlloc one-by-one?

Yes. Merge updates the value in toRelease and puts rel back into the underlying sync.Pool without any synchronization. Release needs to synchronize on the Pool to check if somebody needs to get notified.


pkg/storage/replica_proposal_quota.go, line 141 at r5 (raw file):

			// (if any) and release the quotaReleaseQueue so its allocs are pooled.
			r.mu.proposalQuota.Close("leader change")
			r.mu.proposalQuota.Release(r.mu.quotaReleaseQueue...)

I had forgotten to release this before.

@ajwerner ajwerner force-pushed the ajwerner/quota-pool-adoption branch from af42a40 to b2506df Compare August 13, 2019 06:24
Copy link
Copy Markdown
Contributor

@nvb nvb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 1 of 13 files at r4, 12 of 12 files at r6, 11 of 11 files at r7.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @ajwerner and @andreimatei)


pkg/storage/replica_proposal_quota.go, line 250 at r3 (raw file):

Previously, ajwerner wrote…

Yes. Merge updates the value in toRelease and puts rel back into the underlying sync.Pool without any synchronization. Release needs to synchronize on the Pool to check if somebody needs to get notified.

👍 thanks for encapsulating this concern.


pkg/storage/replica_proposal_quota.go, line 141 at r5 (raw file):

Previously, ajwerner wrote…

I had forgotten to release this before.

Good catch. Did you hit an assertion or anything?


pkg/util/quotapool/intpool_test.go, line 473 at r6 (raw file):

}

// TestIntpoolRelease tests the Release method of intpool to ensure that it releases

Is there more to this or is it just missing a period?


pkg/util/quotapool/intpool_test.go, line 507 at r6 (raw file):

			pools := makePools()
			allocs := make([]*quotapool.IntAlloc, len(c.toAcquire))
			for i, acq := range c.toAcquire {

Let's require that the fields in acq are within their numPools and capacity limits.

Andrew Werner added 2 commits August 13, 2019 10:45
util/quotapool is a generalization and optimization of the excised quotaPool.
Its somewhat different interface requires some code changes, specifically
around synchronization of modifications to the `ProposalData` struct.
Fortunately these changes combined with the logic changes in cockroachdb#39135
leave us with proposal quota tracking which never double frees and exactly
tracks outstanding proposals (modulo proposals which commit but are
re-proposed).

Release note: None
@ajwerner ajwerner force-pushed the ajwerner/quota-pool-adoption branch from b2506df to 0013508 Compare August 13, 2019 14:45
Copy link
Copy Markdown
Contributor Author

@ajwerner ajwerner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TFTR

bors r=nvanbenschoten

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @andreimatei and @nvanbenschoten)


pkg/storage/replica_proposal_quota.go, line 141 at r5 (raw file):

Previously, nvanbenschoten (Nathan VanBenschoten) wrote…

Good catch. Did you hit an assertion or anything?

No. It wasn't a correctness issue, just weren't pooling.


pkg/util/quotapool/intpool_test.go, line 473 at r6 (raw file):

Previously, nvanbenschoten (Nathan VanBenschoten) wrote…

Is there more to this or is it just missing a period?

Finished the sentence.


pkg/util/quotapool/intpool_test.go, line 507 at r6 (raw file):

Previously, nvanbenschoten (Nathan VanBenschoten) wrote…

Let's require that the fields in acq are within their numPools and capacity limits.

It will panic if it's outside of numPools. Requests above capacity are truncated so it's not technically a problem but added an assertion anyway.

craig bot pushed a commit that referenced this pull request Aug 13, 2019
39119: storage: adopt util/quotapool r=nvanbenschoten a=ajwerner

Release note: None

Co-authored-by: Andrew Werner <ajwerner@cockroachlabs.com>
Copy link
Copy Markdown
Contributor

@nvb nvb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 12 files at r8, 11 of 11 files at r9.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @andreimatei)

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Aug 13, 2019

Build succeeded

@craig craig bot merged commit 0013508 into cockroachdb:master Aug 13, 2019
craig bot pushed a commit that referenced this pull request Aug 13, 2019
39609: syncutil: add AssertRHeld methods to RWMutex implementations r=nvanbenschoten a=nvanbenschoten

This mirrors the AssertHeld methods we have on these mutexes and extends them for read locks. These serve as both a useful assertion and good documentation (because they compile away for real builds).

#39119 reminded me that I had been sitting on this commit since June as part of a larger change that won't be making it in any time soon.

Release note: None

Co-authored-by: Nathan VanBenschoten <nvanbenschoten@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants