Skip to content

storage: don't require consensus for no-op requests#24345

Merged
craig[bot] merged 3 commits intocockroachdb:masterfrom
nvb:nvanbenschoten/proposeNoop
Mar 29, 2018
Merged

storage: don't require consensus for no-op requests#24345
craig[bot] merged 3 commits intocockroachdb:masterfrom
nvb:nvanbenschoten/proposeNoop

Conversation

@nvb
Copy link
Copy Markdown
Contributor

@nvb nvb commented Mar 29, 2018

Fixes #23942.

During evaluation, a request can result in having no effect for a variety
of reasons. For instance, a PushTxnRequest may arrive at a transaction
record to find that it is already ABORTED, in which case there's nothing
for it to do.

Before this change, we were still requiring that these empty results be
proposed through Raft. This change fixes this by detecting empty results
and bypassing consensus completely.

Release note (performance improvement): Write requests that result in
no-ops are no longer proposed through Raft.

@nvb nvb requested review from a team, bdarnell and tbg March 29, 2018 20:21
@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

@tbg
Copy link
Copy Markdown
Member

tbg commented Mar 29, 2018

:lgtm:


Reviewed 8 of 8 files at r1.
Review status: all files reviewed at latest revision, all discussions resolved, some commit checks pending.


pkg/storage/replica.go, line 2887 at r1 (raw file):

// evaluating it. The returned ProposalData is partially valid even
// on a non-nil *roachpb.Error and should be proposed through Raft
// if ProposalData.command is non-nil.

Perhaps take a moment to clarify that a non-nil ProposalData.command implies that no error was returned.

Ah, I see you have a comment to this effect in evaluateProposal. I suppose that's ok then.


pkg/storage/replica.go, line 2897 at r1 (raw file):

	res, needConsensus, pErr := r.evaluateProposal(ctx, idKey, ba, spans)

	// Fill out the results even if pErr != nil; we'll return the error below.

l


pkg/storage/replica.go, line 2990 at r1 (raw file):

	// following conditions is true:
	// 1. the request created a non-empty write batch.
	// 2. the request had an impact on the MVCCStats.

2 sounds like it would imply 1, though the stats fixing is likely the exception. Name that example, I think it's illuminating.


pkg/storage/replica.go, line 3157 at r1 (raw file):

			EndTxns: endTxns,
		}
		proposal.finishRaftApplication(pr)

Make sure to update the comment on finishRaftApplication to make sure it's clear that this isn't always called from the Raft goroutine. Or rename it appropriately.


pkg/storage/replica_test.go, line 8723 at r1 (raw file):

			name: "delete req",
			req:  deleteReq,
			// NB: a tombstone is written even if no value exists at the key.

Good to know.


pkg/storage/engine/batch.go, line 117 at r1 (raw file):
Use the comment from the interface:

// Empty returns whether the batch is empty or not.


Comments from Reviewable

I had a build fail with:

```
should omit type error from declaration of var QueryCanceledError; it will be inferred from the right-hand side
```

I'm not sure why this wasn't caught earlier.

Release note: None
@nvb nvb force-pushed the nvanbenschoten/proposeNoop branch from 30dda78 to cc0b1de Compare March 29, 2018 20:49
@nvb nvb requested review from a team March 29, 2018 20:49
@nvb
Copy link
Copy Markdown
Contributor Author

nvb commented Mar 29, 2018

TFTR!


Review status: 0 of 9 files reviewed at latest revision, 6 unresolved discussions.


pkg/storage/replica.go, line 2897 at r1 (raw file):

Previously, tschottdorf (Tobias Schottdorf) wrote…

l

Did you mean to say something here?


pkg/storage/replica.go, line 2990 at r1 (raw file):

Previously, tschottdorf (Tobias Schottdorf) wrote…

2 sounds like it would imply 1, though the stats fixing is likely the exception. Name that example, I think it's illuminating.

Done.


pkg/storage/replica.go, line 3157 at r1 (raw file):

Previously, tschottdorf (Tobias Schottdorf) wrote…

Make sure to update the comment on finishRaftApplication to make sure it's clear that this isn't always called from the Raft goroutine. Or rename it appropriately.

Done.


pkg/storage/replica_test.go, line 8723 at r1 (raw file):

Previously, tschottdorf (Tobias Schottdorf) wrote…

Good to know.

Yeah, I knew you'd like that one 😃


pkg/storage/engine/batch.go, line 117 at r1 (raw file):

Previously, tschottdorf (Tobias Schottdorf) wrote…

Use the comment from the interface:

// Empty returns whether the batch is empty or not.

This isn't actually an implementation of the interface, so I went with a comment that matched the other comments here.


Comments from Reviewable

@bdarnell
Copy link
Copy Markdown
Contributor

:lgtm:


Reviewed 1 of 9 files at r2, 5 of 8 files at r3, 3 of 3 files at r4.
Review status: all files reviewed at latest revision, 6 unresolved discussions, some commit checks pending.


Comments from Reviewable

@tbg
Copy link
Copy Markdown
Member

tbg commented Mar 29, 2018

Reviewed 9 of 9 files at r2, 8 of 8 files at r3, 3 of 3 files at r4.
Review status: all files reviewed at latest revision, 2 unresolved discussions, some commit checks failed.


pkg/storage/replica.go, line 2897 at r1 (raw file):

Previously, nvanbenschoten (Nathan VanBenschoten) wrote…

Did you mean to say something here?

No, my track pad is broken and randomly clicks and if that happens while I'm typing a comment I let reviewable do things I can't trace after.


pkg/storage/engine/batch.go, line 117 at r1 (raw file):

Previously, nvanbenschoten (Nathan VanBenschoten) wrote…

This isn't actually an implementation of the interface, so I went with a comment that matched the other comments here.

Comment here looks the same, but it's only a nit anyway. I mostly thought this sentence sounded odd.


Comments from Reviewable

nvb added 2 commits March 29, 2018 17:36
Fixes cockroachdb#23942.

During evaluation, a request can result in having no effect for a variety
of reasons. For instance, a PushTxnRequest may arrive at a transaction
record to find that it is already ABORTED, in which case there's nothing
for it to do.

Before this change, we were still requiring that these empty results be
proposed through Raft. This change fixes this by detecting empty results
and bypassing consensus completely.

Release note (performance improvement): Write requests that result in
no-ops are no longer proposed through Raft.
@nvb
Copy link
Copy Markdown
Contributor Author

nvb commented Mar 29, 2018

Review status: all files reviewed at latest revision, 2 unresolved discussions, some commit checks failed.


pkg/storage/engine/batch.go, line 117 at r1 (raw file):

Previously, tschottdorf (Tobias Schottdorf) wrote…

Comment here looks the same, but it's only a nit anyway. I mostly thought this sentence sounded odd.

Done.


Comments from Reviewable

@nvb nvb force-pushed the nvanbenschoten/proposeNoop branch from cc0b1de to feaee5c Compare March 29, 2018 21:36
@nvb
Copy link
Copy Markdown
Contributor Author

nvb commented Mar 29, 2018

bors r=tschottdorf

craig bot added a commit that referenced this pull request Mar 29, 2018
24309: optbuilder: Fix bugs with subqueries, DISTINCT, and ORDER BY r=RaduBerinde a=rytaft

This commit fixes several miscellaneous issues.

1. Subqueries with aggregation and without a `FROM` clause were
   causing the outer query to incorrectly act as if it were in
   a grouping context. This commit fixes the issue by making
   sure the scope of the subquery is different than the scope
   of the outer query.
2. Queries with `DISTINCT` and `ORDER BY` in which the `ORDER BY`
   was ordering on an expression that was also part of the
   `SELECT` list were failing. This commit fixes the issue by
   reusing the existing projection for the `ORDER BY` column.
3. Several redundant columns were previously being synthesized
   for identical expressions. This commit fixes the issue by
   reusing existing columns wherever possible.

Release note: None

24345: storage: don't require consensus for no-op requests r=tschottdorf a=nvanbenschoten

Fixes #23942.

During evaluation, a request can result in having no effect for a variety
of reasons. For instance, a PushTxnRequest may arrive at a transaction
record to find that it is already ABORTED, in which case there's nothing
for it to do.

Before this change, we were still requiring that these empty results be
proposed through Raft. This change fixes this by detecting empty results
and bypassing consensus completely.

Release note (performance improvement): Write requests that result in
no-ops are no longer proposed through Raft.
@craig
Copy link
Copy Markdown
Contributor

craig bot commented Mar 29, 2018

Build succeeded

@craig craig bot merged commit feaee5c into cockroachdb:master Mar 29, 2018
@nvb nvb deleted the nvanbenschoten/proposeNoop branch March 29, 2018 22:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

storage: no-op writes should not be proposed through Raft

4 participants