Skip to content

storage: don't panic on unknown (batch) request#21466

Merged
tbg merged 1 commit intocockroachdb:masterfrom
tbg:unknown-requests
Jan 16, 2018
Merged

storage: don't panic on unknown (batch) request#21466
tbg merged 1 commit intocockroachdb:masterfrom
tbg:unknown-requests

Conversation

@tbg
Copy link
Copy Markdown
Member

@tbg tbg commented Jan 16, 2018

When a new BatchRequest type is introduced, we have to be careful to
not send it to older versions of CockroachDB because the new request
will simply be dropped when unmarshaling the proto. This leads to a
ba.Requests slice containing zero values, which leads to panics.

This commit adds a guard against unknown requests into (*Node).Batch
so that beginning in v2.0, it is less dramatic to forget the correct
cluster migration which gates use of new request types properly.
(I am not suggesting leaving out these migrations when this commit
has matured).

See:
#21345 (comment).

Release note: None

@tbg tbg requested a review from a team January 16, 2018 15:24
@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

@tbg tbg requested a review from bdarnell January 16, 2018 15:24
@tbg
Copy link
Copy Markdown
Member Author

tbg commented Jan 16, 2018

NB: I'm planning to cherry-pick this to v1.1., though I can be convinced otherwise.

@bdarnell
Copy link
Copy Markdown
Contributor

:lgtm:

I'm -1 on cherry-picking to 1.1. If we make sure we have migrations for everything in 2.0, it's unnecessary. We currently allow upgrading from any 1.1.x release to 2.0.y; I'd rather not introduce a requirement that the cluster be upgraded to at least 1.1.5 before 2.0 unless we have to.


Reviewed 4 of 4 files at r1.
Review status: 3 of 4 files reviewed at latest revision, 2 unresolved discussions.


pkg/acceptance/version_upgrade_test.go, line 70 at r1 (raw file):

	// The answer is "not well". v1.1 has the same problem except in late patch
	// releases. v2.0 does not have this issue, but it should be tested once (if)
	// we introduce a new command to v2.1 by adapting the below code. See

So the idea behind this block is that the first new command in 2.1 should uncomment the ba.Add line with the new command to provide a test that it doesn't crash the old node?


pkg/server/node.go, line 835 at r1 (raw file):

	for _, req := range reqs {
		if req.GetValue() == nil {
			return log.Safe("unknown request in batch")

I think it would be best if this was a roachpb.Error so we can cleanly catch it on the client side (for example with RecomputeStats, we could simply fail silently whenever we get the old-version error)


Comments from Reviewable

@tbg tbg force-pushed the unknown-requests branch from 0f045c7 to 9be1010 Compare January 16, 2018 16:39
When a new BatchRequest type is introduced, we have to be careful to
not send it to older versions of CockroachDB because the new request
will simply be dropped when unmarshaling the proto. This leads to a
`ba.Requests` slice containing zero values, which leads to panics.

This commit adds a guard against unknown requests into `(*Node).Batch`
so that beginning in v2.0, it is less dramatic to forget the correct
cluster migration which gates use of new request types properly.
(I am not suggesting leaving out these migrations when this commit
has matured).

See:
cockroachdb#21345 (comment).

Release note: None
@tbg tbg force-pushed the unknown-requests branch from 9be1010 to daaa9b2 Compare January 16, 2018 16:44
@tbg
Copy link
Copy Markdown
Member Author

tbg commented Jan 16, 2018

Agreed. I was suggesting the cherry-pick mostly to be able to cash in on the ba.Add in the acceptance test earlier, but that has been removed anyway.


Review status: 0 of 6 files reviewed at latest revision, 2 unresolved discussions.


pkg/server/node.go, line 835 at r1 (raw file):

Previously, bdarnell (Ben Darnell) wrote…

I think it would be best if this was a roachpb.Error so we can cleanly catch it on the client side (for example with RecomputeStats, we could simply fail silently whenever we get the old-version error)

Good idea, done.


pkg/acceptance/version_upgrade_test.go, line 70 at r1 (raw file):

Previously, bdarnell (Ben Darnell) wrote…

So the idea behind this block is that the first new command in 2.1 should uncomment the ba.Add line with the new command to provide a test that it doesn't crash the old node?

Yeah. I removed this though, doesn't seem to add anything given TestNodeSendUnknownBatchRequest.


Comments from Reviewable

@bdarnell
Copy link
Copy Markdown
Contributor

LGTM


Review status: 0 of 6 files reviewed at latest revision, 2 unresolved discussions, some commit checks pending.


Comments from Reviewable

@tbg tbg merged commit b8a1bb5 into cockroachdb:master Jan 16, 2018
@tbg tbg deleted the unknown-requests branch January 16, 2018 17:41
@tbg tbg restored the unknown-requests branch April 16, 2018 15:28
@tbg tbg deleted the unknown-requests branch May 13, 2018 11:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants