Skip to content

liveness: remove a possible data race#56899

Merged
craig[bot] merged 1 commit intocockroachdb:masterfrom
tbg:remove-race
Nov 25, 2020
Merged

liveness: remove a possible data race#56899
craig[bot] merged 1 commit intocockroachdb:masterfrom
tbg:remove-race

Conversation

@tbg
Copy link
Copy Markdown
Member

@tbg tbg commented Nov 19, 2020

This was caught by our grpc race provoking transport while stressing
PR #56812.

Release note: None

This was caught by our grpc race provoking transport while stressing
PR cockroachdb#56812.

Release note: None
@tbg tbg requested a review from irfansharif November 19, 2020 12:54
@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

Copy link
Copy Markdown

@lunevalex lunevalex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for fixing.

Copy link
Copy Markdown
Contributor

@irfansharif irfansharif left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @irfansharif and @tbg)


pkg/kv/kvserver/liveness/liveness.go, line 1265 at r1 (raw file):

		b := txn.NewBatch()
		key := keys.NodeLivenessKey(update.newLiveness.NodeID)
		if err := v.SetProto(&update.newLiveness); err != nil {

I'm not really sure I understand what's going on here. We're racing with GRPCTransportFactory under transport_race.go, and I'm not sure what that means. What exactly is holding onto v here? My reading of this, and the race in #56812, is that the GRPCTransportFactory guy is busy reading through all batch requests to "to catch any mutations of a batch passed to the transport. The deal is that batches passed to the transport are immutable the server is not allowed to mutate anything and this transport makes sure they don't."

But we're still doing that with v.SetProto. So I think this "race" (which is safe I think, we're not mutating the batch after having sent it to the transport) still exists. I think what we want to do instead is call SetProto outside the Txn closure.

@tbg tbg requested a review from irfansharif November 24, 2020 08:02
Copy link
Copy Markdown
Member Author

@tbg tbg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @irfansharif)


pkg/kv/kvserver/liveness/liveness.go, line 1265 at r1 (raw file):

Previously, irfansharif (irfan sharif) wrote…

I'm not really sure I understand what's going on here. We're racing with GRPCTransportFactory under transport_race.go, and I'm not sure what that means. What exactly is holding onto v here? My reading of this, and the race in #56812, is that the GRPCTransportFactory guy is busy reading through all batch requests to "to catch any mutations of a batch passed to the transport. The deal is that batches passed to the transport are immutable the server is not allowed to mutate anything and this transport makes sure they don't."

But we're still doing that with v.SetProto. So I think this "race" (which is safe I think, we're not mutating the batch after having sent it to the transport) still exists. I think what we want to do instead is call SetProto outside the Txn closure.

I'm not sure I follow. This transport mechanism is ensuring that anything passed to gRPC is immutable. We are allocating a new value here and calling .SetProto on it before it's handed to anyone, and it is immutable from that point on. What am I missing?

@tbg
Copy link
Copy Markdown
Member Author

tbg commented Nov 25, 2020

bors r=lunevalex

Irfan lmk if there's something wrong still and I'll follow up.

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Nov 25, 2020

Build succeeded:

@craig craig bot merged commit f1ac4ef into cockroachdb:master Nov 25, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants