Skip to content

admission: squash data race in accessing token bucket#88279

Merged
craig[bot] merged 1 commit intocockroachdb:masterfrom
irfansharif:220920.elastic-data-race
Sep 20, 2022
Merged

admission: squash data race in accessing token bucket#88279
craig[bot] merged 1 commit intocockroachdb:masterfrom
irfansharif:220920.elastic-data-race

Conversation

@irfansharif
Copy link
Copy Markdown
Contributor

@irfansharif irfansharif commented Sep 20, 2022

Fixes #88269, fallout from #88031 -- sloppy assumption that the token bucket type is thread-safe. Instantly reproduced using:

dev test pkg/ccl/changefeedccl \
  -f=TestAlterChangefeedAddTarget/cloudstorage \
  --race --stress --timeout 1m

Release note: None
Release justification: Fixes data race in non-production code

@irfansharif irfansharif requested a review from a team September 20, 2022 19:52
@irfansharif irfansharif requested a review from a team as a code owner September 20, 2022 19:52
@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

Copy link
Copy Markdown
Collaborator

@sumeerbhola sumeerbhola left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 1 of 1 files at r1, all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @irfansharif)


pkg/util/admission/elastic_cpu_granter.go line 102 at r1 (raw file):

	ctx context.Context
	mu  struct {
		syncutil.Mutex

My understanding is that there is no lock ordering between this Mutex and the one in WorkQueue, since neither holds its own mutex when calling into the other. Is that correct?
Worth documenting this in a code comment, since this is different from the other granters: the granters that are used by GrantCoordinator don't have their own mutex, and rely on the one in GrantCoordinator, which is ordered before the one in WorkQueue, since requester.granted() is called while holding GrantCoordinator.mu.

Fixes cockroachdb#88269, fallout from cockroachdb#88031 -- sloppy assumption that the token
bucket type is thread-safe. Instantly reproduced using:

    dev test pkg/ccl/changefeedccl \
      -f=TestAlterChangefeedAddTarget/cloudstorage \
      --race --stress --timeout 1m

Release note: None
@irfansharif irfansharif force-pushed the 220920.elastic-data-race branch from 62f89d6 to 5862861 Compare September 20, 2022 21:01
Copy link
Copy Markdown
Contributor Author

@irfansharif irfansharif left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bors r+

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @sumeerbhola)


pkg/util/admission/elastic_cpu_granter.go line 102 at r1 (raw file):

Previously, sumeerbhola wrote…

My understanding is that there is no lock ordering between this Mutex and the one in WorkQueue, since neither holds its own mutex when calling into the other. Is that correct?
Worth documenting this in a code comment, since this is different from the other granters: the granters that are used by GrantCoordinator don't have their own mutex, and rely on the one in GrantCoordinator, which is ordered before the one in WorkQueue, since requester.granted() is called while holding GrantCoordinator.mu.

Done.

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Sep 20, 2022

Build failed (retrying...):

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Sep 20, 2022

Build succeeded:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

admission: data race under elasticCPUGranter

3 participants