Skip to content

Fix race conditions in attribute asserts#52486

Merged
jaredpar merged 4 commits intodotnet:mainfrom
jaredpar:race
Apr 8, 2021
Merged

Fix race conditions in attribute asserts#52486
jaredpar merged 4 commits intodotnet:mainfrom
jaredpar:race

Conversation

@jaredpar
Copy link
Copy Markdown
Member

@jaredpar jaredpar commented Apr 8, 2021

There are asserts in our CustomAttributeBag to verify we maintain the
invariant that early decoding occurs before the full decoding. The
asserts though done in the getters and take the following form:

bool earlyComplete = IsPartComplete(CustomAttributeBagCompletionPart.EarlyDecodedWellKnownAttributeData);
// If late attributes are complete, early attributes must also be complete
Debug.Assert(!IsPartComplete(CustomAttributeBagCompletionPart.DecodedWellKnownAttributeData) || earlyComplete);
return earlyComplete;

This pattern is subject to race conditions. Consider the case where the
bool earlyComplete statement runs to completion and returns false. Then
another thread swaps in and completes both early and full decoding. At
that point the original thread resumes and the Debug.Assert will fail
yet no invariant has been violated.

Moved the Debug.Assert into the setters where we can reliably test the
state invariants.

closes #52372
related to #52368

There are asserts in our `CustomAttributeBag` to verify we maintain the
invariant that early decoding occurs before the full decoding. The
asserts though done in the getters and take the following form:

```cs
bool earlyComplete = IsPartComplete(CustomAttributeBagCompletionPart.EarlyDecodedWellKnownAttributeData);
// If late attributes are complete, early attributes must also be complete
Debug.Assert(!IsPartComplete(CustomAttributeBagCompletionPart.DecodedWellKnownAttributeData) || earlyComplete);
return earlyComplete;
```

This pattern is subject to race conditions. Consider the case where the
`bool earlyComplete` statement runs to completion and returns `false`. Then
another thread swaps in and completes both early and full decoding. At
that point the original thread resumes and the `Debug.Assert` will fail
yet no invariant has been violated.

Moved the `Debug.Assert` into the setters where we can reliably test the
state invariants.

closes dotnet#52372
related to 52368
@jaredpar jaredpar requested a review from a team as a code owner April 8, 2021 16:18
@jaredpar
Copy link
Copy Markdown
Member Author

jaredpar commented Apr 8, 2021

@dotnet/roslyn-compiler PTAL

@jaredpar jaredpar marked this pull request as draft April 8, 2021 16:48
Copy link
Copy Markdown
Contributor

@AlekseyTs AlekseyTs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM (commit 1)

@AlekseyTs
Copy link
Copy Markdown
Contributor

@jaredpar It looks like bootstrap build is broken. Could it be related to the changes around the asserts?

@jaredpar
Copy link
Copy Markdown
Member Author

jaredpar commented Apr 8, 2021

@AlekseyTs

The tests were failing due to the asserts I added. I had to remove one because it's a fundamentally racy scenario. It's not really possible to assert full decoding hasn't completed before noting early decoding has completed. Another thread can always interrupt and do an early + full decode. Hence I had to remove that. I left the assert that we verify that early decoding is done before full decoding. That is safe.

The bootstrap seems to be failing for the existing issue around the VB compiler crashing that I've been looking into. It's been tough to track down because the compiler is apparently exiting with no information. Been working with Sam on how to best track that down.

@jaredpar jaredpar marked this pull request as ready for review April 8, 2021 17:15
// Early decode must complete before full decode
Debug.Assert(!IsPartComplete(CustomAttributeBagCompletionPart.DecodedWellKnownAttributeData));
var setOnOurThread = Interlocked.CompareExchange(ref _earlyDecodedWellKnownAttributeData, data, null) == null;
NotePartComplete(CustomAttributeBagCompletionPart.EarlyDecodedWellKnownAttributeData);
Copy link
Copy Markdown
Contributor

@AlekseyTs AlekseyTs Apr 8, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Debug.Assert(!IsPartComplete(CustomAttributeBagCompletionPart.DecodedWellKnownAttributeData)); [](start = 12, length = 94)

Perhaps instead of removing the assert it could be changed to:

Debug.Assert(!IsPartComplete(CustomAttributeBagCompletionPart.DecodedWellKnownAttributeData) || IsPartComplete(CustomAttributeBagCompletionPart.EarlyDecodedWellKnownAttributeData));
``` #Closed

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good suggestion. I think I've convinced myself that this is safe 😄

I was concerned about potential read reordering issues but these are just loading the same block of memory and the state changes are atomic on that same piece of memory. Hence this should be robust to that type of problem.

@jaredpar jaredpar requested a review from a team as a code owner April 8, 2021 19:50
@jaredpar jaredpar enabled auto-merge (squash) April 8, 2021 19:50
Comment thread eng/test-rebuild.ps1
# Rebuilds with compilation errors
# Rebuilds with missing references
# Rebuilds with other issues
" --exclude net472\Microsoft.CodeAnalysis.EditorFeatures2.UnitTests.dll" +
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was this change necessitated by other changes in the PR?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, I think there is a flaky issue there. I saw it pop up on a few other PRs. I'm looking at the diff locally right now to seew hat's going on.

@jaredpar jaredpar merged commit 491ae81 into dotnet:main Apr 8, 2021
@ghost ghost added this to the Next milestone Apr 8, 2021
@jaredpar jaredpar deleted the race branch April 8, 2021 21:40
@dibarbet dibarbet modified the milestones: Next, 16.10.P3 Apr 26, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

NullableReferenceTypesTests.NotNullWhenFalse_WithNullLiteral unit test sometimes fails with an assert that might indicate a real product issue

4 participants