Description

We noticed in our release builds that we were getting sporadic crashes with this stack:

Unhandled exception. System.NullReferenceException: Object reference not set to an instance of an object.
   at System.Runtime.CompilerServices.AsyncVoidMethodBuilder.NotifySynchronizationContextOfCompletion()
--- End of stack trace from previous location ---
   at System.Threading.Tasks.Task.<>c.<ThrowAsync>b__128_1(Object state)
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()

I've reduced it down to the code listed in the reproduction steps. It is critical to have the SynchronizationContext be non-null when entering the async void method to reproduce this issue.

I can provide a Time Travel Debug trace for the crash if desired, although it seems easy enough to reproduce that folks can capture their own.

Reproduction Steps

The top-level program below causes the crash when built for release (not debug). On my computer it's usually within seconds. Experiment with the SpinWait count if this doesn't work for you. The value of 25 used below tends to crash on my computer within seconds while a value of 1000 generally crashes between 20 seconds and several minutes. You may need a value in this range reproduce the crash quickly.

SynchronizationContext.SetSynchronizationContext(new());
while (true)
{
    Run();
    Thread.SpinWait(25);
}

async void Run()
{
    await Task.Yield();
    await Task.Yield();
}

Expected behavior

A compiled release build runs without crashing

Actual behavior

The compiled release build readily crashes with the stack:

System.NullReferenceException: Object reference not set to an instance of an object.
   at System.Runtime.CompilerServices.AsyncVoidMethodBuilder.NotifySynchronizationContextOfCompletion()
--- End of stack trace from previous location ---
   at System.Threading.Tasks.Task.<>c.<ThrowAsync>b__128_1(Object state)
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()
   at System.Threading.Thread.StartCallback()

Regression?

I tested on .NET 6.0.27 and the crash was present there as well, so it seems like this has been an issue for quite a while.

Known Workarounds

In our case we:

1. Set up a first chance exception handler to capture the initial NullReferenceException (not the rethrown one that causes the crash)
2. Identify our particular method that was compiling with AsyncVoidMethodBuilder
3. Changed the method to avoid that particular method builder
4. (optional) Wring hands over every other async void method that the compiler makes for our code

Since SynchronizationContext must be non-null to encounter this bug, another workaround would be to try to keep it null.

Configuration

• .NET 8.0.2, .NET 6.0.27
• Windows 11 23H2 (x64) and macOS 14.2.1 (ARM64)
• Appears to be specific to release builds

Other information

This seems related to the previously reported crashes in #40463 and #93969.

When I look at the crash in TTD, the thread originating the NullReferenceException is accessing memory that was very recently zeroed by another thread. I'm not sure how to get good symbols in WinDbg for this particular compiler generated code so I haven't been able to determine where in the C# code the memory is being zeroed.

The quickest fix that I see is to have the three callers of NotifySynchronizationContextOfCompletion pass in their already-checked field value. But, this only gets around the exception. I suspect the memory could still be zeroed too soon and thus NotifySynchronizationContextOfCompletion wouldn't be called when there had, in fact, been a SynchronizationContext assigned at one point.