The Diagnostics Server IPC thread on Windows blocks in IpcStream::DiagnosticsIpc::Accept during shutdown. The blocking call resulting in the shutdown delay is located here:

https://github.com/dotnet/coreclr/blob/b388f6cd87d87f4a07fe966aaa1bc92f245165d9/src/debug/debug-pal/win/diagnosticsipc.cpp#L64