Skip to content

Skip results transfer in MT mode to fix ResultsNodeId race#13417

Merged
JanProvaznik merged 4 commits intodotnet:mainfrom
JanProvaznik:fix/mt-results-transfer-race-13188
Mar 20, 2026
Merged

Skip results transfer in MT mode to fix ResultsNodeId race#13417
JanProvaznik merged 4 commits intodotnet:mainfrom
JanProvaznik:fix/mt-results-transfer-race-13188

Conversation

@JanProvaznik
Copy link
Copy Markdown
Member

Fix for #13188: MT mode ResultsNodeId race condition

Problem

In MT (multi-threaded) mode, builds intermittently crash with:
\
InternalErrorException: MSB0001: Internal MSBuild Error: Results for configuration X were not retrieved from node Y
\\

Root Cause

\BuildRequestConfiguration.ResultsNodeId\ is a single \int\ field shared by all in-proc nodes via the shared \ConfigCache. When multiple in-proc nodes simultaneously need a configuration's results, they race through \HandleRequestBlockedOnResultsTransfer, which unconditionally overwrites \ResultsNodeId\ to the requesting node's ID. The last writer wins, causing other waiters to find a mismatched value when they wake up, failing the assertion in \RequestBuilder.BuildProject().

Key Insight

In MT mode, the entire results transfer mechanism is unnecessary. All in-proc nodes share the same \ConfigCache\ and \ResultsCache\ by reference. When a config is built on any node, the results are immediately accessible to all other in-proc nodes through the shared cache. MT mode also disallows out-of-proc nodes (\Scheduler.cs\ sets OOP node count to 0 when \MultiThreaded=true), so there are no cross-process transfers to worry about.

Fix

Extracted the transfer-needed check into a testable \NeedsResultsTransfer\ method that returns \ alse\ when \MultiThreaded\ mode is active, skipping the entire transfer block.

Tests

  • Unit test (red/green validated): \NeedsResultsTransfer_SkipsInMultiThreadedMode\ — 6 Theory cases covering all MT/ST × node ID combinations. Confirmed: fails without fix, passes with fix.
  • Integration test: \MultiThreadedBuild_SharedConfiguration_DoesNotCrash\ — 8 parallel projects referencing same dependency with different target sets, forcing cache misses that exercise the transfer code path.

No ChangeWave Needed

This is a crash fix, not a behavioral change. The transfer was already a no-op in MT mode (copying data between shared references). Skipping it produces identical build results while preventing the race condition crash.

Fixes #13188

)

In MT mode, all in-proc nodes share the same ConfigCache and ResultsCache,
so results transfer between nodes is unnecessary. The transfer mechanism
races on the shared ResultsNodeId field when multiple nodes simultaneously
call HandleRequestBlockedOnResultsTransfer for the same configuration,
each overwriting ResultsNodeId with their own node ID. This causes
intermittent crashes with 'Results for configuration X were not retrieved
from node Y'.

Fix: extract the transfer-needed check into a testable NeedsResultsTransfer
method that returns false when MultiThreaded mode is active, skipping the
entire transfer block.

Tests:
- Unit test (red/green validated): NeedsResultsTransfer_SkipsInMultiThreadedMode
  with 6 Theory cases covering all MT/ST × node ID combinations
- Integration test: MultiThreadedBuild_SharedConfiguration_DoesNotCrash
  exercises the code path with 8 parallel projects and different target sets

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@JanProvaznik
Copy link
Copy Markdown
Member Author

expert reviewer passe, ready to review

@JanProvaznik JanProvaznik marked this pull request as ready for review March 19, 2026 15:19
Copilot AI review requested due to automatic review settings March 19, 2026 15:19
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes an intermittent crash in MSBuild -mt (multi-threaded, in-proc) builds by skipping the “results transfer” path when it’s unnecessary due to shared in-proc caches, eliminating a race on BuildRequestConfiguration.ResultsNodeId.

Changes:

  • Added RequestBuilder.NeedsResultsTransfer(...) and used it to bypass results-transfer blocking in MT mode.
  • Added unit coverage for NeedsResultsTransfer across MT/ST and node-id combinations.
  • Added an integration-style MT build test exercising shared configuration + differing target sets to prevent regressions of #13188.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
src/Build/BackEnd/Components/RequestBuilder/RequestBuilder.cs Introduces NeedsResultsTransfer and uses it to skip results transfer in MT mode.
src/Build.UnitTests/BackEnd/RequestBuilder_Tests.cs Adds a theory validating NeedsResultsTransfer behavior.
src/Build.UnitTests/BackEnd/BuildManager_Tests.cs Adds an MT integration test intended to reproduce/prevent the ResultsNodeId race crash.

You can also share your feedback on Copilot code review. Take the survey.

Comment thread src/Build.UnitTests/BackEnd/RequestBuilder_Tests.cs Outdated
Comment thread src/Build.UnitTests/BackEnd/RequestBuilder_Tests.cs
Comment thread src/Build.UnitTests/BackEnd/BuildManager_Tests.cs
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@JanProvaznik JanProvaznik requested a review from AR-May March 19, 2026 15:59
Comment thread src/Build/BackEnd/Components/RequestBuilder/RequestBuilder.cs
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@JanProvaznik JanProvaznik enabled auto-merge (squash) March 20, 2026 13:45
@JanProvaznik JanProvaznik merged commit 5071b11 into dotnet:main Mar 20, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Unhandled Exception]: -mt mode intermittent crash "Results for configuration X were not retrieved from node Y"

3 participants