[IBuildEngine callbacks] Stage 2: RequestCores/ReleaseCores by JanProvaznik · Pull Request #13306 · dotnet/msbuild

JanProvaznik · 2026-02-27T15:45:56Z

Context

Stage 2 of implementing IBuildEngine callback support in TaskHost. Stage 1 (#13149) established the packet infrastructure and implemented IsRunningMultipleNodes. This PR adds RequestCores/ReleaseCores (IBuildEngine9) using the same pattern.

Previously, tasks running in TaskHost that called RequestCores or ReleaseCores would get a NotImplementedException. This blocked resource-aware tasks from properly coordinating CPU usage when running out-of-process.

Partial fix for #12863

Changes Made

New Packets:

TaskHostCoresRequest (0x22) — single packet type for both operations, with IsRelease bool to distinguish RequestCores vs ReleaseCores
TaskHostCoresResponse (0x23) — returns GrantedCores (int) for RequestCores, 0 for ReleaseCores acknowledgment

OutOfProcTaskHostNode (TaskHost side):

Replace throw NotImplementedException() with SendCallbackRequestAndWaitForResponse<TaskHostCoresResponse>()
Add input validation (requestedCores > 0, coresToRelease > 0) matching in-process behavior
Gate behind CallbacksSupported — when disabled, RequestCores returns 0, ReleaseCores is no-op (both log MSB5022)
Register TaskHostCoresResponse in packet handler and route in HandlePacket switch

TaskHostTask (Worker node side):

Register TaskHostCoresRequest packet handler
HandleCoresRequest(): forwards RequestCores/ReleaseCores to the in-process IBuildEngine9
The real implicit-core/additional-core accounting happens in the in-process TaskHost class — the OOP node is a thin proxy

Callback Flow

sequenceDiagram
    participant Task as Task Thread<br/>(TaskHost)
    participant Main as Main Thread<br/>(TaskHost)
    participant Worker as TaskHostTask<br/>(Worker Node)

    Task->>Task: BuildEngine.RequestCores(N)
    Task->>Task: Create TaskHostCoresRequest(N, isRelease=false)
    Task->>Task: Store TCS in _pendingCallbackRequests[requestId]
    Task->>Worker: SendData(request)
    Task-->>Task: Block on TCS.Task

    Worker->>Worker: HandleCoresRequest(request)
    Worker->>Worker: granted = _buildEngine.RequestCores(N)
    Worker->>Main: SendData(TaskHostCoresResponse(granted))

    Main->>Main: HandleCallbackResponse(response)
    Main->>Main: _pendingCallbackRequests[requestId].SetResult(response)

    Task-->>Task: TCS unblocks
    Task->>Task: return response.GrantedCores

Testing

Packet serialization (4 tests): Round-trip for request (RequestCores/ReleaseCores variants) and response (granted/acknowledgment)
Integration (5 tests):
- RequestCores_WorksWithExplicitTaskHostFactory — verifies cores granted >= 1
- RequestAndReleaseCores_WorksWithExplicitTaskHostFactory — full request+release cycle
- RequestCores_WorksWhenAutoEjectedInMultiThreadedMode — MT auto-ejection to TaskHost
- RequestCores_LogsErrorWhenCallbacksNotSupported — MSB5022 fallback
- IsRunningMultipleNodes_LogsErrorWhenCallbacksNotSupported — Stage 1 fallback (unchanged)
E2E cross-runtime (1 test): NetTaskHost_CallbackRequestCoresTest via bootstrapped MSBuild
All 20 callback/serialization/lifecycle tests pass, 0 regressions

Documentation

Updated taskhost-threading.md with:

Supported callbacks table (IsRunningMultipleNodes, RequestCores, ReleaseCores)
Resource management flow description

Next Stages

Stage 3: BuildProjectFile/BuildProjectFilesInParallel
Stage 4: Yield/Reacquire

New IPC packet pair for forwarding IBuildEngine9.RequestCores() and ReleaseCores() from TaskHost to the owning worker node. TaskHostCoresRequest (0x22) uses a single packet type with an IsRelease bool to distinguish RequestCores from ReleaseCores — both operations share the same int parameter semantics (core count) and response shape (granted cores or acknowledgment). TaskHostCoresResponse (0x23) returns GrantedCores for RequestCores calls and 0 as acknowledgment for ReleaseCores calls. Both implement ITaskHostCallbackPacket for request/response correlation via RequestId, following the Stage 1 pattern (TaskHostIsRunningMultipleNodes*). The NodePacketType enum values 0x22/0x23 were already reserved; this commit updates their doc comments to use 'owning worker node' terminology. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Replace 'throw NotImplementedException()' stubs with callback-based forwarding using the Stage 1 SendCallbackRequestAndWaitForResponse<T> infrastructure. Both methods: - Validate input (requestedCores/coresToRelease > 0) before IPC roundtrip, matching the in-process TaskHost's fail-fast behavior - Check CallbacksSupported gate — when disabled, RequestCores returns 0 and ReleaseCores is no-op, both logging MSB5022 - Send TaskHostCoresRequest and block on TaskHostCoresResponse via TCS Register TaskHostCoresResponse handler in the constructor and add the response routing case to HandlePacket's switch statement. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Register TaskHostCoresRequest handler and add HandleCoresRequest() which forwards calls to the in-process TaskHost's IBuildEngine9 implementation. The OOP TaskHostNode is a thin proxy — the real implicit-core accounting and scheduler communication happens in the in-process TaskHost class on the worker node side. HandleCoresRequest dispatches based on IsRelease: - RequestCores: forwards to engine9.RequestCores(), returns GrantedCores - ReleaseCores: forwards to engine9.ReleaseCores(), returns 0 (ack) When _buildEngine doesn't implement IBuildEngine9 (shouldn't happen in practice), RequestCores returns 0 and ReleaseCores is no-op — matching the HandleIsRunningMultipleNodesRequest pattern for IBuildEngine2. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

RequestCoresTask is a test task that calls IBuildEngine9.RequestCores(N) and optionally ReleaseCores. It exposes GrantedCores as an output property for assertion. Linked into ExampleTask.csproj for E2E cross-runtime use. TestNetTaskResourceCallback/ is the E2E test project that runs RequestCoresTask via TaskHostFactory with Runtime='NET', mirroring the TestNetTaskCallback/ project from Stage 1. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Packet serialization (4 tests): - TaskHostCoresRequest round-trip for RequestCores and ReleaseCores - TaskHostCoresResponse round-trip for granted cores and acknowledgment Integration (5 tests): - RequestCores via explicit TaskHostFactory — verifies granted >= 1 - RequestCores + ReleaseCores full cycle — verifies no errors - RequestCores auto-ejected in MT mode — verifies TaskHost forwarding - RequestCores fallback when callbacks disabled — verifies MSB5022 - IsRunningMultipleNodes fallback (Stage 1, unchanged) E2E cross-runtime (1 test): - NetTaskHost_CallbackRequestCoresTest via bootstrapped MSBuild Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

- Mark RequestCores/ReleaseCores as implemented in status table - Rename TaskHostResourceRequest/Response to TaskHostCoresRequest/Response - Update phased rollout table with PR dotnet#13306 - Replace Stage 2 design considerations with implementation summary Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

JanProvaznik · 2026-02-27T16:11:59Z

@AR-May @rainersigwald this one should be simple, it is conceptually the same as the previous PR, the juicy one will be the next one.

Copilot

Pull request overview

Adds Stage 2 of TaskHost IBuildEngine callback forwarding by implementing IBuildEngine9.RequestCores / ReleaseCores over the existing callback packet infrastructure, enabling resource-aware tasks to run out-of-proc without NotImplementedException.

Changes:

Introduces TaskHostCoresRequest / TaskHostCoresResponse packets and new NodePacketType entries.
Implements RequestCores / ReleaseCores forwarding in OutOfProcTaskHostNode (TaskHost side) and handles requests in TaskHostTask (worker node side).
Adds/updates unit + integration + E2E tests and a new ExampleNetTask asset for the callback.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/Shared/TaskHostCoresRequest.cs	New callback request packet for RequestCores/ReleaseCores.
src/Shared/TaskHostCoresResponse.cs	New callback response packet returning granted cores / ack.
src/Shared/INodePacket.cs	Renames/defines packet types for cores request/response (0x22/0x23).
src/MSBuild/OutOfProcTaskHostNode.cs	Implements RequestCores/ReleaseCores via callback request/response + handler registration.
src/MSBuild/MSBuild.csproj	Includes new shared packet source files.
src/Build/Microsoft.Build.csproj	Includes new shared packet source files.
src/Build/Instance/TaskFactories/TaskHostTask.cs	Registers/handles cores request packet and forwards to in-proc `IBuildEngine9`.
src/Build.UnitTests/BackEnd/TaskHostCallbackPacket_Tests.cs	Adds round-trip serialization tests for new packets.
src/Build.UnitTests/BackEnd/TaskHostCallback_Tests.cs	Adds integration tests for RequestCores/ReleaseCores through TaskHost.
src/Build.UnitTests/BackEnd/RequestCoresTask.cs	Adds a test task that exercises RequestCores/ReleaseCores.
src/Build.UnitTests/NetTaskHost_E2E_Tests.cs	Adds E2E test to validate RequestCores callback across runtimes.
src/Build.UnitTests/TestAssets/ExampleNetTask/ExampleTask/ExampleTask.csproj	Links the new RequestCores test task into ExampleTask.
src/Build.UnitTests/TestAssets/ExampleNetTask/TestNetTaskResourceCallback/TestNetTaskResourceCallback.csproj	New test asset project invoking the RequestCores task via TaskHostFactory.
src/Build.UnitTests/TestAssets/ExampleNetTask/TestNetTaskResourceCallback/global.json	New global.json for the new test asset.

src/Build.UnitTests/BackEnd/TaskHostCallback_Tests.cs

src/Build.UnitTests/TestAssets/ExampleNetTask/TestNetTaskResourceCallback/global.json

src/MSBuild/MSBuild.csproj

src/Build.UnitTests/NetTaskHost_E2E_Tests.cs

- Mark RequestCores/ReleaseCores as implemented in status table - Rename TaskHostResourceRequest/Response to TaskHostCoresRequest/Response - Update phased rollout table with PR dotnet#13306 - Replace Stage 2 design considerations with implementation summary Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…rceCallback/global.json Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

MSBuild PR dotnet/msbuild#13306 changed OutOfProcTaskHostNode.RequestCores from throwing NotImplementedException to returning 0 (and logging MSB5022) when callbacks aren't supported. This broke four tasks that only caught NotImplementedException: - EmccCompile: caused blazorwasm publish failure (dotnet/sdk#53300) - MonoAOTCompiler, ILStrip, EmitBundleBase: same latent bug The failure chain was: 1. RequestCores returns 0 (no exception thrown) 2. allowedParallelism becomes 0 3. Parallel.ForEach with MaxDegreeOfParallelism=0 throws 4. ReleaseCores(0) in finally block throws ArgumentOutOfRangeException Fix: track cores actually acquired separately. If RequestCores returns 0 or throws any exception, fall back to the original parallelism and skip ReleaseCores. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

JanProvaznik and others added 5 commits February 27, 2026 16:57

JanProvaznik force-pushed the ibuildengine-callbacks-stage2 branch from 4601b9a to 086f16b Compare February 27, 2026 16:00

JanProvaznik force-pushed the ibuildengine-callbacks-stage2 branch from 086f16b to 89594d2 Compare February 27, 2026 16:05

JanProvaznik marked this pull request as ready for review February 27, 2026 16:08

JanProvaznik requested review from AR-May, Copilot and rainersigwald February 27, 2026 16:08

Copilot started reviewing on behalf of JanProvaznik February 27, 2026 16:08 View session

Copilot AI reviewed Feb 27, 2026

View reviewed changes

src/Build.UnitTests/BackEnd/TaskHostCallback_Tests.cs Show resolved Hide resolved

src/Build.UnitTests/TestAssets/ExampleNetTask/TestNetTaskResourceCallback/global.json Show resolved Hide resolved

rainersigwald approved these changes Feb 27, 2026

View reviewed changes

src/MSBuild/MSBuild.csproj Show resolved Hide resolved

src/Build.UnitTests/NetTaskHost_E2E_Tests.cs Show resolved Hide resolved

JanProvaznik and others added 2 commits March 2, 2026 10:54

Update src/Build.UnitTests/TestAssets/ExampleNetTask/TestNetTaskResou…

f348df1

…rceCallback/global.json Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Merge branch 'main' into ibuildengine-callbacks-stage2

7138fac

JanProvaznik enabled auto-merge (squash) March 2, 2026 09:59

JanProvaznik merged commit 7d3313c into dotnet:main Mar 2, 2026
10 checks passed

This was referenced Mar 2, 2026

[release/11.0.1xx-preview2] Source code updates from dotnet/msbuild dotnet/dotnet#5058

Merged

[main] Source code updates from dotnet/msbuild dotnet/dotnet#5183

Merged

[release/10.0.3xx] Source code updates from dotnet/msbuild dotnet/dotnet#5204

Merged

lewing mentioned this pull request Mar 6, 2026

Handle RequestCores returning 0 in TaskHostFactory tasks dotnet/runtime#125276

Draft

This was referenced Mar 9, 2026

Fix RequestCores/ReleaseCores fallback in OOP TaskHost: throw NotImplementedException instead of logging error #13345

Merged

Stage 3: Forward BuildProjectFile* callbacks from OOP TaskHost to worker node #13350

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[IBuildEngine callbacks] Stage 2: RequestCores/ReleaseCores#13306

[IBuildEngine callbacks] Stage 2: RequestCores/ReleaseCores#13306
JanProvaznik merged 7 commits intodotnet:mainfrom
JanProvaznik:ibuildengine-callbacks-stage2

JanProvaznik commented Feb 27, 2026 •

edited

Loading

Uh oh!

JanProvaznik commented Feb 27, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

JanProvaznik commented Feb 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Context

Changes Made

New Packets:

OutOfProcTaskHostNode (TaskHost side):

TaskHostTask (Worker node side):

Callback Flow

Testing

Documentation

Next Stages

Uh oh!

JanProvaznik commented Feb 27, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

JanProvaznik commented Feb 27, 2026 •

edited

Loading