Fix RecursiveDir metadata loss in multithreaded builds by JanProvaznik · Pull Request #13142 · dotnet/msbuild

JanProvaznik · 2026-01-29T13:35:24Z

When using MSBuild's multithreaded mode (-mt / /maxcpucount), the %(RecursiveDir) built-in metadata was lost when items crossed process boundaries to the TaskHost. This caused NuGet pack to flatten directory structures (e.g., build\wix\bundle\bundle.wxs became build\bundle.wxs).

Root cause: TaskParameterTaskItem only copied custom metadata when wrapping ITaskItem for TaskHost serialization. RecursiveDir is a built-in metadata that cannot be derived from just the item spec - it requires the original wildcard pattern (_includeBeforeWildcardExpansionEscaped).

Fix: Explicitly copy RecursiveDir to custom metadata in TaskParameterTaskItem constructor before serialization, so it survives the cross-process boundary.

Also includes related fixes to ProjectItemInstance.cs:

Preserve _projectDirectory in copy constructor and serialization
Fix CreateItem(string, string, string) to use includeBeforeWildcardExpansionEscaped

related: #3121
fixes #13140

Copilot

Pull request overview

Fixes loss of %(RecursiveDir) when items cross the TaskHost process boundary during multithreaded builds, preventing directory flattening in scenarios like dotnet pack.

Changes:

Preserve RecursiveDir by copying it into custom metadata during TaskParameterTaskItem construction for TaskHost serialization.
Preserve ProjectItemInstance.TaskItem’s _projectDirectory across cloning and serialization.
Ensure TaskItemFactory.CreateItem(string, string, string) preserves includeBeforeWildcardExpansionEscaped (needed for correct RecursiveDir semantics).

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File	Description
src/Shared/TaskParameter.cs	Copies `RecursiveDir` into custom metadata so it survives TaskHost serialization.
src/Build/Instance/ProjectItemInstance.cs	Preserves `_projectDirectory` across clone/serialization; fixes task item creation to keep pre-wildcard include.

baronfel · 2026-01-29T13:45:49Z

Oh wow - I think I got this in the RID-specific tool packaging work and worked around it without understanding what was happening. Really great identification of the problem here!

When items were returned from MSBuild tasks (like the MSBuild task's TargetOutputs) or passed to tasks via output parameters, the %(RecursiveDir) built-in metadata was lost. This caused NuGet pack to flatten directory structures (e.g., build\wix\bundle\bundle.wxs became build\bundle.wxs), resulting in NU5118 errors. Root causes found and fixed: 1. TaskExecutionHost.cs: When creating ProjectItemInstance from task outputs, only IncludeEscaped was passed to the constructor, losing IncludeBeforeWildcardExpansionEscaped which is required for RecursiveDir. Fix: Use the 5-parameter constructor that accepts both values. 2. TaskParameter.cs: TaskParameterTaskItem only copied custom metadata when wrapping ITaskItem for TaskHost serialization, but RecursiveDir is a built-in metadata computed from _includeBeforeWildcardExpansionEscaped. Fix: Explicitly copy RecursiveDir to custom metadata so it survives cross-process serialization. Added ContainsKey check before calling GetMetadataValueEscaped to avoid expensive FileMatcher calls. 3. ProjectItemInstance.cs: Related fixes to preserve _projectDirectory in copy constructor, serialization, and CreateItem methods. The CreateItem method now derives projectDirectory from definingProject. Fixes dotnet#3121

JanProvaznik · 2026-02-04T11:39:53Z

looking for 2 reviewers here

rainersigwald

Feels like we should have an end-to-end test pinning the fix too.

MSBuild now preserves RecursiveDir across task boundaries (dotnet/msbuild#13142). Some targets (e.g. sharedfx.targets) bake RecursiveDir into TargetPath/PackagePath via %(RecursiveDir), relying on MSBuild previously losing it after the task boundary. With the fix, NuGet Pack would append RecursiveDir again, doubling the path (e.g. analyzers/dotnet/cs/dotnet/cs/ instead of analyzers/dotnet/cs/). Add an EndsWith guard to skip appending RecursiveDir when PackagePath already contains it. This is backward-compatible with both old and new MSBuild.

This reverts commit eff1a3b.

When using MSBuild's multithreaded mode (-mt / /maxcpucount), the %(RecursiveDir) built-in metadata was lost when items crossed process boundaries to the TaskHost. This caused NuGet pack to flatten directory structures (e.g., build\wix\bundle\bundle.wxs became build\bundle.wxs). Root cause: TaskParameterTaskItem only copied custom metadata when wrapping ITaskItem for TaskHost serialization. RecursiveDir is a built-in metadata that cannot be derived from just the item spec - it requires the original wildcard pattern (_includeBeforeWildcardExpansionEscaped). Fix: Explicitly copy RecursiveDir to custom metadata in TaskParameterTaskItem constructor before serialization, so it survives the cross-process boundary. Also includes related fixes to ProjectItemInstance.cs: - Preserve _projectDirectory in copy constructor and serialization - Fix CreateItem(string, string, string) to use includeBeforeWildcardExpansionEscaped related: #3121 fixes #13140

When using MSBuild's multithreaded mode (-mt / /maxcpucount), the %(RecursiveDir) built-in metadata was lost when items crossed process boundaries to the TaskHost. This caused NuGet pack to flatten directory structures (e.g., build\wix\bundle\bundle.wxs became build\bundle.wxs). Root cause: TaskParameterTaskItem only copied custom metadata when wrapping ITaskItem for TaskHost serialization. RecursiveDir is a built-in metadata that cannot be derived from just the item spec - it requires the original wildcard pattern (_includeBeforeWildcardExpansionEscaped). Fix: Explicitly copy RecursiveDir to custom metadata in TaskParameterTaskItem constructor before serialization, so it survives the cross-process boundary. Also includes related fixes to ProjectItemInstance.cs: - Preserve _projectDirectory in copy constructor and serialization - Fix CreateItem(string, string, string) to use includeBeforeWildcardExpansionEscaped related: dotnet#3121 fixes dotnet#13140

When using MSBuild's -mt mode (/maxcpucount), tasks without [MSBuildMultiThreadableTask] are routed to out-of-process TaskHost sidecars. Items passed to these tasks are serialized via TaskParameterTaskItem, which calls CloneCustomMetadataEscaped() — copying only custom metadata. RecursiveDir is a built-in metadata that is non-derivable (requires _includeBeforeWildcardExpansionEscaped), so it's lost during this serialization, returning empty string on the TaskHost side. This causes dotnet pack /mt to flatten directory structures in NuGet packages (e.g., build/wix/bundle/bundle.wxs becomes build/bundle.wxs). Fix: Explicitly copy RecursiveDir to custom metadata in the TaskParameterTaskItem constructor before serialization, so it survives the cross-process boundary. Unlike the previous attempt (PR dotnet#13142, reverted in PR dotnet#13245), this fix ONLY touches the TaskHost serialization boundary (TaskParameterTaskItem). It does NOT change GatherTaskItemOutputs or ProjectItemInstance, which was what caused the NuGet/sfxproj double-append regression. Items that lose RecursiveDir at the MSBuild task callback boundary (e.g., sfxproj targets returning items via TargetOutputs) are unaffected — they already have RecursiveDir=empty before reaching TaskParameterTaskItem. Fixes dotnet#13140 Related: dotnet#3121 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings January 29, 2026 13:35

Copilot started reviewing on behalf of JanProvaznik January 29, 2026 13:35 View session

Copilot AI reviewed Jan 29, 2026

View reviewed changes

Comment thread src/Shared/TaskParameter.cs

Comment thread src/Shared/TaskParameter.cs

Comment thread src/Build/Instance/ProjectItemInstance.cs Outdated

Comment thread src/Shared/TaskParameter.cs Outdated

JanProvaznik force-pushed the fix-recursivedir-taskhost branch 2 times, most recently from 675e5d0 to f1be335 Compare January 29, 2026 14:00

JanProvaznik force-pushed the fix-recursivedir-taskhost branch from f1be335 to f17258b Compare January 29, 2026 14:01

JanProvaznik commented Jan 30, 2026

View reviewed changes

Comment thread src/Shared/TaskParameter.cs Outdated

Comment thread src/Shared/UnitTests/TaskParameter_Tests.cs Outdated

JanProvaznik added 2 commits January 30, 2026 16:35

Update src/Shared/TaskParameter.cs

da106ca

Update src/Shared/UnitTests/TaskParameter_Tests.cs

1305e39

JanProvaznik mentioned this pull request Feb 2, 2026

Fix RecursiveDir metadata lost when gathering task item outputs #13141

Closed

JanProvaznik self-assigned this Feb 2, 2026

JanProvaznik requested review from ViktorHofer, YuliiaKovalova and rainersigwald February 4, 2026 11:40

ViktorHofer reviewed Feb 4, 2026

View reviewed changes

Comment thread src/Shared/TaskParameter.cs

ViktorHofer approved these changes Feb 4, 2026

View reviewed changes

rainersigwald approved these changes Feb 8, 2026

View reviewed changes

end to end test

2ecab16

JanProvaznik enabled auto-merge (squash) February 9, 2026 10:26

JanProvaznik merged commit eff1a3b into dotnet:main Feb 9, 2026
10 checks passed

dotnet-maestro bot mentioned this pull request Feb 10, 2026

[main] Source code updates from dotnet/msbuild dotnet/dotnet#4746

Closed

JanProvaznik added a commit that referenced this pull request Feb 12, 2026

Revert "Fix RecursiveDir metadata loss in multithreaded builds (#13142)"

57feb26

This reverts commit eff1a3b.

JanProvaznik mentioned this pull request Feb 12, 2026

Revert "Fix RecursiveDir metadata loss in multithreaded builds" #13245

Merged

dotnet-maestro bot mentioned this pull request Feb 13, 2026

[main] Source code updates from dotnet/msbuild dotnet/dotnet#4835

Merged

JanProvaznik mentioned this pull request Mar 3, 2026

Fix RecursiveDir metadata loss in -mt builds (TaskHost boundary only) #13318

Merged

dotnet-maestro bot mentioned this pull request Mar 3, 2026

[release/10.0.3xx] Source code updates from dotnet/msbuild dotnet/dotnet#5204

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix RecursiveDir metadata loss in multithreaded builds#13142

Fix RecursiveDir metadata loss in multithreaded builds#13142
JanProvaznik merged 4 commits intodotnet:mainfrom
JanProvaznik:fix-recursivedir-taskhost

JanProvaznik commented Jan 29, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

baronfel commented Jan 29, 2026

Uh oh!

Uh oh!

Uh oh!

JanProvaznik commented Feb 4, 2026

Uh oh!

Uh oh!

rainersigwald left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

JanProvaznik commented Jan 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

baronfel commented Jan 29, 2026

Uh oh!

Uh oh!

Uh oh!

JanProvaznik commented Feb 4, 2026

Uh oh!

Uh oh!

rainersigwald left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

JanProvaznik commented Jan 29, 2026 •

edited

Loading