Skip to content

Replace Build.SourcesDirectory with System.DefaultWorkingDirectory#16032

Merged
MiYanni merged 3 commits intodotnet:mainfrom
MiYanni:ReplaceSourcesDirectory
Aug 15, 2025
Merged

Replace Build.SourcesDirectory with System.DefaultWorkingDirectory#16032
MiYanni merged 3 commits intodotnet:mainfrom
MiYanni:ReplaceSourcesDirectory

Conversation

@MiYanni
Copy link
Copy Markdown
Member

@MiYanni MiYanni commented Aug 6, 2025

Summary

Currently, Build.SourcesDirectory is used to access the directory for which the code is checked out. However, this predefined variable does not work for multi-repo checkout situations. The proper variable that works for both single-checkout and multi-checkout pipelines is System.DefaultWorkingDirectory. There is also Build.Repository.LocalPath, but that does not have the desired outcome. More explanation in the Details section.

There is an additional change adding repositoryAlias to jobs.yml. This allows you to pass in the repository you want to run publish-build-assets.yml from. Otherwise, the default of self may not have Arcade. This commit encompasses those changes. Not sure if there is a doc that needs to be updated or not. Based on the docs, repositoryAlias seems to be an appropriate name.

image

Docs

https://learn.microsoft.com/en-us/azure/devops/pipelines/build/variables?view=azure-devops&tabs=yaml#build-variables-devops-services

Build.SourcesDirectory

image

Build.Repository.LocalPath

image

System.DefaultWorkingDirectory

image

Details

I found this when trying to do a multi-checkout pipeline that uses Arcade within the primary repo. The SBOM generation task would fail because it couldn't locate the artifacts folder. Therefore, I figured that I should apply this change to all files in the repo to make them work properly with multi-checkout pipelines.

At first, I tested Build.Repository.LocalPath, but there is a reason this is not the correct value to use. See the Example below.

Example

My pipeline, example.yml, is in a repository named RepoA. This pipeline pulls in a different repo, RepoB. In the yml, it would look like this:

RepoA/example.yml

...

resources:
  repositories:
  - repository: RepoB
    type: git
    name: example/RepoB
    ref: refs/heads/main

...

So, you have RepoA that references RepoB. So, when you go to do checkout (which needs to be provided in the yml because of multiple repos), it might look something like this:

RepoA/example.yml

...

steps:
- checkout: self
  path: RepoA
  displayName: Checkout RepoA
- checkout: RepoB
  path: RepoB
  workspaceRepo: true
  displayName: Checkout RepoB

...

The main thing to notice is that RepoB has workspaceRepo: true on it. What does that do? It sets the working directory for Azure Pipelines to RepoB's directory. So, here are what the paths look like for reference:

  • Default: D:\a\_work\1\s (what a single repo checkout uses)
  • RepoA: D:\a\_work\1\RepoA (based on setting path on checkout)
  • RepoB: D:\a\_work\1\RepoB (based on setting path on checkout)

With this knowledge, how does that affect the predefined variables? Here is what you end up with (by spitting out the env vars that represent the predefined variables):

BUILD_SOURCESDIRECTORY         D:\a\_work\1\s
BUILD_REPOSITORY_LOCALPATH     D:\a\_work\1\RepoA
SYSTEM_DEFAULTWORKINGDIRECTORY D:\a\_work\1\RepoB

Since you've set the working directory using the workspaceRepo value, the intended directory is the one defined in System.DefaultWorkingDirectory. Therefore, this is the correct variable to use. Note that all 3 of these variables are set to D:\a\_work\1\s in a single-repo checkout scenario (the typical scenario).

@MiYanni MiYanni requested review from a team, joeloff and mmitche August 6, 2025 22:28
MiYanni added a commit to dotnet/workload-versions that referenced this pull request Aug 7, 2025
@MiYanni
Copy link
Copy Markdown
Member Author

MiYanni commented Aug 7, 2025

I've manually changed these files on a branch in the repo I'm testing. These changes seem to break signing, but I'm not sure how.

##[error].packages\microsoft.dotnet.arcade.sdk\10.0.0-beta.25380.108\tools\Sign.proj(76,5): error : Nuget package D:\a_work\1\source-branch\artifacts\packages\Release\Shipping\[[REDACTED]].nupkg is not signed properly.

You'll see the working directory is D:\a\_work\1\source-branch instead of the default of D:\a\_work\1\s, which is correct with these changes. However, signing doesn't work because of it.

@joeloff
Copy link
Copy Markdown
Member

joeloff commented Aug 7, 2025

I've manually changed these files on a branch in the repo I'm testing. These changes seem to break signing, but I'm not sure how.

##[error].packages\microsoft.dotnet.arcade.sdk\10.0.0-beta.25380.108\tools\Sign.proj(76,5): error : Nuget package D:\a_work\1\source-branch\artifacts\packages\Release\Shipping[[REDACTED]].nupkg is not signed properly.

You'll see the working directory is D:\a\_work\1\source-branch instead of the default of D:\a\_work\1\s, which is correct with these changes. However, signing doesn't work because of it.

were you doing full signing? The signing binlog might help. This seems to be coming from the sign tool.

@MiYanni
Copy link
Copy Markdown
Member Author

MiYanni commented Aug 8, 2025

After some investigation, I determined the only file that could potentially affect signing would be the one that installs MicroBuild. After reverting this, signing worked:
image

That means there is something hardcoded somewhere related to MicroBuild installation and/or signing validation.

@MiYanni
Copy link
Copy Markdown
Member Author

MiYanni commented Aug 8, 2025

@joeloff

were you doing full signing? The signing binlog might help. This seems to be coming from the sign tool.

The signing binlog didn't provide any information. Basically only said "starting signing". I think that's because it couldn't find the actual sign tool or something like that. As mentioned, installing it (MicroBuild) in the default location worked.

@MiYanni
Copy link
Copy Markdown
Member Author

MiYanni commented Aug 8, 2025

I am noticing something that is not correct when using Build.Repository.LocalPath. I may need to close this PR and make a new one using another variable. Let me just make this a draft for now.

@MiYanni MiYanni marked this pull request as draft August 8, 2025 00:43
@MiYanni MiYanni changed the title Replace Build.SourcesDirectory with Build.Repository.LocalPath Replace Build.SourcesDirectory with System.DefaultWorkingDirectory Aug 8, 2025
Comment on lines +15 to 16
# NOTE: There's something that relies on this being in the "default" source directory for tasks such as Signing to work properly.
microBuildOutputFolder: '$(Build.SourcesDirectory)'
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bug with how MicroBuild is handling the output folder. Someone should investigate this as changing the value here breaks Signing. As long as the MicroBuild tools are installed into Build.SourcesDirectory, everything seems to work properly, so this cannot be changed currently.


Optional variables:
EnableDefaultArtifacts Includes *.nupkg, *.vsix and *.wixpack.zip under "/artifacts/packages/**" for sigining.
EnableDefaultArtifacts Includes *.nupkg, *.vsix and *.wixpack.zip under "/artifacts/packages/**" for signing.
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Just a typo fix.

@MiYanni MiYanni marked this pull request as ready for review August 8, 2025 22:56
@MiYanni MiYanni requested a review from a team August 8, 2025 23:55
joeloff
joeloff previously approved these changes Aug 8, 2025
Copy link
Copy Markdown
Member

@joeloff joeloff left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, especially for the use cases in the SDK side where we plan to have multiple repos. But definitely suggest someone from dnceng to take a look in case this can impact other repos that consume Arcade

@MiYanni
Copy link
Copy Markdown
Member Author

MiYanni commented Aug 9, 2025

Also note, I'll need to make similar PRs for Arcade 8 and 9.

@MiYanni MiYanni requested a review from ellahathaway August 9, 2025 00:51
@MiYanni
Copy link
Copy Markdown
Member Author

MiYanni commented Aug 9, 2025

Added @ellahathaway for the aforementioned MicroBuild install problem as I looked through the history and saw it was changed from Agent.TempDirectory to Build.SourcesDirectory, so she might have some more insight.

@ellahathaway
Copy link
Copy Markdown
Member

Added @ellahathaway for the aforementioned MicroBuild install problem as I looked through the history and saw it was changed from Agent.TempDirectory to Build.SourcesDirectory, so she might have some more insight.

The install problem you’re seeing is #15731. @dotnet/dnceng

Copy link
Copy Markdown
Member

@chcosta chcosta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have a strong opinion here. According to the documentation I dug through, the behavioral differences here are super nuanced, and despite the claim, that build.sourcesdirectory and system.defaultworkingdirectory are different, they're not supposed to be. so I don't have a clear recommend. it seems safe either way and I'd defer to someone who has actually spent time exploring those nuances.

The behavior I'd expect is

steps:
- checkout: git://project/first
  path: repo/first-repo

- checkout: git://project/second  
  path: repo/second-repo
  workspaceRepo: true

- script: |
    echo "Current directory: $(pwd)"
    # Output: $(Pipeline.Workspace)/repo/second-repo
    
    echo "Build.SourcesDirectory: $(Build.SourcesDirectory)"
    # Output: $(Pipeline.Workspace)/s (NOT the working directory!)
    
    echo "System.DefaultWorkingDirectory: $(System.DefaultWorkingDirectory)"
    # Output: $(Pipeline.Workspace)/s (NOT the working directory!)

@MiYanni MiYanni merged commit 870e0b2 into dotnet:main Aug 15, 2025
10 checks passed
@MiYanni MiYanni deleted the ReplaceSourcesDirectory branch August 15, 2025 18:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants