Skip to content

Publish source build artifacts#1355

Merged
mthalman merged 5 commits intodotnet:mainfrom
mthalman:dev/mthalman/issue1110-publish-sb-artifacts-2
Jul 2, 2025
Merged

Publish source build artifacts#1355
mthalman merged 5 commits intodotnet:mainfrom
mthalman:dev/mthalman/issue1110-publish-sb-artifacts-2

Conversation

@mthalman
Copy link
Member

Fixes #1110

This updates the UB pipeline to publish source built artifacts. This is necessary in order for source-only builds of the 2xx branch to be able to consume the packages that were produced by the 1xx branch. For example, the 1xx branch would produce the System.Text.Json package from the runtime repo which can then be pulled in as a dependency for the 2xx branch's source build, since runtime source isn't included in that branch.

Originally I tried to include the artifacts tarball in the existing VerticalManifest.xml file that gets produced by a source-only job. That's problematic because of a chicken-and-egg problem. The artifacts tarball includes the VerticalManifest.xml file in it. But since we want the VerticalManifest.xml file to include a reference to this tarball, we can't produce the VerticalManifest.xml file without the artifacts tarball already existing.

Instead a different approach is taken where a separate manifest file is produced. This allows the artifacts tarball to be produced after the VerticalManifest.xml file just as before. And then a new manifest file gets generated that only references the artifacts tarball. This is done by making two passes of the Publish target, referencing different manifests and configuration to include the appropriate files.

Prior to these changes all manifests from source-only jobs were excluded from the processing in the Publish stage. These changes update that to specifically exclude the original VerticalManifest.xml file and only include the manifest file containing the artifacts tarball, whose name is based on the job name for uniqueness purposes.

Some other things that needed to be fixed with these changes:

  • Offline builds need to pass the -E option to sudo in order to flow the system environment variables to the build. Without this option set, AzDO variables like BUILD_BUILDID were not set which are necessary in order to produce a correct manifest file.
  • Roslyn publishing needs an update to skip explicitly adding FilesToPublishToSymbolServer items for source-only builds. These symbols shouldn't be published for source-only and were incorrectly showing up in the manifests for the artifacts tarball without these changes.

@ViktorHofer ViktorHofer requested a review from jkoritzinsky June 30, 2025 15:51
@mthalman mthalman requested a review from ViktorHofer June 30, 2025 18:23
@mthalman mthalman merged commit 15fbd58 into dotnet:main Jul 2, 2025
10 checks passed
@mthalman mthalman deleted the dev/mthalman/issue1110-publish-sb-artifacts-2 branch July 2, 2025 15:22
akoeplinger pushed a commit to dkurepa/dotnet that referenced this pull request Jul 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

[feature bands] Publish source-built packages

2 participants