Skip to content

Simplify SB bootstrap package logic#46575

Merged
MichaelSimons merged 3 commits into
dotnet:mainfrom
MichaelSimons:sb-bootstrap-packages
Feb 7, 2025
Merged

Simplify SB bootstrap package logic#46575
MichaelSimons merged 3 commits into
dotnet:mainfrom
MichaelSimons:sb-bootstrap-packages

Conversation

@MichaelSimons

@MichaelSimons MichaelSimons commented Feb 5, 2025 •
edited
Loading

Copy link
Copy Markdown
Member

Fixes dotnet/source-build#3631

With the UB work, I propose refactoring the buildBootstrapPreviouslySB.csproj to make use of the PortableRid property to:

  1. Simplify the msbuild logic by utilizing the PortableRid property.
  2. Only download the packages for the current RID.
  3. Surface a rid option in the prep script for advanced scenarios (e.g. target a different rid)

@ghost ghost added Area-Infrastructure untriaged Request triage from a team member labels Feb 5, 2025
This was referenced Feb 6, 2025
Open
Closed
Open
Open
Closed
mthalman
mthalman reviewed Feb 6, 2025
View reviewed changes
Comment thread src/SourceBuild/content/eng/bootstrap/buildBootstrapPreviouslySB.csproj Outdated
@MichaelSimons MichaelSimons marked this pull request as ready for review February 6, 2025 22:14
@MichaelSimons MichaelSimons requested review from a team as code owners February 6, 2025 22:14
@MichaelSimons MichaelSimons requested review from mthalman and removed request for a team February 6, 2025 22:14
@MichaelSimons

MichaelSimons commented Feb 6, 2025

Copy link
Copy Markdown
Member Author

@mthalman - I refactored a fair amount vs the initial draft. I think utilizing the PortableRid property with the existing pattern is best to ensure correctness as packages are added/removed.

@MichaelSimons MichaelSimons merged commit cfe99e0 into dotnet:main Feb 7, 2025
@MichaelSimons MichaelSimons deleted the sb-bootstrap-packages branch February 7, 2025 14:50
@omajid

omajid commented Mar 26, 2025

Copy link
Copy Markdown
Member

Any idea how to make option 3 "Surface a rid option in the prep script for advanced scenarios (e.g. target a different rid)" work? I have been trying to do what I did before this change: fetch the artifacts for linux-arm64 on a linux-x64 system. I tried using the following invocation, which doesn't seem to work for me:

./prep-source-build.sh --rid linux-arm64

# Runs
# dotnet/.dotnet/dotnet restore dotnet/artifacts/prep-bootstrap/buildBootstrapPreviouslySB.csproj /bl:artifacts/log/prep-bootstrap.binlog /fileLoggerParameters:LogFile=artifacts/log/prep-bootstrap.log /p:ArchiveDir=dotnet/prereqs/packages/archive/ /p:TargetRid=linux-arm64

But only PortableRid (== linux-x64) is used when computing the PackageNames. And the Private.SourceBuilt.Artifacts.Bootstrap.tar.gz contains only linux-x64 artifacts.

@MichaelSimons

MichaelSimons commented Mar 27, 2025

Copy link
Copy Markdown
Member Author

@omajid, Can you open a new issue for your scenario? It looks like the prep script should be exposing the PortableRid option instead of the TargetRid as that is what is really using to bootstrap the artifacts. To workaround, set PortableRid=linux-arm64 as an environment variable and don't use the TargetRid option.

@omajid omajid mentioned this pull request Mar 27, 2025
Merged
@omajid

omajid commented Mar 27, 2025

Copy link
Copy Markdown
Member

Thanks, I opened #47956

omajid added a commit to omajid/dotnet-sdk that referenced this pull request Apr 1, 2025
@omajid
Make it possible to prep other RIDs and architectures
7d0c3d4 
Before dotnet#46575 (commit cfe99e0), a
./prep-source-build.sh in the VMR would produce a
Private.SourceBuilt.Artifacts.Bootstrap.tar.gz that would contain assets for
all architectuers and portable RIDs. However, after that change, it becomes
difficult to generate bootstrap artifacts for arm64 on x64. The expected
approach doesn't work:

    $ ./prep-source-build.sh --rid linux-arm64

This has no effect, because it only sets TargetRid; the logic that
computes what packages to download only uses PortableRid.

After this commit, running the above command produces a bootstrap
archive that contains arm64 artifacts too:

        $ tar tf ./prereqs/packages/archive/Private.SourceBuilt.Artifacts.Bootstrap.tar.gz | grep linux
        Microsoft.AspNetCore.App.Runtime.linux-arm64.10.0.0-preview.2.25164.1.nupkg
        Microsoft.NETCore.App.Crossgen2.linux-arm64.10.0.0-preview.2.25163.2.nupkg
        Microsoft.NETCore.App.Crossgen2.linux-x64.10.0.0-preview.2.25163.2.nupkg
        Microsoft.NETCore.App.Crossgen2.linux-x64.10.0.0-preview.2.25163.2.symbols.nupkg
        Microsoft.NETCore.App.Host.linux-arm64.10.0.0-preview.2.25163.2.nupkg
        Microsoft.NETCore.App.Runtime.linux-arm64.10.0.0-preview.2.25163.2.nupkg
        runtime.linux-arm64.Microsoft.DotNet.ILCompiler.10.0.0-preview.2.25163.2.nupkg
        runtime.linux-arm64.Microsoft.NETCore.DotNetAppHost.10.0.0-preview.2.25163.2.nupkg
        runtime.linux-arm64.Microsoft.NETCore.ILAsm.10.0.0-preview.2.25163.2.nupkg
        runtime.linux-arm64.Microsoft.NETCore.ILDAsm.10.0.0-preview.2.25163.2.nupkg
        runtime.linux-arm64.Microsoft.NETCore.TestHost.10.0.0-preview.2.25163.2.nupkg
        runtime.linux-arm64.runtime.native.System.IO.Ports.10.0.0-preview.2.25163.2.nupkg
        runtime.linux-x64.Microsoft.DotNet.ILCompiler.10.0.0-preview.2.25163.2.nupkg
        runtime.linux-x64.Microsoft.DotNet.ILCompiler.10.0.0-preview.2.25163.2.symbols.nupkg
omajid added a commit to omajid/dotnet-sdk that referenced this pull request Apr 2, 2025
@omajid
Make it possible to prep other RIDs and architectures
c1b3e89 
Before dotnet#46575 (commit cfe99e0), a
./prep-source-build.sh in the VMR would produce a
Private.SourceBuilt.Artifacts.Bootstrap.tar.gz that would contain assets for
all architectuers and portable RIDs. However, after that change, it becomes
difficult to generate bootstrap artifacts for arm64 on x64. The expected
approach doesn't work:

    $ ./prep-source-build.sh --rid linux-arm64

This has no effect, because it only sets TargetRid; the logic that
computes what packages to download only uses PortableRid.

After this commit, the following command produces a bootstrap archive on
x64 that contains arm64 artifacts too:

    $ ./prep-source-build.sh --bootstrap-rid linux-arm64

    $ tar tf ./prereqs/packages/archive/Private.SourceBuilt.Artifacts.Bootstrap.tar.gz | grep linux
    Microsoft.AspNetCore.App.Runtime.linux-arm64.10.0.0-preview.2.25164.1.nupkg
    Microsoft.NETCore.App.Crossgen2.linux-arm64.10.0.0-preview.2.25163.2.nupkg
    Microsoft.NETCore.App.Crossgen2.linux-x64.10.0.0-preview.2.25163.2.nupkg
    Microsoft.NETCore.App.Crossgen2.linux-x64.10.0.0-preview.2.25163.2.symbols.nupkg
    Microsoft.NETCore.App.Host.linux-arm64.10.0.0-preview.2.25163.2.nupkg
    Microsoft.NETCore.App.Runtime.linux-arm64.10.0.0-preview.2.25163.2.nupkg
    runtime.linux-arm64.Microsoft.DotNet.ILCompiler.10.0.0-preview.2.25163.2.nupkg
    runtime.linux-arm64.Microsoft.NETCore.DotNetAppHost.10.0.0-preview.2.25163.2.nupkg
    runtime.linux-arm64.Microsoft.NETCore.ILAsm.10.0.0-preview.2.25163.2.nupkg
    runtime.linux-arm64.Microsoft.NETCore.ILDAsm.10.0.0-preview.2.25163.2.nupkg
    runtime.linux-arm64.Microsoft.NETCore.TestHost.10.0.0-preview.2.25163.2.nupkg
    runtime.linux-arm64.runtime.native.System.IO.Ports.10.0.0-preview.2.25163.2.nupkg
    runtime.linux-x64.Microsoft.DotNet.ILCompiler.10.0.0-preview.2.25163.2.nupkg
    runtime.linux-x64.Microsoft.DotNet.ILCompiler.10.0.0-preview.2.25163.2.symbols.nupkg
omajid added a commit to omajid/dotnet-sdk that referenced this pull request Apr 3, 2025
@omajid
Make it possible to prep other RIDs and architectures
0422ea6 
Before dotnet#46575 (commit cfe99e0), a
./prep-source-build.sh in the VMR would produce a
Private.SourceBuilt.Artifacts.Bootstrap.tar.gz that would contain assets for
all architectuers and portable RIDs. However, after that change, it becomes
difficult to generate bootstrap artifacts for arm64 on x64. The expected
approach doesn't work:

    $ ./prep-source-build.sh --rid linux-arm64

This has no effect, because it only sets TargetRid; the logic that
computes what packages to download only uses PortableRid.

After this commit, the following command produces a bootstrap archive on
x64 that contains arm64 artifacts too:

    $ ./prep-source-build.sh --bootstrap-rid linux-arm64

    $ tar tf ./prereqs/packages/archive/Private.SourceBuilt.Artifacts.Bootstrap.tar.gz | grep linux
    Microsoft.AspNetCore.App.Runtime.linux-arm64.10.0.0-preview.2.25164.1.nupkg
    Microsoft.NETCore.App.Crossgen2.linux-arm64.10.0.0-preview.2.25163.2.nupkg
    Microsoft.NETCore.App.Crossgen2.linux-x64.10.0.0-preview.2.25163.2.nupkg
    Microsoft.NETCore.App.Crossgen2.linux-x64.10.0.0-preview.2.25163.2.symbols.nupkg
    Microsoft.NETCore.App.Host.linux-arm64.10.0.0-preview.2.25163.2.nupkg
    Microsoft.NETCore.App.Runtime.linux-arm64.10.0.0-preview.2.25163.2.nupkg
    runtime.linux-arm64.Microsoft.DotNet.ILCompiler.10.0.0-preview.2.25163.2.nupkg
    runtime.linux-arm64.Microsoft.NETCore.DotNetAppHost.10.0.0-preview.2.25163.2.nupkg
    runtime.linux-arm64.Microsoft.NETCore.ILAsm.10.0.0-preview.2.25163.2.nupkg
    runtime.linux-arm64.Microsoft.NETCore.ILDAsm.10.0.0-preview.2.25163.2.nupkg
    runtime.linux-arm64.Microsoft.NETCore.TestHost.10.0.0-preview.2.25163.2.nupkg
    runtime.linux-arm64.runtime.native.System.IO.Ports.10.0.0-preview.2.25163.2.nupkg
    runtime.linux-x64.Microsoft.DotNet.ILCompiler.10.0.0-preview.2.25163.2.nupkg
    runtime.linux-x64.Microsoft.DotNet.ILCompiler.10.0.0-preview.2.25163.2.symbols.nupkg
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mthalman mthalman mthalman approved these changes

Assignees

No one assigned

Labels

Area-Infrastructure untriaged Request triage from a team member

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Prep bootstrap logic should only pull packages for current rid by default

3 participants

@MichaelSimons @omajid @mthalman