Add proposal for further build and publish unification#78
Add proposal for further build and publish unification#78dsplaisted wants to merge 1 commit intomasterfrom
Conversation
|
|
||
| However, `PrivateAssets="All"` by default also excludes assets from the package from the publish output. It's not entirely clear why these two behaviors were linked to the same metadata, but it originates from the pre-1.0 project.json days. | ||
|
|
||
| We think now is a good time to take the breaking change to separate these concepts. The main use of PrivateAssets is to control whether assets flow to consuming projects, and the affect on Publish behavior doesn't seem to be documented. |
There was a problem hiding this comment.
I think there's probably too much reliance on PrivateAssets=All -> no publish to make the break across the board. Can we base it on TFM?
There was a problem hiding this comment.
There are "build-time" only tools today that document putting PrivateAssets=All on your PackageReference. For example:
https://github.com/dotnet/sourcelink/blob/master/README.md#githubcom-and-github-enterprise
With this plan, these docs would need to change to also add Publish=False, correct?
The scenario is someone may use Microsoft.SourceLink.GitHub to get the commit hash embedded into their WPF/console/web application exe, but they don't want the SourceLink assemblies to get deployed with their application.
There was a problem hiding this comment.
I think it really depends on whether these packages have any actual runtime assets in lib or runtimes/. Otherwise, they'd technically be included in publish, but wouldn't actually contribute any assets to it, right? The particular package you mention has only build, buildCrossTargeting, and tools/ folders, so I would think not having Publish=false would be benign. Another one just like this that I'm aware of is xliff-tasks (because I wrote it) and is in the same boat. There, the crucial thing is not to flow the dependency. which private assets would still do. I'm not sure how to validate that most tooling packages of this kind are like this, though.
There was a problem hiding this comment.
Good point.
Just to clarify one part of your statement:
I think it really depends on whether these packages have any actual runtime assets
Should be changed to
I think it really depends on whether these packages and their dependencies have any actual runtime assets
|
|
||
| A [runtime package store](https://docs.microsoft.com/en-us/dotnet/core/deploying/runtime-store) is a known set of packages that exist in a target environment that don't need to be included in the app's publish output for framework-dependent apps. Manifests which describe the packages that will be available in a runtime store can be specified via the `--manifest` parameter to `dotnet publish`, or via the `TargetManifestFiles` MSBuild property. Packages specified in these manifests will not be included in the Publish output. | ||
|
|
||
| We would like to also exclude the same package assets from the Build output. This will help us unify the Build and Publish output. The build output should still be runnable on the dev machine whether the runtime store is installed or not, as the NuGet package cache should still be listed in the additionalProbingPaths of the runtimeconfig.dev.json file, so the assets should be loaded from there. |
There was a problem hiding this comment.
I was actually wondering if we can get rid of the .dev.json file if the build output is deployable. Otherwise, you have to remember not to ship it. It still gives your build output a "works only on my machine" potential.
There was a problem hiding this comment.
we can get rid of the .dev.json file
+100. We've accidentally deployed the dev.json file a few times. In fact, just found another instance of this yesterday - dotnet/aspnetcore#10958
|
If these changes were made, what would be the use of dotnet publish? To me, it doesn't seem to have any use anymore? Or am I missing something? |
|
@jmezach There are scenarios where Publish would have additional logic. From the proposal:
|
| @@ -0,0 +1,28 @@ | |||
| # Build and Publish tweaks in the .NET Core SDK | |||
|
|
|||
| In .NET Core 3, we are making Build output more similar to the Publish output. In .NET Core 2.x and prior, dependencies from NuGet packages are not copied into the Build output. At runtime, they are loaded from the NuGet package cache. This meant that the Build output was not portable to other machines. | |||
There was a problem hiding this comment.
This meant that the Build output was not portable to other machines.
If making the Build output portable to other machines is a goal, are we also no longer writing additionalProbingPaths in the .runtimeconfig.dev.json file?
{
"runtimeOptions": {
"additionalProbingPaths": [
"C:\\Users\\eerhardt\\.dotnet\\store\\|arch|\\|tfm|",
"C:\\Users\\eerhardt\\.nuget\\packages",
"C:\\Program Files\\dotnet\\sdk\\NuGetFallbackFolder"
]
}
}There was a problem hiding this comment.
We are still writing it, but I think it needs to be conditioned on something. I mentioned this elsewhere here, I think. I would say the last two should be conditioned on CopyLocalLockFileAssemblies being false and the first one would require a new opt in like UseLocalRuntimeStore (breaking, though we are making other breaks to dotnet store right now.)
|
|
||
| ## `Publish="False"` on `PackageReference` should apply to Build | ||
|
|
||
| If a `PackageReference` has `Publish="False"` metadata, then assets from that package will not be included in the Publish output. In current previews of .NET Core 3, those assets will be included in the Build output but not the Publish output. However, when targeting previous versions of .NET Core, no assets from NuGet packages would be included in the Build output. Also, we are not aware of a scenario where you would want to specify that assets should not be in the Publish output, but would still want them in the Build output. |
There was a problem hiding this comment.
Also, we are not aware of a scenario where you would want to specify that assets should not be in the Publish output, but would still want them in the Build output.
My understanding of the original scenario here was to do with an "original CLI Tool" feature. In that feature/scenario, the tool's dependency graph and the project's dependency graph would be unified, and the tool would run in the context of your project. That way the tool could do things like load types/execute code/etc. I believe that was how the original dotnet test (pre-vstest implementation) worked.
So with that scenario, you'd want to run a CLI Tool against your build-time assets, and some of those dependencies (like the tool itself) weren't needed at publish time.
I hope the above CLI Tooling feature/functionality is no longer in use. But that is my understanding of the driving scenario here to have a "build-time only" dependencies.
There was a problem hiding this comment.
I hope the above CLI Tooling feature/functionality is no longer in use.
How is this functionality triggered. Is there an option on the tool reference or was this something that predates 1.0 even? We're making other changes to limit what CLI tools can do. For example, maximum TFM of 2.2 due to some design flaws. CLI tools are essentially deprecated vs local and global tools, but we need to be deliberate in what we support and do not support.
There was a problem hiding this comment.
Here was a long standing issue with the above feature - https://github.com/dotnet/cli/issues/2645. The issue was that this functionality didn't work when the project was a library (i.e. netstandard) because nothing was "runnable" and brought a runtime (neither the project nor the tool). Hopefully you can glean some answers from there.
My memory is a bit foggy, but I remember the tool itself would come through a dependency, and the CLI knew how to find it in the dependencies, and invoke it. Nate called it a dependency command or "dependency"-type tool in that issue.
|
//cc @nkolev92 for private assets |
|
|
||
| ## `PrivateAssets="All"` on `PackageReference` should not imply `Publish="False"` | ||
|
|
||
| Setting `PrivateAssets="All"` on a `PackageReference` prevents that package from flowing to consuming projects. This means that if a NuGet package is created from the project via Pack, the package with `PrivateAssets="All"` won't be included as a package dependency. This is typically used for packages which are used as part of the build process of a library, but are not dependencies once the library has been compiled. (See also "development dependency" packages.) |
There was a problem hiding this comment.
(See also "development dependency" packages.)
I don't see a section for this. I'm guessing you were thinking about packages with developmentDependency=true in their nuspec metadata, right? The NuGet client might be impacted by this because I believe they add 'PrivateAssets="All"' when you install a developmentDependency package in the VS UI.
There was a problem hiding this comment.
The tooling adds the extra metadata, but I'm not sure in which way you expect NuGet be impacted. @natemcmaster can you please elaborate?
I think @dsplaisted is just drawing a parallel to dev dependencies.
| @@ -0,0 +1,28 @@ | |||
| # Build and Publish tweaks in the .NET Core SDK | |||
|
|
|||
| In .NET Core 3, we are making Build output more similar to the Publish output. In .NET Core 2.x and prior, dependencies from NuGet packages are not copied into the Build output. At runtime, they are loaded from the NuGet package cache. This meant that the Build output was not portable to other machines. | |||
There was a problem hiding this comment.
For what it's worth, NuGet folks would really prefer it if the global packages folder was not referred to as a cache in specs and docs :)
There's no expiration policy there, it's an install location.
|
|
||
| ## `Publish="False"` on `PackageReference` should apply to Build | ||
|
|
||
| If a `PackageReference` has `Publish="False"` metadata, then assets from that package will not be included in the Publish output. In current previews of .NET Core 3, those assets will be included in the Build output but not the Publish output. However, when targeting previous versions of .NET Core, no assets from NuGet packages would be included in the Build output. Also, we are not aware of a scenario where you would want to specify that assets should not be in the Publish output, but would still want them in the Build output. |
There was a problem hiding this comment.
Is publish new metadata?
If implemented as proposed, how does it differ from excludeAssets?
cc @rrelyea
|
|
||
| ## `PrivateAssets="All"` on `PackageReference` should not imply `Publish="False"` | ||
|
|
||
| Setting `PrivateAssets="All"` on a `PackageReference` prevents that package from flowing to consuming projects. This means that if a NuGet package is created from the project via Pack, the package with `PrivateAssets="All"` won't be included as a package dependency. This is typically used for packages which are used as part of the build process of a library, but are not dependencies once the library has been compiled. (See also "development dependency" packages.) |
There was a problem hiding this comment.
The tooling adds the extra metadata, but I'm not sure in which way you expect NuGet be impacted. @natemcmaster can you please elaborate?
I think @dsplaisted is just drawing a parallel to dev dependencies.
|
Exactly what I was looking for, please make this into .NET Core 3. Would be so much easier to develop, test and publish plugins. With Desktop Apps coming along, we need plugin/extension dev use-cases enabled in the SDK. |
|
@dsplaisted should this be merged or closed? |
|
We ended up not doing most of these breaking changes. |
This PR proposes some changes to the SDK behavior that will help us unify the Build and Publish logic. Some of these could be breaking changes, but we believe the impact will be low.