Overhaul diagnostics in upgrade to not silently ignore parse failures by BillyONeal · Pull Request #1486 · microsoft/vcpkg-tool

BillyONeal · 2024-09-05T06:15:48Z

In #1473 it was discovered that there were several issues with how vcpkg upgrade works:

Structurally, upgrade couldn't tell the difference between a malformed port or a port that totally didn't exist. This meant that parse errors got translated into "not found" which is bad when someone might be editing a port.
upgrade was duplicating a lot of the package load functionality that should be handled by the way the rest of the tool handles loading ports from disk.
There were no tests for much of the upgrade functionality.

Consider the following before and after results. The crash was already fixed in #1473 but other improvements are here.

Before:

PS D:\vcpkg> Set-EmptyTestPort -Name example -Version 1 -PortsRoot D:\vcpkg\ports
PS D:\vcpkg> .\vcpkg.exe install example
Computing installation plan...
The following packages will be built and installed:
    example:x64-windows@1
Detecting compiler hash for triplet x64-windows...
Compiler found: C:/Program Files/Microsoft Visual Studio/2022/Enterprise/VC/Tools/MSVC/14.41.34120/bin/Hostx64/x64/cl.exe
Restored 1 package(s) from C:\Users\bion\AppData\Local\vcpkg\archives in 55.8 ms. Use --debug to see more details.
Installing 1/1 example:x64-windows@1...
Elapsed time to handle example:x64-windows: 2.2 ms
example:x64-windows package ABI: eb2404a5feb27c3ca35c3362ae7c7eb56390ecfd60374404b7e825339fa7a88e
Total install time: 2.42 ms
PS D:\vcpkg> .\vcpkg.exe upgrade example
The following packages are up-to-date:
  example:x64-windows
PS D:\vcpkg> .\vcpkg.exe upgrade
All installed packages are up-to-date.
PS D:\vcpkg> .\vcpkg.exe upgrade vcpkg-cmake
error: The following packages are not installed:
  vcpkg-cmake:x64-windows
internal error: D:\a\_work\1\s\src\vcpkg\commands.upgrade.cpp(183): vcpkg has crashed; no additional details are available.
Please open an issue at https://github.com/microsoft/vcpkg/issues/new?template=other-type-of-bug-report.md&labels=category:vcpkg-bug with detailed steps to reproduce the problem.
PS D:\vcpkg> Remove-Item D:\vcpkg\ports\example -Recurse -Force
PS D:\vcpkg> .\vcpkg.exe upgrade
All installed packages are up-to-date.
PS D:\vcpkg> .\vcpkg.exe upgrade example
error: The following packages do not have a valid CONTROL or vcpkg.json:
  example:x64-windows
internal error: D:\a\_work\1\s\src\vcpkg\commands.upgrade.cpp(183): vcpkg has crashed; no additional details are available.
Please open an issue at https://github.com/microsoft/vcpkg/issues/new?template=other-type-of-bug-report.md&labels=category:vcpkg-bug with detailed steps to reproduce the problem.
PS D:\vcpkg> Set-EmptyTestPort -Name example -Version 2 -PortsRoot D:\vcpkg\ports
PS D:\vcpkg> Set-EmptyTestPort -Name example-malformed -Version 2 -PortsRoot D:\vcpkg\ports
PS D:\vcpkg> .\vcpkg.exe install example example-malformed --recurse
Computing installation plan...
The following packages are already installed:
    example:x64-windows@1
The following packages will be built and installed:
    example-malformed:x64-windows@2
Detecting compiler hash for triplet x64-windows...
Compiler found: C:/Program Files/Microsoft Visual Studio/2022/Enterprise/VC/Tools/MSVC/14.41.34120/bin/Hostx64/x64/cl.exe
Restored 0 package(s) from C:\Users\bion\AppData\Local\vcpkg\archives in 98 us. Use --debug to see more details.
example:x64-windows is already installed
Installing 1/1 example-malformed:x64-windows@2...
Building example-malformed:x64-windows@2...
-- Skipping post-build validation due to VCPKG_POLICY_EMPTY_PACKAGE
Stored binaries in 1 destinations in 33 ms.
Elapsed time to handle example-malformed:x64-windows: 85 ms
example-malformed:x64-windows package ABI: a566feeff21f116d018c0b8c231e090eb7e601962cbc1687ca5a3f64cab1500f
Total install time: 85.6 ms
PS D:\vcpkg> Set-EmptyTestPort -Name example-malformed -Version 2 -PortsRoot D:\vcpkg\ports -Malformed
PS D:\vcpkg> .\vcpkg.exe upgrade
The following packages will be rebuilt:
  * example:x64-windows@2
Additional packages (*) will be modified to complete this operation.
If you are sure you want to rebuild the above packages, run this command with the --no-dry-run option.
PS D:\vcpkg> .\vcpkg.exe upgrade example-malformed
error: The following packages do not have a valid CONTROL or vcpkg.json:
  example-malformed:x64-windows
internal error: D:\a\_work\1\s\src\vcpkg\commands.upgrade.cpp(183): vcpkg has crashed; no additional details are available.
Please open an issue at https://github.com/microsoft/vcpkg/issues/new?template=other-type-of-bug-report.md&labels=category:vcpkg-bug with detailed steps to reproduce the problem.

After:

PS D:\vcpkg> Copy-Item D:\vcpkg-tool\out\build\Win-x64-Debug-WithArtifacts\vcpkg.exe vcpkg.exe -force
PS D:\vcpkg> Set-EmptyTestPort -Name example -Version 1 -PortsRoot D:\vcpkg\ports
PS D:\vcpkg> .\vcpkg.exe install example
Computing installation plan...
The following packages will be built and installed:
    example:x64-windows@1
Detecting compiler hash for triplet x64-windows...
Compiler found: C:/Program Files/Microsoft Visual Studio/2022/Enterprise/VC/Tools/MSVC/14.41.34120/bin/Hostx64/x64/cl.exe
Restored 1 package(s) from C:\Users\bion\AppData\Local\vcpkg\archives in 46.3 ms. Use --debug to see more details.
Installing 1/1 example:x64-windows@1...
Elapsed time to handle example:x64-windows: 3.01 ms
example:x64-windows package ABI: eb2404a5feb27c3ca35c3362ae7c7eb56390ecfd60374404b7e825339fa7a88e
Total install time: 3.33 ms
PS D:\vcpkg> .\vcpkg.exe upgrade example
The following packages are up-to-date:
  example:x64-windows@1
All installed packages are up-to-date.
PS D:\vcpkg> .\vcpkg.exe upgrade
The following packages are up-to-date:
  example:x64-windows@1
All installed packages are up-to-date.
PS D:\vcpkg> .\vcpkg.exe upgrade vcpkg-cmake
error: The following explicitly requested upgrades are for packages that are not installed.
  vcpkg-cmake:x64-windows
PS D:\vcpkg> Remove-Item D:\vcpkg\ports\example -Recurse -Force
PS D:\vcpkg> .\vcpkg.exe upgrade
warning: While determining which ports to upgrade, the following parse errors were encountered. Ports with parse errors won't be considered out of date and thus won't be upgraded. To consider these errors in the future, run `vcpkg upgrade` with --`no-keep-going`.
D:\vcpkg\ports\example: error: example does not exist
All installed packages are up-to-date.
PS D:\vcpkg> .\vcpkg.exe upgrade example
D:\vcpkg\ports\example: error: example does not exist
PS D:\vcpkg> Set-EmptyTestPort -Name example -Version 2 -PortsRoot D:\vcpkg\ports
PS D:\vcpkg> Set-EmptyTestPort -Name example-malformed -Version 2 -PortsRoot D:\vcpkg\ports
PS D:\vcpkg> .\vcpkg.exe install example example-malformed --recurse
Computing installation plan...
The following packages are already installed:
    example:x64-windows@1
The following packages will be built and installed:
    example-malformed:x64-windows@2
Detecting compiler hash for triplet x64-windows...
Compiler found: C:/Program Files/Microsoft Visual Studio/2022/Enterprise/VC/Tools/MSVC/14.41.34120/bin/Hostx64/x64/cl.exe
Restored 1 package(s) from C:\Users\bion\AppData\Local\vcpkg\archives in 55.1 ms. Use --debug to see more details.
example:x64-windows is already installed
Installing 1/1 example-malformed:x64-windows@2...
Elapsed time to handle example-malformed:x64-windows: 3.61 ms
example-malformed:x64-windows package ABI: a566feeff21f116d018c0b8c231e090eb7e601962cbc1687ca5a3f64cab1500f
Total install time: 4.07 ms
PS D:\vcpkg> Set-EmptyTestPort -Name example-malformed -Version 2 -PortsRoot D:\vcpkg\ports -Malformed
PS D:\vcpkg> .\vcpkg.exe upgrade
warning: While determining which ports to upgrade, the following parse errors were encountered. Ports with parse errors won't be considered out of date and thus won't be upgraded. To consider these errors in the future, run `vcpkg upgrade` with --`no-keep-going`.
D:\vcpkg\ports\example-malformed\vcpkg.json:3:17: error: Trailing comma in an object
  on expression:   "version": "2",
                                 ^
The following packages will be rebuilt:
  * example:x64-windows@2
Additional packages (*) will be modified to complete this operation.
If you are sure you want to rebuild the above packages, run this command with the --no-dry-run option.
PS D:\vcpkg> .\vcpkg.exe upgrade --no-dry-run
warning: While determining which ports to upgrade, the following parse errors were encountered. Ports with parse errors won't be considered out of date and thus won't be upgraded. To consider these errors in the future, run `vcpkg upgrade` with --`no-keep-going`.
D:\vcpkg\ports\example-malformed\vcpkg.json:3:17: error: Trailing comma in an object
  on expression:   "version": "2",
                                 ^
The following packages will be rebuilt:
  * example:x64-windows@2
Additional packages (*) will be modified to complete this operation.
Detecting compiler hash for triplet x64-windows...
Compiler found: C:/Program Files/Microsoft Visual Studio/2022/Enterprise/VC/Tools/MSVC/14.41.34120/bin/Hostx64/x64/cl.exe
Restored 0 package(s) from C:\Users\bion\AppData\Local\vcpkg\archives in 898 us. Use --debug to see more details.
Removing 1/2 example:x64-windows
Elapsed time to handle example:x64-windows: 3.38 ms
Installing 2/2 example:x64-windows@2...
Building example:x64-windows@2...
-- Skipping post-build validation due to VCPKG_POLICY_EMPTY_PACKAGE
Stored binaries in 1 destinations in 75 ms.
Elapsed time to handle example:x64-windows: 131 ms
example:x64-windows package ABI: f1295c003a61ea194816d94d194f5202bd54bfcd7abcf018d6f53c1e732eca8b
Total install time: 135 ms
RESULTS
  example:x64-windows: REMOVED: 3.38 ms
  example:x64-windows: SUCCEEDED: 131 ms

SUMMARY FOR x64-windows
  SUCCEEDED: 1
  REMOVED: 1
PS D:\vcpkg> .\vcpkg.exe upgrade example-malformed
D:\vcpkg\ports\example-malformed\vcpkg.json:3:17: error: Trailing comma in an object
  on expression:   "version": "2",
                                 ^
PS D:\vcpkg>

…equired, as it has the _required semantics.

…ectory.

…pkg repo, and test the 'port name supplied' case.

…ider as that is the semantics of that function.

…n `spec`.

* Tests for named nonexistent ports having better error messages * Tests for named malformed ports printing the error message

BillyONeal · 2024-09-05T06:17:31Z

warning: While determining which ports to upgrade, the following parse errors were encountered. Ports with parse errors won't be considered out of date and thus won't be upgraded. To consider these errors in the future, run `vcpkg upgrade` with --`no-keep-going`.
D:\vcpkg\ports\example: error: example does not exist

Wait I missed this, that's wrong. Stand by...

BillyONeal · 2024-09-05T07:32:20Z

The 'hack' I did to make the baseline version not exist doesn't work :(

BillyONeal added 15 commits September 3, 2024 12:44

Rename RegistryEntry::try_load_port to RegistryEntry::try_load_port_r…

d4863b2

…equired, as it has the _required semantics.

Fix plumbing for try_load_port to not fail when passed not-a-port-dir…

fbdae71

…ectory.

Improve upgrade test to both use an empty port rather than a whole vc…

92cb12e

…pkg repo, and test the 'port name supplied' case.

Add a test for microsoft#1473

1d0d403

Rename get_control_file to get_control_file_required for PortfileProv…

2421225

…ider as that is the semantics of that function.

Plumb error handling of overlay-ports.

f56459d

Rename IVersionedPortfileProvider::get_control_file to _required.

58e8f2f

Rename IBaselineProvider::get_baseline_version to add _required.

9c18696

Demote get_port_versions to IFullVersionedPortfileProvider

28e072b

Rename get_port_versions to add _required.

8102a2a

Consistently use port_name when only a name is expected, rather tha…

e23e034

…n `spec`.

Add VersionedPackageSpec and adapt_to_string

166911f

Rename VersionedPortfileProviderImpl::load_control_file to add _required

997e70a

Add the optional parts to PortFileProvider and associated interfaces.

ca7ae5f

Actually fix printing diagnostics for all the upgrade things

15b5869

* Tests for named nonexistent ports having better error messages * Tests for named malformed ports printing the error message

BillyONeal marked this pull request as draft September 5, 2024 06:49

Actually test the better removal message.

45d92a7

BillyONeal closed this Sep 5, 2024

BillyONeal mentioned this pull request Sep 18, 2024

upgrade.ps1: Test without needing a whole git clone #1494

Merged

BillyONeal mentioned this pull request Oct 28, 2024

Fix try_load_port to match its documented interface. #1533

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Overhaul diagnostics in upgrade to not silently ignore parse failures#1486

Overhaul diagnostics in upgrade to not silently ignore parse failures#1486
BillyONeal wants to merge 16 commits intomicrosoft:mainfrom
BillyONeal:fix-upgrade-behavior

BillyONeal commented Sep 5, 2024

Uh oh!

BillyONeal commented Sep 5, 2024

Uh oh!

BillyONeal commented Sep 5, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

BillyONeal commented Sep 5, 2024

Uh oh!

BillyONeal commented Sep 5, 2024

Uh oh!

BillyONeal commented Sep 5, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant