Make comparison of installed files case-insensitive by vicroms · Pull Request #1483 · microsoft/vcpkg-tool

vicroms · 2024-08-27T19:47:18Z

Fixes the check that prevents two ports from installing a file with the same name by making the comparison case insensitive.

BillyONeal · 2024-08-28T21:57:46Z

src/vcpkg/commands.install.cpp

@@ -234,11 +234,23 @@ namespace vcpkg
            // The VS2015 standard library requires comparison operators of T and U
            // to also support comparison of T and T, and of U and U, due to debug checks.
 #if _MSC_VER <= 1910


We don't care about VS2015 so you can delete this if you want. Not worth resetting testing if that's the only change.

BillyONeal · 2024-08-28T21:59:00Z

azure-pipelines/e2e-ports/duplicate-file-a/vcpkg.json

+{
+  "name": "duplicate-file-a",
+  "version": "0.0.1"
+}


Please format-manifest these.

dg0yt · 2024-09-26T05:22:50Z

This change leads to a CI regression in https://dev.azure.com/vcpkg/public/_build/results?buildId=107274&view=results:

Android builds are on Linux with a case-sensitive filesystem.
tmx installs tmx.h
tmxparser installs Tmx.h.
CI now reports

 error: The following files are already installed in /mnt/vcpkg-ci/installed/arm64-android and are in conflict with tmxparser:arm64-android
 Installed by tmx:arm64-android  
 include/tmx.h

dg0yt · 2024-09-26T05:27:04Z

The CI run for the tool update, https://dev.azure.com/vcpkg/public/_build/results?buildId=107231&view=results, had a warning only:

warning: File /Users/vcpkg/Data/installed/arm64-osx/include/Tmx.h was already present and will be overwritten

BillyONeal · 2024-09-26T20:08:48Z

@dg0yt I don't believe this is a regression, I believe this was broken before and we are finally detecting that it is broken.

dg0yt · 2024-09-27T04:26:26Z

Last vcpkg.ci run now, https://dev.azure.com/vcpkg/public/_build/results?buildId=107290

x64-android, case-sensitive filesystem: No error.

x64-osx, case-insensitive filesystem: warning

warning: File /Users/vcpkg/Data/installed/arm64-osx/include/Tmx.h was already present and will be overwritten

But there is the error in that vcpkg.pr run. Only for two in three android triplets.!?

dg0yt · 2024-09-27T04:33:54Z

Hm, this PR here claims to "Make comparison of installed files case-insensitive" but actually changes "case_insensitive_ascii_less", not "case_insensitive_ascii_equals".

The particular case of interest has this input set (order undefined):

include/tmx.h
include/Tmx.h.in
include/Tmx.h

dg0yt · 2024-10-03T04:19:16Z

vcpkg.pr https://dev.azure.com/vcpkg/public/_build/results?buildId=107523 Errors:

influxdb-cxx:x64-osx:/include/Proxy.h
libproxy:x64-osx:/include/proxy.h

coin-or-cgl:x64-osx:/lib/libCgl.a
games101-cgl:x64-osx:/lib/libCGL.a
# (Debug variant differs by to 'd` suffix.)

All ports were built from source, due to the CMake update (?) and touching the baseline.

vcpkg.ci https://dev.azure.com/vcpkg/public/_build/results?buildId=107290 Only warnings about overwriting.

@dg0yt

This was first pointed out by @dg0yt in microsoft#1483 (comment) but somehow we missed that. It was discovered again by @ras0219-msft trying to write a post-build lint using the CI file lists of recent builds for new ports. `vcpkglib.h`: Remove SortedVector as the sort order we get here is not actually useful `commands.install.cpp`: * Change file_pack to InstalledFile with named values instead of first and second. (file_pack sounded like more than one file was involved) * Apply `std::move` a few places it was OK to do that. This also caused `extract_files_in_triplet` to get inlined into `build_list_of_installed_files`, its only caller. Note the call to `string::erase` and a move rather than making a copy of the suffix the `(const string&, size_t)` constructor. * Extract `check_for_install_conflicts` from `install_package` to make it clearer that `installed_files`, `package_files`, and `intersection` are dead after this check is complete (and thus those can be moved-from) * `stable_sort` to regroup by package name so that the sorted order within each group is preserved. Also added a comment for this. (I deleted that sort call entirely in a previous attempt because I initially read it as an attempt to sort the results of `set_intersection`, which are always already sorted. Missing that distinction is the whole reason I introduced `InstalledFile`) I want to fix that we are doing redundant `get_files_recursive` here but we don't actually have a `get_files_recursive_lexically_proximate`, only `_directories_` and `_regular_files_` versions, so will do that as a subsequent PR. Example with the unit test repro before and after: ```console PS C:\Dev\vcpkg> .\vcpkg.exe install --overlay-ports "C:\Dev\vcpkg-tool\azure-pipelines\e2e-ports" a-conflict b-conflict --binarysource clear Computing installation plan... The following packages will be built and installed: a-conflict:x64-windows@0 -- C:\Dev\vcpkg-tool\azure-pipelines\e2e-ports\a-conflict b-conflict:x64-windows@0 -- C:\Dev\vcpkg-tool\azure-pipelines\e2e-ports\b-conflict Detecting compiler hash for triplet x64-windows... Compiler found: C:/Program Files/Microsoft Visual Studio/2022/Enterprise/VC/Tools/MSVC/14.44.35207/bin/Hostx64/x64/cl.exe Installing 1/2 a-conflict:x64-windows@0... Building a-conflict:x64-windows@0... C:\Dev\vcpkg-tool\azure-pipelines\e2e-ports\a-conflict: info: installing overlay port from here -- Performing post-build validation Elapsed time to handle a-conflict:x64-windows: 111 ms a-conflict:x64-windows package ABI: 7d4b5f05d8fc195e77b84d5fb7722b9e23385392dd9f115632193da544222b9b Installing 2/2 b-conflict:x64-windows@0... Building b-conflict:x64-windows@0... C:\Dev\vcpkg-tool\azure-pipelines\e2e-ports\b-conflict: info: installing overlay port from here -- Performing post-build validation warning: File C:\Dev\vcpkg\installed\x64-windows\include/conflict-a-HEADER-only-MIXED.h was already present and will be overwritten warning: File C:\Dev\vcpkg\installed\x64-windows\include/CONFLICT-a-header-ONLY-mixed2.h was already present and will be overwritten Elapsed time to handle b-conflict:x64-windows: 112 ms b-conflict:x64-windows package ABI: 8010665bf1025c1dabed383a54f504013f467efc9ebbf1513380670586dbfd49 Total install time: 227 ms Installed contents are licensed to you by owners. Microsoft is not responsible for, nor does it grant any licenses to, third-party packages. Some packages did not declare an SPDX license. Check the `copyright` file for each package for more information about their licensing. a-conflict is header-only and can be used from CMake via: find_path(A_CONFLICT_INCLUDE_DIRS "CONFLICT-A-HEADER-ONLY-CAPS.h") target_include_directories(main PRIVATE ${A_CONFLICT_INCLUDE_DIRS}) b-conflict is header-only and can be used from CMake via: find_path(B_CONFLICT_INCLUDE_DIRS "CONFLICT-B-HEADER-ONLY-CAPS.h") target_include_directories(main PRIVATE ${B_CONFLICT_INCLUDE_DIRS}) All requested installations completed successfully in: 227 ms ```` Broken. ``` PS C:\Dev\vcpkg> .\vcpkg.exe x-ci-clean Clearing contents of C:\Dev\vcpkg\buildtrees Clearing contents of C:\Dev\vcpkg\installed Clearing contents of C:\Dev\vcpkg\packages PS C:\Dev\vcpkg> C:\Dev\vcpkg-tool\out\build\Win-x64-Debug-WithArtifacts\vcpkg.exe install --overlay-ports "C:\Dev\vcpkg-tool\azure-pipelines\e2e-ports" a-conflict b-conflict --binarysource clear Computing installation plan... The following packages will be built and installed: a-conflict:x64-windows@0 -- C:\Dev\vcpkg-tool\azure-pipelines\e2e-ports\a-conflict b-conflict:x64-windows@0 -- C:\Dev\vcpkg-tool\azure-pipelines\e2e-ports\b-conflict Detecting compiler hash for triplet x64-windows... Compiler found: C:/Program Files/Microsoft Visual Studio/2022/Enterprise/VC/Tools/MSVC/14.44.35207/bin/Hostx64/x64/cl.exe Installing 1/2 a-conflict:x64-windows@0... Building a-conflict:x64-windows@0... C:\Dev\vcpkg-tool\azure-pipelines\e2e-ports\a-conflict: info: installing overlay port from here -- Performing post-build validation Elapsed time to handle a-conflict:x64-windows: 114 ms a-conflict:x64-windows package ABI: 96c3b79bd6770ff5944d733c4d3272d5d4d1042a1a7cba65d2677aae5847226f Installing 2/2 b-conflict:x64-windows@0... Building b-conflict:x64-windows@0... C:\Dev\vcpkg-tool\azure-pipelines\e2e-ports\b-conflict: info: installing overlay port from here -- Performing post-build validation error: The following files are already installed in C:/Dev/vcpkg/installed/x64-windows and are in conflict with b-conflict:x64-windows Installed by a-conflict:x64-windows: include/CONFLICT-a-header-ONLY-mixed.h include/CONFLICT-a-header-ONLY-mixed2.h Elapsed time to handle b-conflict:x64-windows: 102 ms Please ensure you're using the latest port files with `git pull` and `vcpkg update`. Then check for known issues at: https://github.com/microsoft/vcpkg/issues?q=is%3Aissue+is%3Aopen+in%3Atitle+b-conflict You can submit a new issue at: https://github.com/microsoft/vcpkg/issues/new?template=report-package-build-failure.md&title=%5Bb-conflict%5D+Build+error+on+x64-windows Include '[b-conflict] Build error' in your bug report title, the following version information in your bug description, and attach any relevant failure logs from above. vcpkg-tool version: 2999-12-31-unknownhash-debug vcpkg-scripts version: 6ecbbbdf31 2025-08-20 (5 days ago) ``` Also drive-by fixes the floating list being generated improperly and that more than "error" was colored red, and removes a few SortedVector uses that were immediately thrown away in test code.

@dg0yt

* Correctly detect case insensitive conflicts This was first pointed out by @dg0yt in #1483 (comment) but somehow we missed that. It was discovered again by @ras0219-msft trying to write a post-build lint using the CI file lists of recent builds for new ports. `vcpkglib.h`: Remove SortedVector as the sort order we get here is not actually useful `commands.install.cpp`: * Change file_pack to InstalledFile with named values instead of first and second. (file_pack sounded like more than one file was involved) * Apply `std::move` a few places it was OK to do that. This also caused `extract_files_in_triplet` to get inlined into `build_list_of_installed_files`, its only caller. Note the call to `string::erase` and a move rather than making a copy of the suffix the `(const string&, size_t)` constructor. * Extract `check_for_install_conflicts` from `install_package` to make it clearer that `installed_files`, `package_files`, and `intersection` are dead after this check is complete (and thus those can be moved-from) * `stable_sort` to regroup by package name so that the sorted order within each group is preserved. Also added a comment for this. (I deleted that sort call entirely in a previous attempt because I initially read it as an attempt to sort the results of `set_intersection`, which are always already sorted. Missing that distinction is the whole reason I introduced `InstalledFile`) I want to fix that we are doing redundant `get_files_recursive` here but we don't actually have a `get_files_recursive_lexically_proximate`, only `_directories_` and `_regular_files_` versions, so will do that as a subsequent PR. Example with the unit test repro before and after: ```console PS C:\Dev\vcpkg> .\vcpkg.exe install --overlay-ports "C:\Dev\vcpkg-tool\azure-pipelines\e2e-ports" a-conflict b-conflict --binarysource clear Computing installation plan... The following packages will be built and installed: a-conflict:x64-windows@0 -- C:\Dev\vcpkg-tool\azure-pipelines\e2e-ports\a-conflict b-conflict:x64-windows@0 -- C:\Dev\vcpkg-tool\azure-pipelines\e2e-ports\b-conflict Detecting compiler hash for triplet x64-windows... Compiler found: C:/Program Files/Microsoft Visual Studio/2022/Enterprise/VC/Tools/MSVC/14.44.35207/bin/Hostx64/x64/cl.exe Installing 1/2 a-conflict:x64-windows@0... Building a-conflict:x64-windows@0... C:\Dev\vcpkg-tool\azure-pipelines\e2e-ports\a-conflict: info: installing overlay port from here -- Performing post-build validation Elapsed time to handle a-conflict:x64-windows: 111 ms a-conflict:x64-windows package ABI: 7d4b5f05d8fc195e77b84d5fb7722b9e23385392dd9f115632193da544222b9b Installing 2/2 b-conflict:x64-windows@0... Building b-conflict:x64-windows@0... C:\Dev\vcpkg-tool\azure-pipelines\e2e-ports\b-conflict: info: installing overlay port from here -- Performing post-build validation warning: File C:\Dev\vcpkg\installed\x64-windows\include/conflict-a-HEADER-only-MIXED.h was already present and will be overwritten warning: File C:\Dev\vcpkg\installed\x64-windows\include/CONFLICT-a-header-ONLY-mixed2.h was already present and will be overwritten Elapsed time to handle b-conflict:x64-windows: 112 ms b-conflict:x64-windows package ABI: 8010665bf1025c1dabed383a54f504013f467efc9ebbf1513380670586dbfd49 Total install time: 227 ms Installed contents are licensed to you by owners. Microsoft is not responsible for, nor does it grant any licenses to, third-party packages. Some packages did not declare an SPDX license. Check the `copyright` file for each package for more information about their licensing. a-conflict is header-only and can be used from CMake via: find_path(A_CONFLICT_INCLUDE_DIRS "CONFLICT-A-HEADER-ONLY-CAPS.h") target_include_directories(main PRIVATE ${A_CONFLICT_INCLUDE_DIRS}) b-conflict is header-only and can be used from CMake via: find_path(B_CONFLICT_INCLUDE_DIRS "CONFLICT-B-HEADER-ONLY-CAPS.h") target_include_directories(main PRIVATE ${B_CONFLICT_INCLUDE_DIRS}) All requested installations completed successfully in: 227 ms ```` Broken. ``` PS C:\Dev\vcpkg> .\vcpkg.exe x-ci-clean Clearing contents of C:\Dev\vcpkg\buildtrees Clearing contents of C:\Dev\vcpkg\installed Clearing contents of C:\Dev\vcpkg\packages PS C:\Dev\vcpkg> C:\Dev\vcpkg-tool\out\build\Win-x64-Debug-WithArtifacts\vcpkg.exe install --overlay-ports "C:\Dev\vcpkg-tool\azure-pipelines\e2e-ports" a-conflict b-conflict --binarysource clear Computing installation plan... The following packages will be built and installed: a-conflict:x64-windows@0 -- C:\Dev\vcpkg-tool\azure-pipelines\e2e-ports\a-conflict b-conflict:x64-windows@0 -- C:\Dev\vcpkg-tool\azure-pipelines\e2e-ports\b-conflict Detecting compiler hash for triplet x64-windows... Compiler found: C:/Program Files/Microsoft Visual Studio/2022/Enterprise/VC/Tools/MSVC/14.44.35207/bin/Hostx64/x64/cl.exe Installing 1/2 a-conflict:x64-windows@0... Building a-conflict:x64-windows@0... C:\Dev\vcpkg-tool\azure-pipelines\e2e-ports\a-conflict: info: installing overlay port from here -- Performing post-build validation Elapsed time to handle a-conflict:x64-windows: 114 ms a-conflict:x64-windows package ABI: 96c3b79bd6770ff5944d733c4d3272d5d4d1042a1a7cba65d2677aae5847226f Installing 2/2 b-conflict:x64-windows@0... Building b-conflict:x64-windows@0... C:\Dev\vcpkg-tool\azure-pipelines\e2e-ports\b-conflict: info: installing overlay port from here -- Performing post-build validation error: The following files are already installed in C:/Dev/vcpkg/installed/x64-windows and are in conflict with b-conflict:x64-windows Installed by a-conflict:x64-windows: include/CONFLICT-a-header-ONLY-mixed.h include/CONFLICT-a-header-ONLY-mixed2.h Elapsed time to handle b-conflict:x64-windows: 102 ms Please ensure you're using the latest port files with `git pull` and `vcpkg update`. Then check for known issues at: https://github.com/microsoft/vcpkg/issues?q=is%3Aissue+is%3Aopen+in%3Atitle+b-conflict You can submit a new issue at: https://github.com/microsoft/vcpkg/issues/new?template=report-package-build-failure.md&title=%5Bb-conflict%5D+Build+error+on+x64-windows Include '[b-conflict] Build error' in your bug report title, the following version information in your bug description, and attach any relevant failure logs from above. vcpkg-tool version: 2999-12-31-unknownhash-debug vcpkg-scripts version: 6ecbbbdf31 2025-08-20 (5 days ago) ``` Also drive-by fixes the floating list being generated improperly and that more than "error" was colored red, and removes a few SortedVector uses that were immediately thrown away in test code.

vicroms added 2 commits August 27, 2024 12:43

compare case insensitive

60f44d2

Add tests

39608a7

vicroms force-pushed the error-on-file-overwrite branch 2 times, most recently from 6f82d08 to 97098bd Compare August 27, 2024 22:27

Add e2e test

88a2370

vicroms force-pushed the error-on-file-overwrite branch from 97098bd to 88a2370 Compare August 27, 2024 22:37

vicroms marked this pull request as ready for review August 27, 2024 22:57

BillyONeal reviewed Aug 28, 2024

View reviewed changes

BillyONeal added 2 commits August 28, 2024 15:00

Merge remote-tracking branch 'origin/main' into error-on-file-overwrite

80d031c

Format-manifest and remove VS2015 workarounds.

f6f2062

BillyONeal approved these changes Aug 28, 2024

View reviewed changes

BillyONeal enabled auto-merge (squash) August 28, 2024 22:16

BillyONeal merged commit b8b3c1b into microsoft:main Aug 28, 2024

dg0yt mentioned this pull request Sep 26, 2024

[boost] Update to 1.86.0 microsoft/vcpkg#40465

Merged

This was referenced Oct 5, 2024

Update macOS to 2024-10-02. microsoft/vcpkg#41307

Closed

Ensure that public headers are prefixed with 'InfluxDB". offa/influxdb-cxx#263

Closed

[influxdb-cxx] Move headers due to conflict with Proxy.h microsoft/vcpkg#41452

Merged

vicroms deleted the error-on-file-overwrite branch April 14, 2025 18:20

BillyONeal mentioned this pull request Aug 26, 2025

Correctly detect case insensitive conflicts #1768

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make comparison of installed files case-insensitive#1483

Make comparison of installed files case-insensitive#1483
BillyONeal merged 5 commits intomicrosoft:mainfrom
vicroms:error-on-file-overwrite

vicroms commented Aug 27, 2024 •

edited

Loading

Uh oh!

BillyONeal Aug 28, 2024

Uh oh!

BillyONeal Aug 28, 2024

Uh oh!

dg0yt commented Sep 26, 2024

Uh oh!

dg0yt commented Sep 26, 2024

Uh oh!

BillyONeal commented Sep 26, 2024

Uh oh!

dg0yt commented Sep 27, 2024

Uh oh!

dg0yt commented Sep 27, 2024

Uh oh!

dg0yt commented Oct 3, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

vicroms commented Aug 27, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

BillyONeal Aug 28, 2024

Choose a reason for hiding this comment

Uh oh!

BillyONeal Aug 28, 2024

Choose a reason for hiding this comment

Uh oh!

dg0yt commented Sep 26, 2024

Uh oh!

dg0yt commented Sep 26, 2024

Uh oh!

BillyONeal commented Sep 26, 2024

Uh oh!

dg0yt commented Sep 27, 2024

Uh oh!

dg0yt commented Sep 27, 2024

Uh oh!

dg0yt commented Oct 3, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

vicroms commented Aug 27, 2024 •

edited

Loading