Our CI relies on a global vcpkg installation for each platform. Build jobs happen in unique directories where we run vcpkg install to deal with any necessary dependencies. After the merging of #11757 we started using manifest files, which worked perfectly (thanks @strega-nil!).
Describe the bug
The manifest PR (#11757) added a filesystem lock (.vcpkg-root) which multiple build scripts might try to acquire simultaneously. The current timeout of 1.5-ish seconds is not enough since the lock seems acquired until the vcpkg install command finishes. See:
|
System::printf("Waiting to take filesystem lock on %s...\n", path.u8string()); |
|
auto wait = std::chrono::milliseconds(100); |
|
// waits, at most, a second and a half. |
|
while (wait < std::chrono::milliseconds(1000)) |
|
{ |
|
std::this_thread::sleep_for(wait); |
|
if (system_try_take_file_lock() || ec) |
|
{ |
|
return res; |
|
} |
|
wait *= 2; |
|
} |
Calling
vcpkg install in directories with manifest files shouldn't modify the global
%VCPKG_ROOT%/installed folder (afaik!), but place everything in
./vcpkg_installed instead. Thus, it seems unnecessary to lock
vcpkg globally for minutes at a time.
Environment
To reproduce
Steps to reproduce the behavior:
- Simultaneously run two instances of
./vcpkg install in different repositories.
- One of the instances will fail to acquire the filesytem lock on .vcpkg-root:
$ vcpkg install
Waiting to take filesystem lock on /Users/ci/vcpkg/.vcpkg-root...
Failed to take the filesystem lock on /Users/ci/vcpkg/.vcpkg-root:
Resource busy
Expected behavior
I think there's two possible solutions to this problem (or even a combination of both):
- Adding
--lock-timeout <seconds> (or similar) argument for any vcpkg command that might try to take the lock, so that multiple vcpkg install invocations could execute in series for a sufficiently high timeout.
- Only acquire the lock when writing to files under
%VCPKG_ROOT%. When installing dependencies from vcpkg.json manifest files, everything will be written to vcpkg_installed folder, so there's no need to lock vcpkg globally while that happens.
Our CI relies on a global vcpkg installation for each platform. Build jobs happen in unique directories where we run
vcpkg installto deal with any necessary dependencies. After the merging of #11757 we started using manifest files, which worked perfectly (thanks @strega-nil!).Describe the bug
The manifest PR (#11757) added a filesystem lock (.vcpkg-root) which multiple build scripts might try to acquire simultaneously. The current timeout of 1.5-ish seconds is not enough since the lock seems acquired until the
vcpkg installcommand finishes. See:vcpkg/toolsrc/src/vcpkg/base/files.cpp
Lines 932 to 943 in a28bfe7
Calling
vcpkg installin directories with manifest files shouldn't modify the global%VCPKG_ROOT%/installedfolder (afaik!), but place everything in./vcpkg_installedinstead. Thus, it seems unnecessary to lock vcpkg globally for minutes at a time.Environment
To reproduce
Steps to reproduce the behavior:
./vcpkg installin different repositories.Expected behavior
I think there's two possible solutions to this problem (or even a combination of both):
--lock-timeout <seconds>(or similar) argument for any vcpkg command that might try to take the lock, so that multiplevcpkg installinvocations could execute in series for a sufficiently high timeout.%VCPKG_ROOT%. When installing dependencies from vcpkg.json manifest files, everything will be written tovcpkg_installedfolder, so there's no need to lock vcpkg globally while that happens.