[vcpkg] Add experimental $X_VCPKG_ASSET_SOURCES for source caching#13639
[vcpkg] Add experimental $X_VCPKG_ASSET_SOURCES for source caching#13639ras0219-msft merged 11 commits intomicrosoft:masterfrom
Conversation
|
#12532 tries to achieve similar things, might want to coordinate with Jack |
|
We may need to provide a way for mirror maintainers so that they can easily download all necessary source packages to their computers. |
1a6498c to
48623b1
Compare
|
The current implementation is intended to upload all downloaded binaries to the mirror (using HTTP PUT) -- this obviously needs to be configurable in a productized version. |
ddf6599 to
465cb16
Compare
465cb16 to
f3bd0cb
Compare
|
Fixes #11040. |
* [vcpkg] Add experimental x-azblob binary provider * [vcpkg] Test azblob storage provider in CI * [vcpkg] Address some CR comments from #13639 * [vcpkg] Fixup azure-pipelines * [vcpkg] Fix regression where the downloaded package is purged before decompressing * [vcpkg] Further refactor vcpkg::Downloads * [vcpkg] Enable OSX for x-azblob testing * [vcpkg] Reduce diff against master * [vcpkg] Extract Downloads::details::split_uri_view * [vcpkg] Address PR comments * [vcpkg] Add testing and metrics for x-azblob * [vcpkg] Add docs for x-azblob This includes a note that it is currently experimental * [vcpkg] Address CR comments * [vcpkg] Revert pipeline changes except OSX to minimize disruption Co-authored-by: Robert Schumacher <roschuma@microsoft.com> Co-authored-by: Billy Robert O'Neal III <bion@microsoft.com>
9dbd341 to
9b8824b
Compare
|
I very much hope that this PR can be deployed within this year. |
|
Any update for this draft? It's such a great feature! @BillyONeal |
|
@theidexisted Robert (@ras0219) has been busy with other things. |
|
Cross posting from #13941 (comment):
This is the primary issue holding up this PR; we need to either:
|
c7364b2 to
28c3c0a
Compare
* [vcpkg] Add experimental x-azblob binary provider * [vcpkg] Test azblob storage provider in CI * [vcpkg] Address some CR comments from microsoft#13639 * [vcpkg] Fixup azure-pipelines * [vcpkg] Fix regression where the downloaded package is purged before decompressing * [vcpkg] Further refactor vcpkg::Downloads * [vcpkg] Enable OSX for x-azblob testing * [vcpkg] Reduce diff against master * [vcpkg] Extract Downloads::details::split_uri_view * [vcpkg] Address PR comments * [vcpkg] Add testing and metrics for x-azblob * [vcpkg] Add docs for x-azblob This includes a note that it is currently experimental * [vcpkg] Address CR comments * [vcpkg] Revert pipeline changes except OSX to minimize disruption Co-authored-by: Robert Schumacher <roschuma@microsoft.com> Co-authored-by: Billy Robert O'Neal III <bion@microsoft.com>
|
Three errors in the previous CI:
|
950d15d to
7d5a228
Compare
|
Looks all passed. |
strega-nil-ms
left a comment
There was a problem hiding this comment.
LGTM! Thanks @ras0219
|
FYI, my project is seeing some build errors after this merge that look possibly related. The error log is here: googleapis/google-cloud-cpp#6699 |
|
I have problems with downloads after this merge, closely related to the old (and somehow closed but not fixed #7207) I will revive #18171 hoping for merge |
Was fixed by #18283 right now |
| endforeach() | ||
| endif() | ||
| vcpkg_execute_in_download_mode( | ||
| COMMAND "$ENV{VCPKG_COMMAND}" x-download "${downloaded_file_path}" "${vcpkg_download_distfile_SHA512}" ${urls} ${request_headers} --debug |
There was a problem hiding this comment.
@ras0219-msft This change introduced the use of the x-download vcpkg tool command which was not availabe in version 2021-01-13-d67989bce1043b98092ac45996a8230a059a2d7e of the tool. This already created a lot of issues, e.g. #18676. IMO this change needs to be complemented by use of
vcpkg_minimum_required(VERSION 2021-05-05)either here where the new command is used, or globally in ports.cmake where the minimum is 2021-01-13 at the moment.
(I don't want to send a PR blindly because it will rebuild the world.)
There was a problem hiding this comment.
while you are very kind in principle, experience has already demonstrated many times that already merged PRs are rarely seen by vcpkg team.
i’d open a new issue at minimum, or send a PR anyway at best.
There was a problem hiding this comment.
@cenit Not really, I think we should force the user to update vcpkg and download vcpkg binary using bootstrap.
There was a problem hiding this comment.
There were probably more than 10 issues related to x-download, and none was taken as a reason to demand a change.
I would send a PR if it could be discussed before having another world rebuild blocking the CI pipeline. The script audits are heavy enough.
So as a start, I prefer to have another comment close to the offending change, adressing author, assignees and reviewers which normally remain subscribed. An issue or PR can follow.
Work in progress; subject to change. Based on the strategy in #12532.
This PR merges several other PRs; prefer reviewing these individual pieces instead:
- [ ] [vcpkg_from_git] Add SHA512 argument, deprecate usage without argument. #13941Due to some significant design obstacles, I've cut support forvcpkg_from_git()from this initial implementation.This PR requires microsoft/vcpkg-tool#10