Avoid cloning cached registries#25
Merged
christopher-dG merged 13 commits intojulia-actions:mainfrom Jan 5, 2024
Merged
Conversation
Contributor
Author
|
Uncovered a performance issue while testing this against a real workflow. The |
Contributor
Author
|
Note that Julia seems to be keeping track of registry URLs it has updated and there isn't any symlink specific logic: ❯ mkdir -p /tmp/julia-depot2/registries
❯ cd /tmp/julia-depot2/registries
❯ export JULIA_DEPOT_PATH=/tmp/julia-depot2
❯ git clone https://github.com/JuliaRegistries/General General1
Cloning into 'General1'...
remote: Enumerating objects: 792469, done.
remote: Counting objects: 100% (12926/12926), done.
remote: Compressing objects: 100% (492/492), done.
remote: Total 792469 (delta 12567), reused 12681 (delta 12433), pack-reused 779543
Receiving objects: 100% (792469/792469), 229.36 MiB | 29.74 MiB/s, done.
Resolving deltas: 100% (432253/432253), done.
Updating files: 100% (42240/42240), done.
❯ git clone https://github.com/JuliaRegistries/General General2
Cloning into 'General2'...
remote: Enumerating objects: 792469, done.
remote: Counting objects: 100% (13033/13033), done.
remote: Compressing objects: 100% (846/846), done.
remote: Total 792469 (delta 12363), reused 12761 (delta 12183), pack-reused 779436
Receiving objects: 100% (792469/792469), 230.05 MiB | 34.76 MiB/s, done.
Resolving deltas: 100% (432061/432061), done.
Updating files: 100% (42240/42240), done.
❯ julia -e 'using Pkg; Pkg.Registry.update()'
Updating registry at `/tmp/julia-depot2/registries/General1`
Updating git-repo `https://github.com/JuliaRegistries/General` |
Contributor
Author
|
Validated that these changes work in a private workflow. |
IanButterworth
approved these changes
Jan 4, 2024
Contributor
Author
|
I've validated the merge commit works in a private workflow |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Depends on:
Fixes:
registrywhen loaded from cache #19 (when usingjulia-actions/cachebefore this action withcache-registries: true. Enabled by default injulia-actions/cache@v1.5.0)Moves away from cloning to a temporary directory in order to interact better with
julia-actions/cachewhen registry cloning is enabled. The key component here is to use a consistent name of the registry such that we know the repo has already been downloaded and we can skip an additional clone. In order to do this I've opted to clone the registry using the repo name into the depot registry directory. As some registries can have different registry names from repo names I've added a symlink into the depot registry as well such that both names co-exist.Luckily, Julia is smart enough to avoid updating both registries. For example: