Skip to content

treewide: use mirror URLs where available#144825

Merged
lovesegfault merged 1 commit intoNixOS:masterfrom
alyssais:treewide-mirror
Nov 6, 2021
Merged

treewide: use mirror URLs where available#144825
lovesegfault merged 1 commit intoNixOS:masterfrom
alyssais:treewide-mirror

Conversation

@alyssais
Copy link
Copy Markdown
Member

@alyssais alyssais commented Nov 6, 2021

Candidates identified with

git grep -F "$(grep -v '#' pkgs/build-support/fetchurl/mirrors.nix |
	grep -v nixos.org | grep :// | cut -d '"' -f 2)"

and then manually reviewed and tested.

Motivation for this change
Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 21.11 Release Notes (or backporting 21.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Candidates identified with

	git grep -F "$(grep -v '#' pkgs/build-support/fetchurl/mirrors.nix |
		grep -v nixos.org | grep :// | cut -d '"' -f 2)"

and then manually reviewed and tested.
@alyssais alyssais added the 6.topic: fetch Fetchers (e.g. fetchgit, fetchsvn, ...) label Nov 6, 2021
@github-actions github-actions bot added 6.topic: ocaml OCaml is a general-purpose, high-level, multi-paradigm programming language. and removed 6.topic: fetch Fetchers (e.g. fetchgit, fetchsvn, ...) labels Nov 6, 2021
@ofborg ofborg bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. labels Nov 6, 2021
@lovesegfault lovesegfault merged commit 58a1813 into NixOS:master Nov 6, 2021
@alyssais alyssais deleted the treewide-mirror branch November 6, 2021 10:06
pbsds added a commit to pbsds/nixpkgs that referenced this pull request Mar 3, 2025
repeat of NixOS#144825, part of NixOS#346453, should be 0 rebuilds

Madie using

```
export NIXPKGS_ALLOW_UNFREE=1
export NIXPKGS_ALLOW_INSECURE=1
export NIXPKGS_ALLOW_BROKEN=1

git reset
git restore .

nix eval -f pkgs/build-support/fetchurl/mirrors.nix --json |
    jq '
        to_entries[]
            | .key as $key
            | .value[]
            | @sh "rg \(.) -l pkgs/ -tnix | xe sd \(.) mirror://\($key)/"
        ' -r |
    bash -x

git restore -- pkgs/build-support/fetchurl/mirrors.nix
git restore -- pkgs/applications/networking/browsers/
git restore -- pkgs/development/haskell-modules/

test -s packages.json || ( set -x;
    time nix-env --extra-experimental-features no-url-literals --option system x86_64-linux -f ./. -qaP --json --meta --show-trace --no-allow-import-from-derivation > packages.json
)

MODIFIED="$(git ls-files --modified)"

jq <packages.json 'to_entries[] | select(.value.meta.position==null|not) | "\(.key)\t\(.value.meta.position)"' -r |
    sed -e "s#\t$(realpath .)/#\t#" |
    sed -e 's#:\([0-9]*\)$#\t\1#' |
    grep --fixed-strings "$MODIFIED" |
    while read attrpath position col; do
        if test -z $(git diff -- "$position"); then
            continue
        fi
        if nix-build . -A "$attrpath".src \
        && nix-build . -A "$attrpath".src --check \
        && nix-build . -A "$attrpath".patches \
        && nix-build . -A "$attrpath".patches --check
        then
            git add -- "$position"
        else
            git reset -- "$position"
            git restore -- "$position"
        fi
    done
```

Then I verified the FOD hashes for each `--patch` i accepted.

FOD hash verification:

```
$ nix-prefetch-url $(nix eval -f default.nix activemq.src.url --raw | tee >( cat >&2; echo >&2 )) | xargs nix hash to-sri --type sha256
mirror://apache/activemq/6.1.5/apache-activemq-6.1.5-bin.tar.gz
path is '/nix/store/gmh0hzsglbfyrcqkjiyf0kvwcyqqgz5p-apache-activemq-6.1.5-bin.tar.gz'
sha256-JrLLSm6+Be1vSBTTryRcZfbiGK0PrmJ/pM2uYnGuN9E=

$ nix-prefetch-url $(nix eval -f default.nix libxmlxx5.src.url --raw | tee >( cat >&2; echo >&2 )) | xargs nix hash to-sri --type sha256
mirror://gnome/sources/libxml++/5.4/libxml++-5.4.0.tar.xz
path is '/nix/store/8zy5makss1dcijprf5bilz5qqz59xg2p-libxml++-5.4.0.tar.xz'
sha256-6aI8Q2aGqUaY0hOOa8uvhJEh1jv6D1DcNP77/XlWaEg=
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: ocaml OCaml is a general-purpose, high-level, multi-paradigm programming language. 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants