Skip to content

Add nix flake prefetch-inputs command#13565

Merged
edolstra merged 1 commit intomasterfrom
prefetch-inputs
Aug 21, 2025
Merged

Add nix flake prefetch-inputs command#13565
edolstra merged 1 commit intomasterfrom
prefetch-inputs

Conversation

@edolstra
Copy link
Copy Markdown
Member

Motivation

This command fetches all inputs of a flake in parallel. This can be a lot faster than the serialized on-demand fetching during regular flake evaluation. The downside is that it may fetch inputs that aren't normally used.

Example runtime for

$ chmod -R u+w /tmp/nix2; rm -rf /tmp/nix2; rm ~/.cache/nix/fetcher-cache-v3.sqlite*; rm -rf ~/.cache/nix/tarball-cache/ ~/.cache/nix/gitv3/; time nix flake prefetch-inputs --store /tmp/nix2 https://api.flakehub.com/f/pinned/informalsystems/cosmos.nix/0.3.0/018ce9ed-d0be-7ce5-81b6-a3c6e3ae1187/source.tar.gz

with http-connections = 1:

real    4m11.859s
user    2m6.931s
sys     0m25.619s

and http-connections = 25 (the default):

real    0m57.146s
user    2m49.506s
sys     0m36.008s

Based on DeterminateSystems#127.

Context


Add 👍 to pull requests you find important.

The Nix maintainer team uses a GitHub project board to schedule and track reviews.

This command fetches all inputs of a flake in parallel.

Example runtime for

  $ chmod -R u+w /tmp/nix2; rm -rf /tmp/nix2; rm ~/.cache/nix/fetcher-cache-v3.sqlite*; rm -rf ~/.cache/nix/tarball-cache/ ~/.cache/nix/gitv3/; time nix flake prefetch-inputs --store /tmp/nix2 https://api.flakehub.com/f/pinned/informalsystems/cosmos.nix/0.3.0/018ce9ed-d0be-7ce5-81b6-a3c6e3ae1187/source.tar.gz

with http-connections = 1:

real    4m11.859s
user    2m6.931s
sys     0m25.619s

and http-connections = 25 (the default):

real    0m57.146s
user    2m49.506s
sys     0m36.008s
@github-actions github-actions bot added documentation new-cli Relating to the "nix" command labels Jul 28, 2025
{
auto flake = lockFlake();

ThreadPool pool{fileTransferSettings.httpConnections};
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using http-connections is weird if your inputs aren't all tarballs.
Looks like the setting needs to be generalized, or we should apply a different strategy.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not perfect, but I didn't want to add another setting just for this command. http-connections is the closest to want we want to express here (i.e. an operation that is network-bound).

@roberth roberth added this to Nix team Aug 19, 2025
@github-project-automation github-project-automation bot moved this to Triage in Nix team Aug 19, 2025
@edolstra
Copy link
Copy Markdown
Member Author

Team discussion:

  • Open an issue for creating a separate setting.
  • Otherwise looks good.

@edolstra edolstra removed this from Nix team Aug 20, 2025
@nixos-discourse
Copy link
Copy Markdown

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2025-08-20-nix-team-meeting-minutes-242-241/68245/1

@edolstra edolstra merged commit 3826d51 into master Aug 21, 2025
27 checks passed
@edolstra edolstra deleted the prefetch-inputs branch August 21, 2025 10:30
@nixos-discourse
Copy link
Copy Markdown

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nix-2-31-0-released/68465/1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation new-cli Relating to the "nix" command

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants