Skip to content

fetchFromGitHub: use passthru instead of // to expose rev and friends; fetchgit: support adding extra passthru values#370432

Closed
TomaSajt wants to merge 2 commits intoNixOS:masterfrom
TomaSajt:fetchgithub-stuff
Closed

fetchFromGitHub: use passthru instead of // to expose rev and friends; fetchgit: support adding extra passthru values#370432
TomaSajt wants to merge 2 commits intoNixOS:masterfrom
TomaSajt:fetchgithub-stuff

Conversation

@TomaSajt
Copy link
Copy Markdown
Contributor

@TomaSajt TomaSajt commented Jan 3, 2025

Closes #370418

Other than the main change about passing some values with passthru instead of // there are some minor extra changes which should not cause any issues:

  • Rename passthruAttrs to extraFetcherAttrs to not have two meanings of passthru in the same file.
  • inherit name is now a bit earlier in the construction of the combined attrset.
    Since we don't filter name out from @attrs the // extraFetcherAttrs will automatically include the name value if it is passed to the function.
    This means the only thing inherit name; does is that it sets the default value.
    I could replace inherit name; with name = "source"; and everything would be the same.
    • Should I do this? Then we could remove the name ? "source" from the function args

Note: I did not update fetchFromGitLab and other fetchers.

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@github-actions github-actions bot added the 6.topic: fetch Fetchers (e.g. fetchgit, fetchsvn, ...) label Jan 3, 2025
@nix-owners nix-owners bot requested a review from philiptaron January 3, 2025 00:26
@github-actions github-actions bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. labels Jan 3, 2025
Copy link
Copy Markdown
Contributor

@philiptaron philiptaron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really like this direction, and I'd love to see the other fetchers brought into line. I have a few questions about order of application and expectations from these derivation factory functions. They're more curious than hard opinions.

gitRepoUrl = url;
inherit tag;
};
} // passthru;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really like how this allows the caller to provide values for passthru. I don't like how the caller can override gitRepoUrl and tag, since I can't really come up with good usecases for those that aren't just the user shooting themselves in the foot. What do you think?

The other way would let the user provide whatever, but would assure users that gitRepoUrl and tag would be canonical from this function... so this:

passthru = passthru // {
  gitRepoUrl = url;
  inherit tag;
};

I could also see rev (aka revWithTag) being on this list for similar reasons.

Copy link
Copy Markdown
Contributor Author

@TomaSajt TomaSajt Jan 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My personal opinion is this: if the consumer sets a value, it should be set or respected in some way (e.g. override default value, append to a default list, etc.). A NOOP is much more surprising than a broken package.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I also don't like NOOP. Maybe I'm trying to construct a static language out of a dynamic one, but I do like this:

optionalAttrs =
if (builtins.intersectAttrs attrs forcedAttrs == { }) then
builtins.removeAttrs attrs [ "replacements" ]
else
throw "Passing any of ${builtins.concatStringsSep ", " (builtins.attrNames forcedAttrs)} to replaceVarsWith is not supported.";

The idea being that the parameter is there to supplement the set of values in passthru but not override values which are expected to have a definite meaning computed in the function.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gitRepoUrl is IIRC used in nix-update, but i believe you should be allowed to clobber it if you really want to.

inherit tag rev deepClone fetchSubmodules sparseCheckout fetchLFS; url = gitRepoUrl;
inherit tag rev deepClone fetchSubmodules sparseCheckout fetchLFS;
url = gitRepoUrl;
passthru = { inherit owner repo; } // passthru;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same update ordering question here.

passthru = {
inherit gitRepoUrl;
};
passthru = { inherit owner repo tag rev gitRepoUrl; } // passthru;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And here.

@philiptaron philiptaron requested review from Atemu and pbsds January 3, 2025 01:35
@philiptaron
Copy link
Copy Markdown
Contributor

Adding @Atemu and @pbsds, both of whom have been in the zone recently, for their ideas and guidance also.

@TomaSajt
Copy link
Copy Markdown
Contributor Author

TomaSajt commented Jan 3, 2025

Also note that even if this fixes calling overrideAttrs on derivation, you still won't be able to utilize the fixed point finalAttrs feature, since it doesn't use it internally. You would only be able to change values non-reactively.


Sidenote: I really wish the main attrset of stdenv.mkDerivation acted like passthru by default (a.k.a. not evaluated and not passed to the builder as shell variables), as it would make supporting fixed point overrideAttrs stuff much easier, (and maybe even be able to replace .override)

@pbsds
Copy link
Copy Markdown
Member

pbsds commented Jan 4, 2025

This does some of what I drafted as patch#2 in Mic92/nix-update#281 and forgot about 👍

One property I'd like for this PR to achieve is for src.passthru to be the same for both forceFetchGit=false; and forceFetchGit=true;

@TomaSajt
Copy link
Copy Markdown
Contributor Author

TomaSajt commented Jan 4, 2025

One property I'd like for this PR to achieve is for src.passthru to be the same for both forceFetchGit=false; and forceFetchGit=true;

We could do that, though I'm not 100% it's for the best. passthru is only used here to expose variables through drv.* and not through drv.passthru.*.


What's interesting too is the fact that we could use .overrideAttrs { rev = "new-rev"; } on fetchgit-based stuff and it would work, since rev is only used as a bash variable.
However, if we also passed passthru.rev it would still allow the above behaviour, but when getting .rev of the derivation, it would get the passthru.rev value, which is still the old value.

(I'm not sure if you'd ever want to .overrideAttrs the rev value since there's a better .override interface, but still good to keep in mind)

@pbsds
Copy link
Copy Markdown
Member

pbsds commented Jan 4, 2025

Ah, i didn't consider that. Your approach is sound!

@SuperSandro2000
Copy link
Copy Markdown
Member

also see #294329

@TomaSajt
Copy link
Copy Markdown
Contributor Author

Superseded by #456751

@TomaSajt TomaSajt closed this Nov 28, 2025
@TomaSajt TomaSajt deleted the fetchgithub-stuff branch January 20, 2026 18:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

2.status: merge conflict This PR has merge conflicts with the target branch 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md 6.topic: fetch Fetchers (e.g. fetchgit, fetchsvn, ...) 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fetchFromGitHub: use passthru instead of // to support overrideAttrs

5 participants