Skip to content

pnpm.fetchDeps: ensure consistent permissions, add versioning#422975

Merged
Scrumplex merged 4 commits intoNixOS:masterfrom
gepbird:pnpm-fetch-deps-2
Jul 14, 2025
Merged

pnpm.fetchDeps: ensure consistent permissions, add versioning#422975
Scrumplex merged 4 commits intoNixOS:masterfrom
gepbird:pnpm-fetch-deps-2

Conversation

@gepbird
Copy link
Copy Markdown
Contributor

@gepbird gepbird commented Jul 6, 2025

Fixes #422889
Closes #350063

Feedback is welcome about the versioning interface :)

This is very WIP as I have to go now, haven't even tested and may need to do changes for the configHook too

cc @jfraudeau

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • Nixpkgs 25.11 Release Notes (or backporting 25.05 Nixpkgs Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
  • NixOS 25.11 Release Notes (or backporting 25.05 NixOS Release notes)
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md, pkgs/README.md, maintainers/README.md and other contributing documentation in corresponding paths.

Add a 👍 reaction to pull requests you find important.

@gepbird gepbird requested review from Scrumplex and obreitwi July 6, 2025 17:06
@gepbird gepbird force-pushed the pnpm-fetch-deps-2 branch from 912d318 to ebfe10f Compare July 7, 2025 23:00
@nixpkgs-ci nixpkgs-ci bot added the 6.topic: nodejs Node.js is a free, open-source, cross-platform JavaScript runtime environment label Jul 7, 2025
@gepbird
Copy link
Copy Markdown
Contributor Author

gepbird commented Jul 7, 2025

Do we need to also version configHook? From the original issue I heard this is working fine for people. I'm just afraid in the future the configHook will only be compatible with a fetcherVersion <= x fetchDeps and we'd need a new version of configHook. And that that point it would be easier to keep them on the same version, what do you think @Scrumplex ?

(I'm still a bit short on time to do proper research)

@nixpkgs-ci nixpkgs-ci bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. labels Jul 7, 2025
@Scrumplex
Copy link
Copy Markdown
Member

What if we store fetcherVersion in $out (if greater than 1) so that we can apply logic in config hook for different fetcher versions

@gepbird
Copy link
Copy Markdown
Contributor Author

gepbird commented Jul 8, 2025

What if we store fetcherVersion in $out (if greater than 1) so that we can apply logic in config hook for different fetcher versions

Great idea, much better than having to duplicate the fetcher version and doing something like this:

nativeBuildInputs = [
  (pnpm_10.configHook { fetcherVersion = 2; })
];

# or this:
PNPM_FETCHER_VERSION = 2;

@gepbird gepbird force-pushed the pnpm-fetch-deps-2 branch from ebfe10f to e8f1627 Compare July 9, 2025 13:01
@nixpkgs-ci nixpkgs-ci bot added 6.topic: python Python is a high-level, general-purpose programming language. 8.has: documentation This PR adds or changes documentation 10.rebuild-linux: 101-500 This PR causes between 101 and 500 packages to rebuild on Linux. 10.rebuild-darwin: 11-100 This PR causes between 11 and 100 packages to rebuild on Darwin. and removed 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. labels Jul 9, 2025
Copy link
Copy Markdown
Member

@Scrumplex Scrumplex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, the first commit causes every pnpm package to fail evaluating. Perhaps we should first add the arg and default it to 1, then do the treewide change and then enforce setting the arg

@gepbird gepbird force-pushed the pnpm-fetch-deps-2 branch from e8f1627 to efe2575 Compare July 9, 2025 15:46
@gepbird gepbird changed the title pnpm.fetchDeps: ensure consistent hashes, add versioning pnpm.fetchDeps: ensure consistent permissions, add versioning Jul 9, 2025
@gepbird gepbird force-pushed the pnpm-fetch-deps-2 branch 2 times, most recently from 252fe9b to 4c5f683 Compare July 9, 2025 16:38
@gepbird gepbird marked this pull request as ready for review July 9, 2025 16:38
@nixpkgs-ci nixpkgs-ci bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label Jul 10, 2025
@gepbird gepbird force-pushed the pnpm-fetch-deps-2 branch from 4c5f683 to 764f78a Compare July 11, 2025 09:55
@ofborg ofborg bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Jul 11, 2025
@nixpkgs-ci nixpkgs-ci bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label Jul 12, 2025
@obreitwi
Copy link
Copy Markdown
Member

Looks good from my side 👍

❓ Out of curiosity: Did you already evaluate how many hashes would actually change if fetcherVersion was set to 2, already? I would not expect a lot.

Since the original issue seemed OS-dependent and only occurred under specific circumstances, I would have expected Hydra builds to have failed if wrong hashes were submitted in PRs.

@nixpkgs-ci nixpkgs-ci bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Jul 14, 2025
@Scrumplex Scrumplex merged commit 0eec779 into NixOS:master Jul 14, 2025
23 of 27 checks passed
@Scrumplex
Copy link
Copy Markdown
Member

I think this should be backported to 25.05. Because of the release notes change, this will have to be done manually.

Comment on lines +122 to +124
# NOTE: For reasons not yet fully understood, pnpm might create files with
# inconsistent permissions, for example inside the ubuntu-24.04
# github actions runner.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is probably related to the sandbox. What installed was used? The cachix one with the sanbox or another one? I think I've hit the same problem before, see NuschtOS/search@2ab7909

Copy link
Copy Markdown
Contributor Author

@gepbird gepbird Jul 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In #350063 nix-quick-install was used and the pnpm.fetchDeps generated result had 555 permissions on a ubuntu-24.04 runner and 444 permissions on ubuntu-22.04 runner (and their local env)

I'm not sure which runner was used in #422889, cc @jfraudeau @jmoggr

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, when we tried it with nix-quick-install we also got the issue. cachix/install-nix works. I think the major difference is the sandbox.

@gepbird gepbird deleted the pnpm-fetch-deps-2 branch July 14, 2025 12:08
@nixos-discourse
Copy link
Copy Markdown

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/breaking-changes-announcement-for-unstable/17574/86

@Scrumplex Scrumplex added the 8.has: port to stable This PR already has a backport to the stable release. label Jul 15, 2025
nanoyaki added a commit to nanoyaki/nanopkgs that referenced this pull request Jul 19, 2025
nanoyaki added a commit to nanoyaki/md2img that referenced this pull request Jul 19, 2025
nanoyaki added a commit to nanoyaki/nanopkgs that referenced this pull request Jul 19, 2025
@gepbird gepbird mentioned this pull request Jul 20, 2025
13 tasks
stackptr added a commit to stackptr/rc that referenced this pull request Jul 21, 2025
nanoyaki added a commit to nanoyaki/md2img that referenced this pull request Jul 22, 2025
@kachick kachick mentioned this pull request Oct 30, 2025
13 tasks
@snue snue mentioned this pull request Jan 24, 2026
13 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: nodejs Node.js is a free, open-source, cross-platform JavaScript runtime environment 6.topic: python Python is a high-level, general-purpose programming language. 8.has: documentation This PR adds or changes documentation 8.has: port to stable This PR already has a backport to the stable release. 10.rebuild-darwin: 11-100 This PR causes between 11 and 100 packages to rebuild on Darwin. 10.rebuild-linux: 101-500 This PR causes between 101 and 500 packages to rebuild on Linux. 12.approvals: 1 This PR was reviewed and approved by one person. 12.approved-by: package-maintainer This PR was reviewed and approved by a maintainer listed in any of the changed packages.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pnpm.fetchDeps: non-deterministic hash

5 participants