testers.testEqualContents: Fix diffoscope false positive by roberth · Pull Request #393381 · NixOS/nixpkgs

roberth · 2025-03-26T12:05:12Z

Fixes testEqualContents: false positive on directory size #393375

Not excited about this solution, but it gets the job done.
Ideally diffoscope could implement NAR-style content equality semantics for files and directories, but until it does, we need a fix.

Things done

Add a 👍 reaction to pull requests you find important.

Fixes NixOS#393375

roberth · 2025-03-26T12:13:13Z

@ofborg build tests.testers.testEqualContents

tie · 2025-03-26T12:47:07Z

pkgs/build-support/testers/default.nix

    printf '%s\n' "$assertion"
-    if ! diffoscope --no-progress --text-color=always --exclude-directory-metadata=no -- "$actual" "$expected"
+    # --exclude-directory-metadata=yes: This reports false positives such as the internal directory sizes,
+    # so we work around this by excluding the directory metadata, and then checking the executable bits later.


Do we need to check executable bit for directories? I assume directories should always have executable bit set in Nix store since it cannot be represented in NAR.

Hm, though this doesn’t seem to be the case now…

nix-repl> :b pkgs.runCommand "foo" { } "mkdir -p $out/foo && chmod -x $out/foo" This derivation produced the following outputs: out -> /nix/store/1xnsf9n8ar37vhbb0w6fjciaf8bkz7w1-foo $ ls -al /nix/store/1xnsf9n8ar37vhbb0w6fjciaf8bkz7w1-foo total 0 dr-xr-xr-x 1 root root 6 Jan 1 1970 ./ drwxrwxr-t 1 root nixbld 11M Mar 26 15:41 ../ dr--r--r-- 1 root root 0 Jan 1 1970 foo/ $ ls -al /nix/store/1xnsf9n8ar37vhbb0w6fjciaf8bkz7w1-foo/foo ls: cannot access '/nix/store/1xnsf9n8ar37vhbb0w6fjciaf8bkz7w1-foo/foo/.': Permission denied ls: cannot access '/nix/store/1xnsf9n8ar37vhbb0w6fjciaf8bkz7w1-foo/foo/..': Permission denied total 0 d????????? ? ? ? ? ? ./ d????????? ? ? ? ? ? ../ $ nix nar pack /nix/store/1xnsf9n8ar37vhbb0w6fjciaf8bkz7w1-foo >foo.nar $ nix nar ls --json --recursive foo.nar / {"entries":{"foo":{"entries":{},"type":"directory"}},"type":"directory"}

nix-repl> :b pkgs.runCommand "foo" { } "mkdir -p $out/foo && touch $out/foo/bar && chmod -x $out/foo" This derivation produced the following outputs: out -> /nix/store/6nbxwp349lfbk3yrd2mgv3nya78s7bp8-foo $ ls -al /nix/store/6nbxwp349lfbk3yrd2mgv3nya78s7bp8-foo total 0 dr-xr-xr-x 1 root root 6 Jan 1 1970 ./ drwxrwxr-t 1 root nixbld 11M Mar 26 15:49 ../ dr--r--r-- 1 root root 6 Jan 1 1970 foo/ $ ls -al /nix/store/6nbxwp349lfbk3yrd2mgv3nya78s7bp8-foo/foo ls: cannot access '/nix/store/6nbxwp349lfbk3yrd2mgv3nya78s7bp8-foo/foo/.': Permission denied ls: cannot access '/nix/store/6nbxwp349lfbk3yrd2mgv3nya78s7bp8-foo/foo/..': Permission denied ls: cannot access '/nix/store/6nbxwp349lfbk3yrd2mgv3nya78s7bp8-foo/foo/bar': Permission denied total 0 d????????? ? ? ? ? ? ./ d????????? ? ? ? ? ? ../ -????????? ? ? ? ? ? bar $ nix nar pack /nix/store/6nbxwp349lfbk3yrd2mgv3nya78s7bp8-foo >foo.nar error: getting status of '/nix/store/6nbxwp349lfbk3yrd2mgv3nya78s7bp8-foo/foo/bar': Permission denied

IIUC diffoscope considers the executable bit of a file to be part of the directory, and not part of a file, similarly to the typical unix file system data model or even that of Git. This is different from the NAR data model.

directory entry: name -> mode, content/inode ^^^^^^^^^^^^^--- diffoscope unit of composition NAR item ("File System Object") mode, content ^^^^^^^^^^^^^^--------- NAR unit of composition store path content root

Do we need to check executable bit for directories?

Indeed NAR does not record this bit, so we shouldn't encounter any differences there.
This seems to be a bug in Nix store path canonicalization.
What kind of store did you use to achieve this - what does nix store ping say?

What kind of store did you use to achieve this - what does nix store ping say?

This is a default NixOS installation, user is trusted.

$ nix store ping warning: 'nix store ping' is a deprecated alias for 'nix store info' Store URL: daemon Version: 2.25.5 Trusted: 1

cat /etc/nix/nix.conf

# WARNING: this file is generated from the nix.* options in # your NixOS configuration, typically # /etc/nixos/configuration.nix. Do not edit it! allowed-users = * auto-optimise-store = false builders = cores = 0 experimental-features = nix-command flakes max-jobs = auto require-sigs = true sandbox = true sandbox-fallback = false substituters = https://cache.nixos.org/ system-features = nixos-test benchmark big-parallel kvm trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= trusted-substituters = trusted-users = root @wheel use-xdg-base-directories = true warn-dirty = false extra-sandbox-paths =

I can also reproduce this on stable Nix version from nixos-24.11:

Details

$ nix store info Store URL: daemon Version: 2.24.12 Trusted: 1

$ nix repl Nix 2.24.12 Type :? for help. nix-repl> :lf nixpkgs Added 16 variables. nix-repl> lib.version "24.11.20250327.b9c013a" nix-repl> pkgs = legacyPackages.x86_64-linux nix-repl> :b pkgs.runCommand "foo" { } "mkdir -p $out/foo && touch $out/foo/bar && chmod -x $out/foo" This derivation produced the following outputs: out -> /nix/store/q4xw50il9f6fi6xs644wisqdx0vr6x1f-foo [1 built, 0.0 MiB DL]

$ nix nar pack /nix/store/q4xw50il9f6fi6xs644wisqdx0vr6x1f-foo >/dev/null error: getting status of '/nix/store/q4xw50il9f6fi6xs644wisqdx0vr6x1f-foo/foo/bar': Permission denied

$ stat /nix/store/q4xw50il9f6fi6xs644wisqdx0vr6x1f-foo/foo File: /nix/store/q4xw50il9f6fi6xs644wisqdx0vr6x1f-foo/foo Size: 6 Blocks: 0 IO Block: 4096 directory Device: 0,43 Inode: 161713277 Links: 1 Access: (0444/dr--r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-03-27 19:51:13.000000000 +0300 Modify: 1970-01-01 03:00:01.000000000 +0300 Change: 2025-03-27 19:51:13.297001247 +0300 Birth: 2025-03-27 19:51:13.292001218 +0300

Alternatively,

nix build --impure --expr 'with import ./. { }; let script = "mkdir -p $out/foo && touch $out/foo/bar && chmod -x $out/foo"; badDir = runCommand "bad-directory-permissions" { } script; in runCommand "cat-file" { inherit badDir; } "stat $badDir/foo && cat $badDir/foo/bar"'

cat-file> File: /nix/store/99swczlgqpmkqvsd8vzzlqygxw56giqz-bad-directory-permissions/foo cat-file> Size: 6 Blocks: 0 IO Block: 4096 directory cat-file> Device: 0,33 Inode: 242781643 Links: 1 cat-file> Access: (0444/dr--r--r--) Uid: (65534/ nobody) Gid: (65534/ nogroup) cat-file> Access: 2025-03-27 16:59:12.000000000 +0000 cat-file> Modify: 1970-01-01 00:00:01.000000000 +0000 cat-file> Change: 2025-03-27 16:59:12.461890171 +0000 cat-file> Birth: 2025-03-27 16:59:12.461890171 +0000 cat-file> cat: /nix/store/99swczlgqpmkqvsd8vzzlqygxw56giqz-bad-directory-permissions/foo/bar: Permission denied

Looks like permissions are set in canonicaliseTimestampAndPermissions and 0444 is not updated to 0555 for directories.

E.g. 0554 is canonicalized, but not 0444.

Details

$ nix build -L --impure --expr 'with import ./. { }; let script = "mkdir -p $out/foo && touch $out/foo/bar && chmod o-x $out/foo"; badDir = runCommand "bad-directory-permissions" { } script; in runCommand "cat-file" { inherit badDir; } "stat $badDir/foo && cat $badDir/foo/bar && touch $out"' cat-file> File: /nix/store/42fsalzfn075vhhxbmrvbma37v3wf5wz-bad-directory-permissions/foo cat-file> Size: 6 Blocks: 0 IO Block: 4096 directory cat-file> Device: 0,33 Inode: 242782051 Links: 1 cat-file> Access: (0555/dr-xr-xr-x) Uid: (65534/ nobody) Gid: (65534/ nogroup) cat-file> Access: 2025-03-27 17:18:14.000000000 +0000 cat-file> Modify: 1970-01-01 00:00:01.000000000 +0000 cat-file> Change: 2025-03-27 17:18:14.682332844 +0000 cat-file> Birth: 2025-03-27 17:18:14.682332844 +0000

Perhaps

-if (mode != 0444 && mode != 0555) { +bool is_dir = S_ISDIR(st.st_mode); +if ((mode != 0444 || is_dir) && mode != 0555) { mode = (st.st_mode & S_IFMT) | 0444 - | (st.st_mode & S_IXUSR ? 0111 : 0); + | (st.st_mode & S_IXUSR || is_dir ? 0111 : 0); if (chmod(path.c_str(), mode) == -1) throw SysError("changing mode of '%1%' to %2$o", path, mode); }

Thank you. I've forwarded this to nix, NixOS/nix#12786

tie · 2025-03-26T12:57:24Z

Not excited about this solution, but it gets the job done.

It’s fine as a temporary workaround, but I think we should submit a patch that removes st_size from directory stat comparison (since it’s filesystem-dependent). Note that directory entries are always diffed before stat metadata.
See

Note: I don’t have Debian Salsa account and registration confirmation can take up to a few days (see https://wiki.debian.org/Salsa/Doc#Users).

tie · 2025-03-26T21:44:21Z

Submitted fix in https://salsa.debian.org/reproducible-builds/diffoscope/-/merge_requests/150

tie · 2025-03-31T16:05:54Z

This issue should be fixed upstream in https://diffoscope.org/news/diffoscope-292-released (starting with version 293).
nixos-24.11 is currently at 283, unstable at 289.
Changelog: https://salsa.debian.org/reproducible-builds/diffoscope/-/blob/master/debian/changelog?ref_type=heads

The underlying bug seems to have been fixed in diffoscope 293 [1] [2]. Our nixpkgs input has 295. [1]: NixOS/nixpkgs#393381 (comment) [2]: https://diffoscope.org/news/diffoscope-292-released/

philiptaron

Now that both diffoscope and Nix have fixes, we can close this without merging I believe.

roberth · 2025-08-08T19:59:09Z

Thanks again, tie!

testers.testEqualContents: Fix diffoscope false positive

36eba9c

Fixes NixOS#393375

roberth requested a review from tie March 26, 2025 12:05

github-actions bot added the 6.topic: testing Tooling for automated testing of packages and modules label Mar 26, 2025

nix-owners bot requested a review from philiptaron March 26, 2025 12:06

github-actions bot added 10.rebuild-darwin: 11-100 This PR causes between 11 and 100 packages to rebuild on Darwin. 10.rebuild-linux: 11-100 This PR causes between 11 and 100 packages to rebuild on Linux. labels Mar 26, 2025

roberth added the backport release-24.11 label Mar 26, 2025

This was referenced Mar 26, 2025

C API / settings: remove nix-flake-c global init NixOS/nix#12759

Merged

docs: document unsafeGetAttrPos NixOS/nix#12760

Merged

tie reviewed Mar 26, 2025

View reviewed changes

This was referenced Mar 28, 2025

Directory permissions not always canonicalized NixOS/nix#12786

Closed

Intermittant CI failure NixOS/nix#12690

Closed

edolstra mentioned this pull request Mar 31, 2025

Disable packaging-overriding test NixOS/nix#12812

Merged

This was referenced Mar 31, 2025

Disable packaging-overriding test (backport #12812) NixOS/nix#12816

Merged

Disable packaging-overriding test (backport #12812) NixOS/nix#12819

Merged

wegank added the 2.status: merge conflict This PR has merge conflicts with the target branch label Apr 2, 2025

xokdvium mentioned this pull request May 29, 2025

flake: Restore packaging-overriding check NixOS/nix#13299

Merged

leona-ya removed the backport release-24.11 label Jul 1, 2025

philiptaron mentioned this pull request Jul 23, 2025

libstore: always canonicalize directory permissions NixOS/nix#13526

Merged

philiptaron requested changes Aug 6, 2025

View reviewed changes

roberth closed this Aug 8, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

testers.testEqualContents: Fix diffoscope false positive#393381

testers.testEqualContents: Fix diffoscope false positive#393381
roberth wants to merge 1 commit intoNixOS:masterfrom
roberth:issue-393375

roberth commented Mar 26, 2025 •

edited

Loading

Uh oh!

roberth commented Mar 26, 2025

Uh oh!

tie Mar 26, 2025 •

edited

Loading

Uh oh!

roberth Mar 27, 2025

Uh oh!

tie Mar 27, 2025 •

edited

Loading

Uh oh!

tie Mar 27, 2025 •

edited

Loading

Uh oh!

roberth Mar 28, 2025

Uh oh!

tie commented Mar 26, 2025 •

edited

Loading

Uh oh!

tie commented Mar 26, 2025

Uh oh!

tie commented Mar 31, 2025 •

edited

Loading

Uh oh!

philiptaron left a comment

Uh oh!

roberth commented Aug 8, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

roberth commented Mar 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Things done

Uh oh!

roberth commented Mar 26, 2025

Uh oh!

tie Mar 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

roberth Mar 27, 2025

Choose a reason for hiding this comment

Uh oh!

tie Mar 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tie Mar 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

roberth Mar 28, 2025

Choose a reason for hiding this comment

Uh oh!

tie commented Mar 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tie commented Mar 26, 2025

Uh oh!

tie commented Mar 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

philiptaron left a comment

Choose a reason for hiding this comment

Uh oh!

roberth commented Aug 8, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

roberth commented Mar 26, 2025 •

edited

Loading

tie Mar 26, 2025 •

edited

Loading

tie Mar 27, 2025 •

edited

Loading

tie Mar 27, 2025 •

edited

Loading

tie commented Mar 26, 2025 •

edited

Loading

tie commented Mar 31, 2025 •

edited

Loading