Describe the bug
Note that this bug is specific for content-addressed derivations. When input-addressed, it's a given that introducing env vars that don't do anything will give a different output.
Due to separateDebugInfo inserting a build-id based on the (input-addressed) paths that are used, enabling that option makes a package (that's built using the nixUnstable ca-derivations feature) non-reproducible based on the environment.
E.g. if an environment variable (that isn't used in the build) is set, this causes the build-id to change and thus also the content-addressed package.
A way to resolve this is to not make the build-id based on the contents (+ the path, presumably), but to set it manually to a value (maybe a sha256 of only the contents?). I'm not 100% sure how that should work, but it does not seem too hard to implement.
This is related to NixOS/nix#5220 as well. Hydra is probably doing something it shouldn't, but regardless this should be fixed too.
Steps To Reproduce
Steps to reproduce the behavior:
- Enable the
ca-derivations experimental feature.
- Download this zip with a reproducer: content-address-unreproducible-debug-info.zip
nix-shell -p diffoscope --run "diffoscope $(nix-build 1.nix) $(nix-build 2.nix)"
- Observe different Build ID (and store paths).
Expected behavior
Build is reproducible and produces the same hash when unrelated environment variables are introduced, even with enableDebugInfo on (in content-addressed mode).
Additional context
Resolving this will improve reproducibility of important derivations, since the ones with enableDebugInfo are typically very low-level core libraries.
Since the Build ID is dependent on the input-addressed path, any change in stdenv / dependencies may cause the build to give a different result, even though the only difference is the Build ID. I think this is undesirable.
Notify maintainers
@regnat -> so you're aware of this issue before rolling out ca-derivations in nixpkgs. Thanks for the great work you've done and are doing!
Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.
[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
- system: `"x86_64-linux"`
- host os: `Linux 5.15.7, NixOS, 22.05 (Quokka)`
- multi-user?: `yes`
- sandbox: `yes`
- version: `nix-env (Nix) 2.5.0pre20211206_d1aaa7e`
- channels(root): `"nixos-22.05pre335103.6daa4a5c045, nixpkgs-22.05pre335103.6daa4a5c045"`
- channels(rick): `""`
- nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`
Describe the bug
Note that this bug is specific for content-addressed derivations. When input-addressed, it's a given that introducing env vars that don't do anything will give a different output.
Due to separateDebugInfo inserting a build-id based on the (input-addressed) paths that are used, enabling that option makes a package (that's built using the nixUnstable ca-derivations feature) non-reproducible based on the environment.
E.g. if an environment variable (that isn't used in the build) is set, this causes the build-id to change and thus also the content-addressed package.
A way to resolve this is to not make the build-id based on the contents (+ the path, presumably), but to set it manually to a value (maybe a sha256 of only the contents?). I'm not 100% sure how that should work, but it does not seem too hard to implement.
This is related to NixOS/nix#5220 as well. Hydra is probably doing something it shouldn't, but regardless this should be fixed too.
Steps To Reproduce
Steps to reproduce the behavior:
ca-derivationsexperimental feature.nix-shell -p diffoscope --run "diffoscope $(nix-build 1.nix) $(nix-build 2.nix)"Expected behavior
Build is reproducible and produces the same hash when unrelated environment variables are introduced, even with enableDebugInfo on (in content-addressed mode).
Additional context
Resolving this will improve reproducibility of important derivations, since the ones with enableDebugInfo are typically very low-level core libraries.
Since the Build ID is dependent on the input-addressed path, any change in stdenv / dependencies may cause the build to give a different result, even though the only difference is the Build ID. I think this is undesirable.
Notify maintainers
@regnat -> so you're aware of this issue before rolling out ca-derivations in nixpkgs. Thanks for the great work you've done and are doing!
Metadata
Please run
nix-shell -p nix-info --run "nix-info -m"and paste the result.