Skip to content

buildRustCrate: nondeterministic across different values of NIX_BUILD_CORES #130309

Closed
Closed
buildRustCrate: nondeterministic across different values of NIX_BUILD_CORES#130309
Labels
0.kind: bugSomething is broken2.status: stalehttps://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md6.topic: reproducible buildsRun nix-build twice and get the same result.6.topic: rustGeneral-purpose programming language emphasizing performance, type safety, and concurrency.
@lagoda-antithesis

Description

@lagoda-antithesis
lagoda-antithesis
opened on Jul 15, 2021

Love the nix project, but I ran into something that caused me some consternation, and I think it might be a bug.

Describe the bug
Pull #57936 changed buildRustCrate from using a fixed number of codegen-units to a variable number.

It looks like this is not purely an optimization, and actually results in different binaries at different settings.

In support of this, the rustc docs note that:

When a crate is split into multiple codegen units, LLVM is able to process them in parallel. Increasing parallelism may speed up compile times, but may also produce slower code. Setting this to 1 may improve the performance of generated code, but may be slower to compile.

I'm new here, and I don't know what the right way to proceed is, but I'm happy to put in a PR reversing 4c89619152a9084f104f883145298c8c89b0616d if that would help. Other plausible options include removing the flag (setting it to the rustc default of 16) or making it configurable (so that different values produce different derivations instead of conflicting builds of the same derivation).

To Reproduce
Steps to reproduce the behavior:

  1. Write a minimal use of buildRustCrate
$ cat min_failing_rust.nix 

let 
  pkgs = (import (builtins.fetchTarball { name = "nixpkgs"; url = "https://github.com/nixos/nixpkgs/archive/7e9b0dff974c89e070da1ad85713ff3c20b0ca97.tar.gz"; }) {}).pkgsStatic;
in with pkgs; pkgs.buildRustCrate {
  crateName = "test";
  version = "0.1.0";
  edition = "2018";
  libPath = "src/main.rs";
  authors = [ "Test" ];    
  src = runCommand "make_minimal_rust" {} ''
    mkdir -p $out/src
    cat << 'END_RUST' >> $out/src/main.rs
    fn main () {
        let buffer = String::new();
        println!("This is a test file");
    }
    END_RUST
    cat << 'END_RUST' > $out/Cargo.toml
    [package]
    name = "test"
    authors = ["Test"]
    [lib]
    name = "test"
    path = "src/main.rs"
    END_RUST
  '';
}
  1. Build with two different numbers of cores (and see that they go to the same nix store path).
$ nix-build --no-out-link min_failing_rust.nix --cores 4
/nix/store/x06pld61jaa8z7zq031zf3jy4nxbi93g-rust_test-0.1.0-x86_64-unknown-linux-musl

$ nix-build --no-out-link min_failing_rust.nix --cores 8
/nix/store/x06pld61jaa8z7zq031zf3jy4nxbi93g-rust_test-0.1.0-x86_64-unknown-linux-musl
  1. Build with two different numbers of cores passing --check (and see the second one fail)
$ nix-build --no-out-link min_failing_rust.nix --cores 4 --check
checking outputs of '/nix/store/m3rwvq9bg2lqmrcf0x4rlcn4rkx8vr6k-rust_test-0.1.0-x86_64-unknown-linux-musl.drv'...
unpacking sources
unpacking source archive /nix/store/iw77h5kfcddpyplbmc75a8fjiaiks0ld-make_minimal_rust
source root is make_minimal_rust
patching sources
updateAutotoolsGnuConfigScriptsPhase
configuring
Running cd .
building
Building src/main.rs (test)
Running rustc --crate-name test src/main.rs --out-dir target/lib -L dependency=target/deps --cap-lints allow -C opt-level=3 -C codegen-units=4 -C incremental=no --remap-path-prefix=/build=/ --edition 2018 --target x86_64-unknown-linux-musl -C linker=x86_64-unknown-linux-musl-gcc -C metadata=d249ed237e -C extra-filename=-d249ed237e --crate-type lib --color always
Building test (src/main.rs)
Running rustc --crate-name test src/main.rs --crate-type bin -C opt-level=3 -C codegen-units=4 -C incremental=no --remap-path-prefix=/build=/ --edition 2018 --target x86_64-unknown-linux-musl -C linker=x86_64-unknown-linux-musl-gcc --out-dir target/bin -L dependency=target/deps --extern test=target/lib/libtest-d249ed237e.rlib --cap-lints allow --color always
installing
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/x06pld61jaa8z7zq031zf3jy4nxbi93g-rust_test-0.1.0-x86_64-unknown-linux-musl
shrinking /nix/store/x06pld61jaa8z7zq031zf3jy4nxbi93g-rust_test-0.1.0-x86_64-unknown-linux-musl/bin/test
x86_64-unknown-linux-musl-strip is /nix/store/635hpnadj6rhz2wbi3yfaxpqb5556qsw-x86_64-unknown-linux-musl-binutils-2.35.1/bin/x86_64-unknown-linux-musl-strip
stripping (with command x86_64-unknown-linux-musl-strip and flags -S) in /nix/store/x06pld61jaa8z7zq031zf3jy4nxbi93g-rust_test-0.1.0-x86_64-unknown-linux-musl/bin
patching script interpreter paths in /nix/store/x06pld61jaa8z7zq031zf3jy4nxbi93g-rust_test-0.1.0-x86_64-unknown-linux-musl
checking for references to /build/ in /nix/store/x06pld61jaa8z7zq031zf3jy4nxbi93g-rust_test-0.1.0-x86_64-unknown-linux-musl...
shrinking RPATHs of ELF executables and libraries in /nix/store/8f063pdg5nf811p00ajbyfh1hi9qhmrl-rust_test-0.1.0-x86_64-unknown-linux-musl-lib
x86_64-unknown-linux-musl-strip is /nix/store/635hpnadj6rhz2wbi3yfaxpqb5556qsw-x86_64-unknown-linux-musl-binutils-2.35.1/bin/x86_64-unknown-linux-musl-strip
stripping (with command x86_64-unknown-linux-musl-strip and flags -S) in /nix/store/8f063pdg5nf811p00ajbyfh1hi9qhmrl-rust_test-0.1.0-x86_64-unknown-linux-musl-lib/lib
patching script interpreter paths in /nix/store/8f063pdg5nf811p00ajbyfh1hi9qhmrl-rust_test-0.1.0-x86_64-unknown-linux-musl-lib
checking for references to /build/ in /nix/store/8f063pdg5nf811p00ajbyfh1hi9qhmrl-rust_test-0.1.0-x86_64-unknown-linux-musl-lib...
/nix/store/x06pld61jaa8z7zq031zf3jy4nxbi93g-rust_test-0.1.0-x86_64-unknown-linux-musl

$ nix-build --no-out-link min_failing_rust.nix --cores 8 --check
checking outputs of '/nix/store/m3rwvq9bg2lqmrcf0x4rlcn4rkx8vr6k-rust_test-0.1.0-x86_64-unknown-linux-musl.drv'...
unpacking sources
unpacking source archive /nix/store/iw77h5kfcddpyplbmc75a8fjiaiks0ld-make_minimal_rust
source root is make_minimal_rust
patching sources
updateAutotoolsGnuConfigScriptsPhase
configuring
Running cd .
building
Building src/main.rs (test)
Running rustc --crate-name test src/main.rs --out-dir target/lib -L dependency=target/deps --cap-lints allow -C opt-level=3 -C codegen-units=8 -C incremental=no --remap-path-prefix=/build=/ --edition 2018 --target x86_64-unknown-linux-musl -C linker=x86_64-unknown-linux-musl-gcc -C metadata=d249ed237e -C extra-filename=-d249ed237e --crate-type lib --color always
Building test (src/main.rs)
Running rustc --crate-name test src/main.rs --crate-type bin -C opt-level=3 -C codegen-units=8 -C incremental=no --remap-path-prefix=/build=/ --edition 2018 --target x86_64-unknown-linux-musl -C linker=x86_64-unknown-linux-musl-gcc --out-dir target/bin -L dependency=target/deps --extern test=target/lib/libtest-d249ed237e.rlib --cap-lints allow --color always
installing
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/x06pld61jaa8z7zq031zf3jy4nxbi93g-rust_test-0.1.0-x86_64-unknown-linux-musl
shrinking /nix/store/x06pld61jaa8z7zq031zf3jy4nxbi93g-rust_test-0.1.0-x86_64-unknown-linux-musl/bin/test
x86_64-unknown-linux-musl-strip is /nix/store/635hpnadj6rhz2wbi3yfaxpqb5556qsw-x86_64-unknown-linux-musl-binutils-2.35.1/bin/x86_64-unknown-linux-musl-strip
stripping (with command x86_64-unknown-linux-musl-strip and flags -S) in /nix/store/x06pld61jaa8z7zq031zf3jy4nxbi93g-rust_test-0.1.0-x86_64-unknown-linux-musl/bin
patching script interpreter paths in /nix/store/x06pld61jaa8z7zq031zf3jy4nxbi93g-rust_test-0.1.0-x86_64-unknown-linux-musl
checking for references to /build/ in /nix/store/x06pld61jaa8z7zq031zf3jy4nxbi93g-rust_test-0.1.0-x86_64-unknown-linux-musl...
shrinking RPATHs of ELF executables and libraries in /nix/store/8f063pdg5nf811p00ajbyfh1hi9qhmrl-rust_test-0.1.0-x86_64-unknown-linux-musl-lib
x86_64-unknown-linux-musl-strip is /nix/store/635hpnadj6rhz2wbi3yfaxpqb5556qsw-x86_64-unknown-linux-musl-binutils-2.35.1/bin/x86_64-unknown-linux-musl-strip
stripping (with command x86_64-unknown-linux-musl-strip and flags -S) in /nix/store/8f063pdg5nf811p00ajbyfh1hi9qhmrl-rust_test-0.1.0-x86_64-unknown-linux-musl-lib/lib
patching script interpreter paths in /nix/store/8f063pdg5nf811p00ajbyfh1hi9qhmrl-rust_test-0.1.0-x86_64-unknown-linux-musl-lib
checking for references to /build/ in /nix/store/8f063pdg5nf811p00ajbyfh1hi9qhmrl-rust_test-0.1.0-x86_64-unknown-linux-musl-lib...
derivation '/nix/store/m3rwvq9bg2lqmrcf0x4rlcn4rkx8vr6k-rust_test-0.1.0-x86_64-unknown-linux-musl.drv' may not be deterministic: output '/nix/store/x06pld61jaa8z7zq031zf3jy4nxbi93g-rust_test-0.1.0-x86_64-unknown-linux-musl' differs
error: build of '/nix/store/m3rwvq9bg2lqmrcf0x4rlcn4rkx8vr6k-rust_test-0.1.0-x86_64-unknown-linux-musl.drv' failed

Expected behavior
I should be able to run nix-build --no-out-link min_failing_rust.nix --cores 4 --check and nix-build --no-out-link min_failing_rust.nix --cores 8 --check in either order and have both succeed.

Notify maintainers

Metadata

$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 5.4.78, NixOS, 20.09.2016.19db3e5ea27 (Nightingale)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.3.9`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute: `pkgs.buildRustCrate`
# a list of nixos modules affected by the problem
module:

Metadata

Metadata

Assignees

No one assigned

    Labels

    0.kind: bugSomething is broken2.status: stalehttps://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md6.topic: reproducible buildsRun nix-build twice and get the same result.6.topic: rustGeneral-purpose programming language emphasizing performance, type safety, and concurrency.

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions