Skip to content

llvmPackages: Clean up outputs#111487

Merged
Ericson2314 merged 1 commit intoNixOS:stagingfrom
Ericson2314:llvm-outputs-and-cross-compile-tools
Apr 30, 2021
Merged

llvmPackages: Clean up outputs#111487
Ericson2314 merged 1 commit intoNixOS:stagingfrom
Ericson2314:llvm-outputs-and-cross-compile-tools

Conversation

@Ericson2314
Copy link
Copy Markdown
Member

@Ericson2314 Ericson2314 commented Jan 31, 2021

Motivation for this change

Inspired by #100574 but we should worry about testing cross after because this is already enough. This is going to require some better upstream support for split outputs.

I left a comment in https://reviews.llvm.org/D28234. If that can go through, I'm happy to backport or just hack up llvm-config another way. I made https://reviews.llvm.org/D99484, see the fairly long commit message there for more info.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
      • llvm lld lldb clang-unwrapped libcxx libcxxabi compiler-rt libunwind lldClang for all LLVM versions that have it!
    • macOS
      • stdenv
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@ofborg ofborg bot added 6.topic: rust General-purpose programming language emphasizing performance, type safety, and concurrency. 6.topic: stdenv Standard environment 6.topic: vim Advanced text editor labels Jan 31, 2021
@Ericson2314 Ericson2314 force-pushed the llvm-outputs-and-cross-compile-tools branch from 12c9e24 to 05f8433 Compare January 31, 2021 22:33
@Ericson2314 Ericson2314 mentioned this pull request Feb 20, 2021
10 tasks
@Ericson2314 Ericson2314 force-pushed the llvm-outputs-and-cross-compile-tools branch from 05f8433 to 6024d59 Compare March 24, 2021 05:19
@ofborg ofborg bot added 10.rebuild-darwin-stdenv This PR causes stdenv to rebuild on Darwin and must target a staging branch. 8.has: package (new) This PR adds a new package 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 5001+ This PR causes many rebuilds on Darwin and must target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 5001+ This PR causes many rebuilds on Linux and must target the staging branches. labels Mar 24, 2021
@Ericson2314 Ericson2314 force-pushed the llvm-outputs-and-cross-compile-tools branch 3 times, most recently from 8f34317 to 7068520 Compare March 26, 2021 14:27
@Ericson2314 Ericson2314 changed the base branch from master to staging March 26, 2021 18:49
@Ericson2314 Ericson2314 force-pushed the llvm-outputs-and-cross-compile-tools branch 2 times, most recently from 9b8a3ba to 516acdd Compare March 27, 2021 16:04
@github-actions github-actions bot added 6.topic: ocaml OCaml is a general-purpose, high-level, multi-paradigm programming language. 6.topic: python Python is a high-level, general-purpose programming language. labels Mar 27, 2021
@Ericson2314 Ericson2314 force-pushed the llvm-outputs-and-cross-compile-tools branch 2 times, most recently from c09afdf to 162f786 Compare March 27, 2021 22:26
@ofborg ofborg bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label Mar 27, 2021
@Ericson2314 Ericson2314 force-pushed the llvm-outputs-and-cross-compile-tools branch 2 times, most recently from ead6b9b to 659faca Compare March 28, 2021 21:51
@Ericson2314 Ericson2314 changed the title Super WIP: llvmPackages: Clean up outputs, cross compile tools themselves WIP: llvmPackages: Clean up outputs, cross compile tools themselves Mar 29, 2021
@Ericson2314 Ericson2314 changed the title WIP: llvmPackages: Clean up outputs, cross compile tools themselves WIP: llvmPackages: Clean up outputs Mar 29, 2021
@ofborg ofborg bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Mar 29, 2021
Copy link
Copy Markdown
Member

@sternenseemann sternenseemann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall this looks very sane to me, although I haven't reviewed the patches and haven't done any build testing yet (what sort of testing have you done?).

I wonder if we can merge then -manpages packages into the main one or at least install the manpages to the man output if man pages are enabled (I guess having them always enabled complicates the bootstrap because of python).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did you remove the getLib here?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since cc is defined as libllvm.out, the output is "predetermined" and stuff like lib.getLib won't work. it's a bummer but the only solution I can think of is to do "out" and "bin" instead of "out" and "lib" (i.e change the default output).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense. I wonder if this behavior of lib.get* is actually necessary, but it is probably not a thing we should include here if it is at all possible since it would need additional testing.

@Ericson2314 Ericson2314 force-pushed the llvm-outputs-and-cross-compile-tools branch 2 times, most recently from ffd09c0 to d60cff0 Compare April 30, 2021 03:14
Also begin to start work on cross compilation, though that will have to
be finished later.

The patches are based on the first version of
https://reviews.llvm.org/D99484. It's very annoying to do the
back-porting but the review has uncovered nothing super major so I'm
fine sticking with what I've got.

Beyond making the outputs work, I also strove to re-sync the packages,
as they have been drifting pointlessly apart for some time.

----

Other misc notes, highly incomplete

- lvm-config-native and llvm-config are put in `dev` because they are
  tools just for build time.

- Clang no longer has an lld dep. That was introduced in
  db29857, but if clang needs help
  finding lld when it is used we should just pass it flags / put in the
  resource dir. Providing it at build time increases critical path
  length for no good reason.

----

A note on `nativeCC`:

`stdenv` takes tools from the previous stage, so:

1. `pkgsBuildBuild`: `(?1, x, x)`
2. `pkgsBuildBuild.stdenv.cc`: `(?0, ?1, x)`

while:

1. `pkgsBuildBuild`: `(?1, x, x)`
2. `pkgsBuildBuild.targetPackages`: `(x, x, ?2)`
3. `pkgsBuildBuild.targetPackages.stdenv.cc`: `(?1, x, x)`
@Ericson2314 Ericson2314 force-pushed the llvm-outputs-and-cross-compile-tools branch from d60cff0 to 7869d16 Compare April 30, 2021 05:41
@Ericson2314
Copy link
Copy Markdown
Member Author

@jonringer I'm curious what you think about this re stability and 21.05. I think there will be some breakage from the output splitting, but it should be fairly rote to fix.

@jonringer
Copy link
Copy Markdown
Contributor

I would feel a lot better if this had a hydra job associated with it, and we could better capture the regressions.

That being said, the "breaking changes to staging" period hasn't started, so you're still free to merge. If you still feel confident that the changes will have minimal regressions, then I'll take you at your word. I'm more afraid for macos which has llvm as part of their stdenv, and more likely to have regressions that linux with its gccStdenv default

@Ericson2314
Copy link
Copy Markdown
Member Author

@jonringer thanks. So one important distinction is that I think the breakages will be for packages that use LLVM and Clang as libraries, or are scrapping our tool chains for parts to assemble their own toolchain. That's basically the same small subset of packages on both platforms. Anything just using Clang via the default stdenv shouldn't be witnessing this stuff one way or the other.

I'll merge and then, but will be monitoring this change on hydra especially once it hits staging-next.

@Ericson2314 Ericson2314 merged commit 17305d2 into NixOS:staging Apr 30, 2021
@Ericson2314 Ericson2314 deleted the llvm-outputs-and-cross-compile-tools branch April 30, 2021 15:28
@Ericson2314 Ericson2314 mentioned this pull request Apr 30, 2021
4 tasks
This was referenced Apr 30, 2021
@jonringer
Copy link
Copy Markdown
Contributor

These may not be related to these particular changes, but I did a full nixpkgs-review of #121335 and there were a few llvm related failures

builder for '/nix/store/nv1f71l2sx1zkkzfgvysi5mcicms43fw-halide-10.0.0.drv' failed with exit code 1; last 10 log lines:

    Add the installation prefix of "LLVM" to CMAKE_PREFIX_PATH or set
    "LLVM_DIR" to a directory containing one of the above files.  If "LLVM"
    provides a separate development package or SDK, be sure it has been
    installed.


  -- Configuring incomplete, errors occurred!
  See also "/build/source/build/CMakeFiles/CMakeOutput.log".
  See also "/build/source/build/CMakeFiles/CMakeError.log".

hmm, thought there was going to be more when writing the post, just looks like one :)

];

cmakeFlags = [
"-DCOMPILER_RT_OS_DIR="
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC, this causes libclang_rt.osx.a to be installed to $out/lib, instead $out/lib/darwin as it was previously, which means it can no longer be found by the linker driver, which always uses darwin (see MachO::AddLinkRuntimeLib). I noticed it trying to cross compile m4 for aarch64-darwin, but I don't yet know how it works on x86_64-darwin.

Previously

result
└── lib
    ├── darwin
    │   ├── libclang_rt.cc_kext.a
    │   └── libclang_rt.osx.a
    ├── libclang_rt.cc_kext.a -> darwin/libclang_rt.cc_kext.a
    └── libclang_rt.osx.a -> darwin/libclang_rt.osx.a

Now

result
└── lib
    ├── libclang_rt.cc_kext.a
    └── libclang_rt.osx.a

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@thefloweringash sorry about that. Does this effect the current 7-based bootstrapping or just the new one?

@vcunat
Copy link
Copy Markdown
Member

vcunat commented May 12, 2021

I think this is the right place to ask about this regression? (link)

ld: error: cannot open /nix/store/*-clang-wrapper-10.0.1/resource-root/lib/linux/libclang_rt.builtins-x86_64.a

@sternenseemann
Copy link
Copy Markdown
Member

Should be fixed as of #122399

@vcunat
Copy link
Copy Markdown
Member

vcunat commented May 12, 2021

So I assume we should fast-forward that fix to staging-next branch.

@vcunat
Copy link
Copy Markdown
Member

vcunat commented May 12, 2021

Perhaps you'd like to recommend which of the multiple llvmPackages changes in staging-next..staging are more likely to fix stuff than break it?

@sternenseemann
Copy link
Copy Markdown
Member

#122399 seems to be the most critical.

#120790 could help some things, but I didn't encounter an edge case that was fixed by those cleanups at the time, so may not be urgent to fast track (at least for native compilation). There were some eval issues surrounding it, so may not be possible to merge on its own (see comment at the end of the PR).

For #122655, I'd first see how it does in staging.

@Ericson2314
Copy link
Copy Markdown
Member Author

Thanks @sternenseemann. I agree with that summary, and would just add #122044 to that.

@vcunat
Copy link
Copy Markdown
Member

vcunat commented May 13, 2021

I took the pair and it looks like a big improvement. On x86_64-darwin this apparently fixed also occurrences of

dyld: lazy symbol binding failed: Symbol not found: ___isOSVersionAtLeast

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: llvm/clang Issues related to llvmPackages, clangStdenv and related 6.topic: ocaml OCaml is a general-purpose, high-level, multi-paradigm programming language. 6.topic: python Python is a high-level, general-purpose programming language. 6.topic: rust General-purpose programming language emphasizing performance, type safety, and concurrency. 6.topic: stdenv Standard environment 6.topic: vim Advanced text editor 8.has: package (new) This PR adds a new package 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 5001+ This PR causes many rebuilds on Darwin and must target the staging branches. 10.rebuild-darwin-stdenv This PR causes stdenv to rebuild on Darwin and must target a staging branch. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 5001+ This PR causes many rebuilds on Linux and must target the staging branches.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants