stdenv: add buildPlatformCanExecuteHostPlatform#220435
Conversation
This PR adds `stdenv.buildCanExecuteHost`, which is defined as: buildCanExecuteHost = stdenv.buildPlatform.canExecute stdenv.hostPlatform; By making the above expression referenceable by the nice short string `stdenv.buildCanExecuteHost`, I hope people will find it more convenient to type and therefore be more likely to use it instead of `stdenv.buildPlatform==stdenv.hostPlatform` I think that `stdenv.buildPlatform==stdenv.hostPlatform` is almost always the wrong thing, and we should start discouraging it. This PR adds stdenv.buildCanExecuteHost, which is almost always what should be used instead. My buildfarm is a bunch of AMD Opterons. I recently switched over to expressing the `-march=` flag for them the correct way: by creating a new local `buildPlatform` called `opteron` for them which sets `gcc.arch="bdver1"` (and equivalent for `rustc`). I have a few other `x86_64` machines which do not support the `bdver1` instructions (mostly the AVX/SSE stuff). So these machines will segfault if you try to use an `opteron`-compiled `ffmpeg` on them. Therefore, I do two builds of nixpkgs for x86_64 in parallel: one for `opteron` and one for `lib.systems.examples.gnu64` (which is the "vanilla" `x86_64`). Here's the problem: ``` stdenv.buildPlatform != stdenv.hostPlatform ``` because ``` opteron != lib.systems.examples.gnu64 ``` ... but this really a cross-compile from the perspective of everything outside of `stdenv`. Most packages that are doing an equality-test between `buildPlatform` and `hostPlatform` really just want to know if the build platform can execute host binaries -- usually because upstream's build machinery is broken in that case so features have to be disabled. Unfortunately we've established `stdenv.buildPlatform==stdenv.hostPlatform` as the idiom for "is this a cross compile?". So I'd like to make it *more* convenient for package authors to use `stdenv.buildPlatform.canExecute stdenv.hostPlatform`, by making it less verbose for them to type.
|
I like the idea, i am not an expert by any means on the topic, but isn't the architecture inferior list not something that should be used here? nixpkgs/lib/systems/architectures.nix Line 41 in 564e20e sadly it's also not filled for your use case. |
|
Fun fact: there is only one instance of |
pkgs/stdenv/generic/default.nix
Outdated
|
|
||
| stdenv = (stdenv-overridable argsStdenv); | ||
|
|
||
| buildCanExecuteHost = buildPlatform.canExecute hostPlatform; |
There was a problem hiding this comment.
I view this file as almost completely oblivious of internals of buildPlatform or hostPlatform. There is one exception (which we would probably want to remove eventually in favour of explicit stdenv.hostPlatform.isx86_64 & co.):
inherit (hostPlatform)
isDarwin isLinux isSunOS isCygwin isBSD isFreeBSD isOpenBSD
isi686 isx86_32 isx86_64
is32bit is64bit
isAarch32 isAarch64 isMips isBigEndian;There is certain charm of being able to grep for hostPlatform or buildPlatform to see where we peek at the implementation details when building things in individual packages.
The above 2 points very weakly push me to say that I prefer explicit stdenv.buildPlatform.canExecute stdenv.hostPlatform slightly more than external helper.
But then again I almost never use it myself.
There was a problem hiding this comment.
There is certain charm of being able to grep for
hostPlatformorbuildPlatform
This is an excellent point. I have pushed another commit which changes the name to buildPlatformCanExecuteHostPlatform. It's more verbose, but greppability matters.
And yes, I think those inherit (hostPlatform)s should be deleted too.
Indirectly. Thanks for noticing this! ... and that has to be resolved (at least over-conservatively) before I can undraftify this PR. |
|
I've undraftified this, but note that #87909 (or its equivalent) needs to merge before we can start recommending that people change |
|
I feel like this is basically the same as having a |
Do you have a pointer to the discussion? My understanding was that Does anybody think |
|
Same as with the |
It is used, in two places. I'm not proposing to change that. Since Nixpkgs doesn't allow Canadian cross compilers this is a very, very, very unusual and special situation. It only comes up when you have a non-compiler tool with a @sternenseemann it seems like you have aesthetic concerns with this PR, so I'll put it back on the shelf until I deal with #220457 |
cc: @Ericson2314 @alyssais @sternenseemann @trofi for feedback on whether or not this is a good idea.
Description of changes
This PR adds
stdenv.buildCanExecuteHost, which is defined as:Motivation
By making the above expression referenceable by the nice short string
stdenv.buildCanExecuteHost(which never needs to be parenthesized), I hope people will find it more convenient to type and therefore be more likely to use it instead ofstdenv.buildPlatform==stdenv.hostPlatformI think that
stdenv.buildPlatform==stdenv.hostPlatformis almost always the wrong thing, and we should start discouraging it. This PR addsstdenv.buildCanExecuteHost, which is almost always what should be used instead.Specific Example
My buildfarm is a bunch of AMD Opterons. I recently switched over to expressing the
-march=flag for them the correct way: by creating a new localhostPlatformcalledopteronfor them which setsgcc.arch="bdver1"(and equivalent forrustc). Previously I'd just been kludging it intoNIX_CFLAGS_COMPILEwith an overlay.I have a few other
x86_64machines which do not support thebdver1instructions (mostly the AVX/SSE stuff). So these machines will segfault if you try to use anopteron-compiledffmpegon them. Therefore, I do two builds of nixpkgs for x86_64 in parallel: one foropteronand one forlib.systems.examples.gnu64(which is the "vanilla"x86_64).Here's the problem:
because
... but this scenario really is not a cross-compile, except maybe from the perspective the
stdenvbootstrap. Most packages that are doing an equality-test betweenbuildPlatformandhostPlatformreally just want to know if the build platform can execute host binaries -- usually because upstream's build machinery is broken in that case so features have to be disabled.Unfortunately we've established
stdenv.buildPlatform==stdenv.hostPlatformas the idiom for "is this a cross compile?". So I'd like to make it more convenient for package authors to usestdenv.buildPlatform.canExecute stdenv.hostPlatform, by making it less verbose for them to type.