Skip to content

lib/systems/inspect.nix: replace isPowerPC with isPower32BigEndian#168113

Merged
sternenseemann merged 1 commit intomasterfrom
unknown repository
May 26, 2022
Merged

lib/systems/inspect.nix: replace isPowerPC with isPower32BigEndian#168113
sternenseemann merged 1 commit intomasterfrom
unknown repository

Conversation

@ghost
Copy link
Copy Markdown

@ghost ghost commented Apr 10, 2022

Description of changes

Very confusingly, the isPowerPC predicate in lib/systems/inspect.nix does not match powerpc64le!

This is because isPowerPC is defined as

  isPowerPC      = { cpu = cpuTypes.powerpc; };

Where cpuTypes.powerpc is:

  { bits = 32; significantByte = bigEndian; family = "power"; };

This means that the isPowerPC predicate actually only matches the subset of machines marketed under this name which happen to be 32-bit and running in big-endian mode.

This seems like a sharp edge that people could easily cut themselves on. In fact, that has already happened: in linux/kernel/common-config.nix there is a test which will always
fail:

   (stdenv.hostPlatform.isPowerPC && stdenv.hostPlatform.is64bit) ||

Let's rename the predicate to isPower32BigEndian since that's what it really checks. This predicate is used in only three places: ghc, linux/kernel, and nixos's iso-image.nix.

Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
    • powerpc64le-linux
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 22.05 Release Notes (or backporting 21.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
    • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
  • Fits CONTRIBUTING.md.

@github-actions github-actions bot added 6.topic: haskell General-purpose, statically typed, purely functional programming language 6.topic: kernel The Linux kernel 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: module (update) This PR changes an existing module in `nixos/` labels Apr 10, 2022
@ofborg ofborg bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. labels Apr 10, 2022
@ghost ghost marked this pull request as ready for review April 10, 2022 09:45
@ghost ghost mentioned this pull request Apr 10, 2022
14 tasks
Copy link
Copy Markdown
Member

@sternenseemann sternenseemann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GHC supports PowerPC in general (see this wiki page), so to match the intention, the predicate should be changed to isPower everywhere GHC-related, I believe.

I suspect this is the case in other places as well, as it is indeed confusingly named, good that you caught that!

@sternenseemann
Copy link
Copy Markdown
Member

This seems like a sharp edge that people could easily cut themselves on. In fact, that has already happened: in linux/kernel/common-config.nix there is a test which will always fail:

  (stdenv.hostPlatform.isPower && stdenv.hostPlatform.is64bit)

Let's rename the predicate to isPower32BigEndian since that's what it really checks. This predicate is used in only three places: ghc, linux/kernel, and nixos's iso-image.nix.

isPower != isPowerPC, though, so this would succeed for powerpc64 and powerpc64le, as isPower checks the CPU family?!

@ghost
Copy link
Copy Markdown
Author

ghost commented Apr 11, 2022

GHC supports PowerPC in general (see this wiki page)

That's actually what I'm working on next...

isPower != isPowerPC, though, so this would succeed for powerpc64 and powerpc64le, as isPower checks the CPU family?!

Sorry, I cut-and-pasted that line from an unsubmitted earlier version of this PR by mistake. I have corrected my comment at the top of the thread. That line currently reads:

  (stdenv.hostPlatform.isPowerPC && stdenv.hostPlatform.is64bit) ||

the predicate should be changed to isPower everywhere GHC-related, I believe.

Sorry, I don't quite understand? This PR changes the predicate from isPower to isPower32BigEndian. (maybe this was a result of the confusion I caused with my error above).

@ghost
Copy link
Copy Markdown
Author

ghost commented Apr 11, 2022

Squashed @sternenseemann's changes into a single commit and fixed the typo in the commit message. No other changes.

@ghost ghost requested a review from sternenseemann April 11, 2022 00:22
@ghost

This comment was marked as resolved.

@sternenseemann
Copy link
Copy Markdown
Member

Sorry, I don't quite understand? This PR changes the predicate from isPower to isPower32BigEndian. (maybe this was a result of the confusion I caused with my error above).

isPower, not isPowerPC. On master it's isPowerPC atm (aka isPower32BigEndian), but it really should be isPower since GHC has NCG support for all (?) power pc platforms.

@sternenseemann
Copy link
Copy Markdown
Member

By the way @sternenseemann, I'm using the Gentoo bootstrapping binaries for ghc-8.6.5 on powerpc64le. If you know of a better source (Hydra-generated?) please let me know.

There is an official bindist for GHC 8.6.5 for powerpc64le which I'd encourage you to use: https://downloads.haskell.org/~ghc/8.6.5/ghc-8.6.5-powerpc64le-fedora29-linux.tar.xz

I don't think nixpkgs can cross compile GHC at the moment (it can build a GHC cross compiler), GHC's cross build system is probably going to give you a lot of trouble in that case.

@ghost

This comment was marked as resolved.

@Artturin Artturin added 12.approvals: 3+ This PR was reviewed and approved by three or more persons. and removed 12.approvals: 3+ This PR was reviewed and approved by three or more persons. labels Apr 11, 2022
@ghost
Copy link
Copy Markdown
Author

ghost commented Apr 20, 2022

@ofborg eval

(ofborg seems to have become stuck)

@ghost
Copy link
Copy Markdown
Author

ghost commented Apr 20, 2022

Sorry, I don't quite understand? This PR changes the predicate from isPower to isPower32BigEndian. (maybe this was a result of the confusion I caused with my error above).

isPower, not isPowerPC. On master it's isPowerPC atm (aka isPower32BigEndian), but it really should be isPower since GHC has NCG support for all (?) power pc platforms.

@sternenseemann sorry, I still do not understand what you are asking for. I must be really stupid. After this commit, there are no occurrences of the string isPowerPC anywhere in nixpkgs:

nix@moore:/nix/git/nixpkgs$ git rev-parse HEAD
bc623b3ab7d2f37b177d3eab72f2a6d8109ac61f
nix@moore:/nix/git/nixpkgs$ rg isPowerPC
nix@moore:/nix/git/nixpkgs$

If this has not resolved your request, could you please make a self-contained statement of what you are asking me to do?

@ghost
Copy link
Copy Markdown
Author

ghost commented Apr 26, 2022

@sternenseemann in this comment, were you suggesting that the following substitutions be made (on top of this PR)?

s_isPower64_isPowerPc64_
s_isPower32_isPowerPc32_
s_isPower32BigEndian_isPowerPc32BigEndian_

I am fine with that. I originally picked isPower because the architecture is called "POWER" (up until version 10, where they decapitalized it to "Power 10"). "Power PC" was a marketing name used for a subset of IBM/Motorola's POWER-architecture processors. Every "Power PC" branded CPU is a POWER architecture CPU but not vice versa.

On the other hand, the autoconf/gcc multiarch tuple for this architecture is powerpc[64][le], and I'm generally in favor of using the autoconf/gcc multiarch taxonomy wherever possible. There are way too many competing taxonomies in the open source corpus.

So really it doesn't matter to me. If you were asking for the substitutions above just say so and I will implement them (and also update #168111 which I pushed for a merge of because I needed to use that predicate in other PRs). At this point it's still very easy to make changes like this, and I don't have a strong preference one way or the other.

Copy link
Copy Markdown
Member

@sternenseemann sternenseemann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I hope I'll manage to clear up the misunderstanding this time!

My request was not to change how any predicate is named, but which predicate is used in certain places.

For GHC I can say it with some confidence: Where the predicate is used we decide whether the GHC -fasm backend is available or we have to use LLVM. From looking at the code, GHC's Power(PC) backend supports both 32 and 64 bits as well as both endianess settings. Therefore we should check for the CPU family instead of a specific subarchitecture in that place (see the suggestions).

(My suspicion is that it is similar for other uses you touched in this PR, but I guess we shouldn't change anything without checking probably which I haven't so far.)

@ghost
Copy link
Copy Markdown
Author

ghost commented Apr 28, 2022

Latest push applies all of @sternenseemann's suggestions and squashes to a single commit.

Thank you so much for the clarification! Makes perfect sense now.

@ghost ghost requested a review from sternenseemann April 28, 2022 20:54
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked and the xz man page says it supports

PowerPC         4       Big endian only

While it doesn't explicitly say 64bit is not supported, it seems that way. So this seems correct after all!

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bit bizarre and I have no clue what this is supposed to be doing, so let's not touch it. Interestingly the only dir this could be referring to is sound/ppc which is a bit confusing to me.

@Ericson2314 extraIncludeDirs is only used here in nixpkgs, maybe we can remove it?

@sternenseemann
Copy link
Copy Markdown
Member

I think this deserves a brief changelog entry (under “Backwards incompatible changes”) and then we should be good to go!

@ghost
Copy link
Copy Markdown
Author

ghost commented Apr 30, 2022

I think this deserves a brief changelog entry (under “Backwards incompatible changes”)

Done.

I noticed that after your changes, there were only three uses of this predicate remaining. How about we just inline its definition and remove the predicate entirely? Since both renaming it and deleting it are both backwards-incompatible changes, I further suggest doing both at the same time so we only have one such change rather than two.

I've pushed an extra commit onto this PR which does what is described in the previous paragraph. If that's not okay, you can merge all-but-the-last commit in this series.

Let me know if I should squash all or some of the commits.

@ghost
Copy link
Copy Markdown
Author

ghost commented Apr 30, 2022

Hrm, github accepted my push to this branch, but the web UI isn't showing the extra commits. That's weird.

You can see them if you view the branch through my personal repo rather than through the PR: https://github.com/a-m-joseph/nixpkgs/commits/ispowerpc-becomes-ispower32

Okay, github caught up with itself.

Currently waiting for ./md-to-db.sh to compile pandoc, which it's been working on for ~3 hours now...

@github-actions github-actions bot added 8.has: changelog This PR adds or changes release notes 8.has: documentation This PR adds or changes documentation labels Apr 30, 2022
@ghost
Copy link
Copy Markdown
Author

ghost commented Apr 30, 2022

Yep, still waiting for pandoc to build...

Finally finished. It tried to remove the language="" attributes from all the <programlisting> tags, which I assume is not something I ought to commit? So I committed only the lines added. I guess we'll see if CI is okay with that.

@Artturin Artturin added the 12.approvals: 2 This PR was reviewed and approved by two persons. label May 4, 2022
@Artturin Artturin removed the 12.approvals: 2 This PR was reviewed and approved by two persons. label May 11, 2022
Copy link
Copy Markdown
Member

@sternenseemann sternenseemann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found some typose we missed previously.

@sternenseemann
Copy link
Copy Markdown
Member

Is it important to you that this makes 22.05? We can still backport this to the release branch until the actual release is cut.

I'm not sure, I guess it depends on whether you plan on backporting other fixes for powerpc64le-linux later as well.

@ghost
Copy link
Copy Markdown
Author

ghost commented May 24, 2022

Is it important to you that this makes 22.05?

No, it is not -- but thanks for asking.

If the nixpkgs-genie gave me one wish, it would be #168413 so we can finally commit a bootstrap-files hash for mips64el, which has been blocking pretty much all progress on that platform for a few months now.

Also, I know I need to update the manual docbook for this PR. I'm still waiting for pandoc to rebuild. Will do so as soon as it finishes building.

@sternenseemann
Copy link
Copy Markdown
Member

No, it is not -- but thanks for asking.

Okay, that also gives downstream more time to find out this platform attribute is missing – as I've discovered there's no good mechanism for removing these attributes with a good error message for downstream users.

Very confusingly, the `isPowerPC` predicate in
`lib/systems/inspect.nix` does *not* match `powerpc64le`!

This is because `isPowerPC` is defined as

  isPowerPC      = { cpu = cpuTypes.powerpc; };

Where `cpuTypes.powerpc` is:

  { bits = 32; significantByte = bigEndian; family = "power"; };

This means that the `isPowerPC` predicate actually only matches the
subset of machines marketed under this name which happen to be 32-bit
and running in big-endian mode which is equivalent to:

  with stdenv.hostPlatform; isPower && isBigEndian && is32bit

This seems like a sharp edge that people could easily cut themselves
on.  In fact, that has already happened: in
`linux/kernel/common-config.nix` there is a test which will always
fail:

  (stdenv.hostPlatform.isPowerPC && stdenv.hostPlatform.is64bit)

A more subtle case of the strict isPowerPC being used instead of the
moreg general isPower accidentally are the GHC expressions:

  Update pkgs/development/compilers/ghc/8.10.7.nix
  Update pkgs/development/compilers/ghc/8.8.4.nix
  Update pkgs/development/compilers/ghc/9.2.2.nix
  Update pkgs/development/compilers/ghc/9.0.2.nix
  Update pkgs/development/compilers/ghc/head.nix

Since the remaining legitimate use sites of isPowerPC are so few, remove
the isPowerPC predicate completely. The alternative expression above is
noted in the release notes as an alternative.

Co-authored-by: sternenseemann <sternenseemann@systemli.org>
@sternenseemann
Copy link
Copy Markdown
Member

I've squashed the branch and moved the Changelog entry to the 21.11 release notes. Let me know if you're happy with the commit message and I'll merge.

@ghost
Copy link
Copy Markdown
Author

ghost commented May 25, 2022

I've squashed the branch and moved the Changelog entry to the 21.11 release notes. Let me know if you're happy with the commit message and I'll merge.

Thank you! Looks good to me.

@sternenseemann sternenseemann merged commit 8b5e372 into NixOS:master May 26, 2022
@ghost ghost deleted the ispowerpc-becomes-ispower32 branch May 26, 2022 20:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: haskell General-purpose, statically typed, purely functional programming language 6.topic: kernel The Linux kernel 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: changelog This PR adds or changes release notes 8.has: documentation This PR adds or changes documentation 8.has: module (update) This PR changes an existing module in `nixos/` 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants