Skip to content

haskell.compiler.ghcHEAD: support JS backend#208947

Merged
Ericson2314 merged 6 commits intoNixOS:haskell-updatesfrom
sternenseemann:ghc-head-js-backend-try-1
Jan 4, 2023
Merged

haskell.compiler.ghcHEAD: support JS backend#208947
Ericson2314 merged 6 commits intoNixOS:haskell-updatesfrom
sternenseemann:ghc-head-js-backend-try-1

Conversation

@sternenseemann
Copy link
Copy Markdown
Member

@sternenseemann sternenseemann commented Jan 3, 2023

Description of changes

Was able to compile a test jsexe (impurely as the package set doesn't work yet, see the last commit for details)!

To try it out, build nix-build -A pkgsCross.ghcjs.buildPackages.haskell.compiler.ghcHEAD on this branch.

Closes #208879.

Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 23.05 Release Notes (or backporting 22.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
    • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
  • Fits CONTRIBUTING.md.

GHC's js backend depends on systemd via emscripten via closure compiler
via jdk via cups. Before it fails to evaluate, though, since
llvmPackages looks into `targetPackages.stdenv.cc` to determine which
C++ library to use (something that should be rectified in the future).
[Unfortunately], for `pkgsCross.ghcjs`, `stdenv.cc` throws which blows
up evaluating `pkgsCross.buildPackages.llvmPackages.clang`.

This is in principle unnecessary. We want to build
`pkgsCross.ghcjs.buildPackages.haskell.compiler.native-bignum.ghcHEAD`
which depends on `pkgsCross.ghcjs.buildPackages.systemd` which needs
clang and friends only in `nativeBuildInputs`, so
`pkgsCross.ghcjs.buildPackages.buildPackages.llvmPackages.clang`.
Unfortunately, due to the nature of splicing, we first evaluate the
“adjacent” derivation before we can access the spliced derivation we are
actually interested in. If the former
fails (`pkgsCross.ghcjs.buildPackages.llvmPackages.clang`), we can't do
the latter.

The solution is to just not rely on splicing in this case and take
`buildPackages.llvmPackages.clang` directly (relative to
`buildPackages.systemd` in this case!) which avoids the whole problem.

[Unfortunately]: NixOS@c739c42#diff-3209527bd27cbc775f579b1e295b0264c850859c7245d526965cec456b8c70a4R61
This will be required for the JavaScript backend to work.
For the GHC JavaScript backend, we'll use emscripten in place of
targetCC. To avoid having too much special logic for this, we'll make
the emscripten derivation look like the result of wrapCC as far as GHC
is concerned, i.e. we need targetPrefix and bintools.

For bintools, we'll just reexpose emscripten, as it has emar, the only
relevant bintools. That the other ones are missing doesn't matter in
practice, as the GHC build system won't attempt to use them.

targetPrefix can immediately be (ab)used to make sure GHC will correctly
call emcc etc. instead of plain cc.
gmp is part of buildInputs _and_ depsTargetTarget, so we need to check
the host and target platform to be correct. In practice this doesn't
change much though, as gmp.meta.platforms is _quite_ liberal.
Since NixOS#200337 gobject-introspection propagates itself via
depsTargetTargetPropagated, so one doesn't have to add it to every
derivation twice. The problem is that gobject-introspection still is in
a lot of buildInputs and will thus propagate itself again for target,
breaking evaluation unnecessarily if gobject-introspection doesn't
evaluate on whatever the target platform turns out to be.

temurin-bin and openjdk19 caused such a situation via gtk3 which GHC's
JavaScript backend depends on. To fix evaluation of those packages in
pkgsCross.ghcjs.buildPackages, we'll just disable the features pulling
in gtk3 until this is fixed properly.
This is now possible by building a cross compiler for js-unknown-ghjs
using `pkgsCross.ghcjs.buildPackages.haskell.compiler.ghcHEAD`.

To allow this, the following things needed to be done:

* Disable dependencies that wouldn't work:

  - Don't pull in ncurses for terminfo
  - Don't pull in libffi
  - Don't pull in libiconv
  - Don't enable the LLVM backend
  - Enable gmp-less native-bignum backend

* Use emscripten instead of a C compiler. The way this works is inspired
  by emscriptenPackages, but avoids the following flaws:

  - Instead of using a custom configurePhase, just set
    `configureScript = "emconfigure ./configure";` which is much simpler.

  - Create writable EM_CACHE before configuring, as configure scripts
    want to compile test programs.

  Additionally, we need to disable the targetCC check, as it is not
  applicable with emscripten which never appears as part of stdenv.

* Use generic $configureScript in installPhase to be able to work with
  our emconfigure trick.

Note that the corresponding Haskell package set does not work yet. Cabal
doesn't seem to like GHC 9.7 yet and the generic-builder is clueless
about the JS backend.
@github-actions github-actions bot added 6.topic: haskell General-purpose, statically typed, purely functional programming language 6.topic: systemd Software suite that provides an array of system components for Linux operating systems. labels Jan 3, 2023
@ofborg ofborg bot requested review from expipiplus1 and guibou January 3, 2023 23:27
@ofborg ofborg bot added 11.by: package-maintainer This PR was created by a maintainer of all the package it changes. 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. labels Jan 3, 2023
Copy link
Copy Markdown

@ghost ghost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM if the systemd maintainers are okay with the changes to that package (see below).

Built on:

  • x86_64-linux
  • powerpc64le-linux (in progress, required hacks)

Comment on lines +73 to +80
# Note: llvmPackages is explicitly taken from buildPackages instead of relying
# on splicing. Splicing will evaluate the adjacent (pkgsHostTarget) llvmPackages
# which is sometimes problematic: llvmPackages.clang looks at targetPackages.stdenv.cc
# which, in the unfortunate case of pkgsCross.ghcjs, `throw`s. If we explicitly
# take buildPackages.llvmPackages, this is no problem because
# `buildPackages.targetPackages.stdenv.cc == stdenv.cc` relative to us. Working
# around this is important, because systemd is in the dependency closure of
# GHC via emscripten and jdk.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After dropping this commit and rebasing upon #208993 I get the same derivation from pkgsCross.ghcjs.buildPackages.haskell.compiler.ghcHEAD. Would you consider doing that?

Then the systemd expression maintainers don't need to carry this comment reminding them to not do certain things in order to avoid breaking ghcjs... it moves the ghcjs-specific workaround into the preexisting if targetPlatform.isGhcjs block in pkgs/stdenv/cross/default.nix.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess we can revert that later. In principle this change is entirely unproblematic, as the resulting derivation is the exact same one (as can be discerned from the rebuild count).

@sternenseemann
Copy link
Copy Markdown
Member Author

The gtk3 issue will be fixed by #208945.

Copy link
Copy Markdown
Member

@Ericson2314 Ericson2314 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great to see this!

@Ericson2314 Ericson2314 merged commit b92fb60 into NixOS:haskell-updates Jan 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: haskell General-purpose, statically typed, purely functional programming language 6.topic: systemd Software suite that provides an array of system components for Linux operating systems. 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. 11.by: package-maintainer This PR was created by a maintainer of all the package it changes.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants