Skip to content

haskell-generic-builder: allow passing flags to the test suite(s)#126364

Merged
maralorn merged 1 commit intoNixOS:haskell-updatesfrom
sternenseemann:haskell-test-flags
Jun 12, 2021
Merged

haskell-generic-builder: allow passing flags to the test suite(s)#126364
maralorn merged 1 commit intoNixOS:haskell-updatesfrom
sternenseemann:haskell-test-flags

Conversation

@sternenseemann
Copy link
Copy Markdown
Member

Every flag the generic builder receives via testFlags is passed via
--test-option [1] to Setup.hs which in turn passes them to the
underlying test suite binary. These wrapped options are added to
checkFlagsArray in checkPhase. This needs to be done in bash since
without structuredAttrs in nixpkgs so far, Nix arrays aren't properly
translated into bash arrays, so we'd have all sorts of quoting issues
when spaces are involved.

Re-using checkFlags and checkFlagsArray from standard stdenv
setup.sh also results in an additional feature: Using overrideAttrs
checkFlags and checkFlagsArray can additionally be overridden,
which allows passing extra flags to Setup.hs whithout being wrapped
with --test-option.

[1]: See also https://cabal.readthedocs.io/en/3.4/setup-commands.html?highlight=test-option#cmdoption-runhaskell-Setup.hs-test-test-option
According to the cabal-install man page this also allows passing
special variables which are substituted for other values
depending on context.

Motivation for this change

Add a clean way (without abusing testTarget) to pass extra flags to test suites and/or ./Setup test.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • 21.11 Release Notes
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Every flag the generic builder receives via `testFlags` is passed via
`--test-option` [1] to `Setup.hs` which in turn passes them to the
underlying test suite binary. These wrapped options are added to
`checkFlagsArray` in `checkPhase`. This needs to be done in bash since
without structuredAttrs in nixpkgs so far, Nix arrays aren't properly
translated into bash arrays, so we'd have all sorts of quoting issues
when spaces are involved.

Re-using `checkFlags` and `checkFlagsArray` from standard stdenv
setup.sh also results in an additional feature: Using `overrideAttrs`
`checkFlags` and `checkFlagsArray` can additionally be overridden,
which allows passing extra flags to `Setup.hs` whithout being wrapped
with `--test-option`.

[1]: See also https://cabal.readthedocs.io/en/3.4/setup-commands.html?highlight=test-option#cmdoption-runhaskell-Setup.hs-test-test-option
     According to the cabal-install man page this also allows passing
     special variables which are substituted for other values
     depending on context.
@github-actions github-actions bot added the 6.topic: haskell General-purpose, statically typed, purely functional programming language label Jun 9, 2021
@ofborg ofborg bot added 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 5001+ This PR causes many rebuilds on Darwin and must target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 5001+ This PR causes many rebuilds on Linux and must target the staging branches. labels Jun 9, 2021
@sternenseemann
Copy link
Copy Markdown
Member Author

$ nix-build -A haskellPackages.libarchive -A haskellPackages.alpaca-netcode
/nix/store/14wim3ssk7bg757wy1x9w68y0j6hwf3p-libarchive-3.0.2.2
/nix/store/815491m7rc8f58kwnzd9lw6msrx55vqj-alpaca-netcode-0.1.0.0

@maralorn
Copy link
Copy Markdown
Member

maralorn commented Jun 9, 2021

I’ll have to trust your bash magic here. But I don‘t see anything wrong with this.

@sternenseemann
Copy link
Copy Markdown
Member Author

I’ll have to trust your bash magic here. But I don‘t see anything wrong with this.

I'm fairly confident it's fine. It is mostly based on what pkgs/stdenv/generic/setup.sh is doing. I've also tried some alternatives to this approach, but this seems to be by far the best as it adds the ability to use checkFlags and checkFlagsArray as a raw way to pass flags to ./Setup test.

We may need to wait with merging until your 2 weeks though as we are kind of starved for builds, it seems.

, testDepends ? [], testHaskellDepends ? [], testSystemDepends ? [], testFrameworkDepends ? []
, benchmarkDepends ? [], benchmarkHaskellDepends ? [], benchmarkSystemDepends ? [], benchmarkFrameworkDepends ? []
, testTarget ? ""
, testTarget ? "", testFlags ? []
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the arguments aren't documented here, I guess you could add a helper function in haskell.lib and add documentation and an example for this argument.

Although I don't think that is strictly necessary for this PR to be merged, more of a nice-to-have.

I also don't know bash well enough to work out if the checkFlagsArray expression is correct, but it looks like it at least works for your two examples, so it looks good to me!

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit sceptical that it is actually a good thing having all these functions for specific arguments. In some cases they are justified when there is an element of “doing the right thing” to it (like appendConfigureFlags, …) and when the overrides are more complicated yet reusable (generateOptparseApplicativeCompion, justStaticExecutables, ...).

In this case overrideCabal is perfectly fine for setting this and my intention is to document this in the manual, I have already documented all existing arguments to haskellPackages.mkDerivation (which was apparently needed as I had to check the cabal2nix source for some details…) and would add testFlags there as well when this is merged.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking that ideally this documentation could go here in pkgs/development/haskell-modules/generic-builder.nix, immediately above the place where it is defined.

Although it would be good if we could generate the manual directly from this file.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This as well, but I think the actual manual is actually more important at this time, I'm hoping nixdoc will get some love in the future to support this use case as well (it kinda does currently, but also kinda doesn't).

@maralorn maralorn merged commit 222b6c8 into NixOS:haskell-updates Jun 12, 2021
@sternenseemann sternenseemann deleted the haskell-test-flags branch June 12, 2021 23:15
@sternenseemann sternenseemann restored the haskell-test-flags branch July 24, 2021 13:36
@sternenseemann sternenseemann deleted the haskell-test-flags branch February 11, 2025 14:46
sternenseemann added a commit to NixOS/cabal2nix that referenced this pull request Feb 11, 2025
testFlags was introduced in NixOS/nixpkgs#126364
and has been available since NixOS 21.11.

Future work is removing these PostProcessing rules altogether,
see #504.
sternenseemann added a commit to NixOS/cabal2nix that referenced this pull request Feb 14, 2025
testFlags was introduced in NixOS/nixpkgs#126364
and has been available since NixOS 21.11.

Future work is removing these PostProcessing rules altogether,
see #504.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: haskell General-purpose, statically typed, purely functional programming language 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 5001+ This PR causes many rebuilds on Darwin and must target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 5001+ This PR causes many rebuilds on Linux and must target the staging branches.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants