hackage2nix: Split configuration, auto disable hydra builds#121839
hackage2nix: Split configuration, auto disable hydra builds#121839maralorn merged 1 commit intoNixOS:haskell-updatesfrom
Conversation
|
I have the feeling that the prudent way would be to put the update of the broken-dependencies.yaml in an extra script that only branch-maintainers need to run once in a while. Incurring 60 more seconds of waiting on every contributor might be a bit much. |
|
I have now reduced the number of false positives drastically by not marking broken packages as dont-distribute hackage2nix disables hydra for them anyway. |
b15dd1e to
47f8b18
Compare
|
I have the strong feeling that the remaining "false positives" are not really false positives. They are really broken and should be disabled. But since running this does not change hackage-packages.nix at all their builds were already disabled by hackage2nix for some (probably unrelated) reason. By inspection I can‘t find a reason why hackage2nix disables them, but I think having them in this list is still informative telling us, that they are also disabled because of the broken dependency. |
aef865c to
6dc4181
Compare
|
Okay, I know this a lot to digest, but I think its in good shape now. |
|
btw. there is a reason that it would be nice to merge this not so far in the future. I need to painstakingly make sure that I don‘t loose any changes to configuration-hackage2nix.yaml everytime I rebase this commit on haskell-updates. |
d42e2ef to
649adcf
Compare
sternenseemann
left a comment
There was a problem hiding this comment.
Initial thoughts, didn't do a thorough review though.
|
I will rebase this only when the PR is approved. But I think all voiced concerns up to now have been addressed. I talked with @sternenseemann and we agreed that the solution in the nix file is okay for now. |
sternenseemann
left a comment
There was a problem hiding this comment.
I'm not sure I'm a 100% happy with the nix expression, but we can always adjust the script later — for now it seems to do its job reliably.
Merging this should be a priority since the config format is kinda integral to our workflow and sees a lot of changes.
Also we shouldn't forget to update the documentation…
We split configuration-hackage2nix.yaml into multiple files. We bump cabal2nix-unstable to get support for multiple config files in hackage2nix. * The file main.yaml is only supposed to be edited by humans. * The file stackage.yaml is only supposed to be updated by the update-stackage.sh * The file broken.yaml can be edited by humans, but probably future helpers will want to insert broken packages into this file based on hydra reports. * The file transitive-broken.yaml is newly introduced to be generated by regenerate-transitive-broken-packages.sh regenerate-transitive-broken-packages.sh makes a nix query (in transitive-broken-packages.nix) which evaluates all haskellPackages once with and once without "allowBroken" this way it get's a list of packages which are broken by some transitive dependency, but does not disable packages which have eval errors not caused by a broken package.
Motivation for this change
This is the change mentioned in NixOS/cabal2nix#494.
It’s still a draft and needs to be adapted after #494 is merged. (This concerns especially the regenerate-hackage-packages script, which is patched to do local things right now.)
Since we have multiple scripts editing hackage2nix config and users also doing that, we decided it would be prudent to split the config in multiple files.
This also introduces a script to disable the build of packages which have a broken eval caused by broken dependencies in haskellPackages.
Our dream is that we can drastically reduce the number of packages that we need to mark as broken manually by propagating the dontDistribute flag to reverse dependencies automatically.
This has the added benefit that if a package is fixed reverse dependencies have the chance of also being fixed without human intervention.
I have a few false positives (mostly elm packages) right now which get disabled on hydra for reasons not transparent to me. Need to investigate that.
Also commit messages and code could use more documentation.