Skip to content

Replace Stack with cabal-install on CI#199

Merged
utdemir merged 3 commits intomasterfrom
ud/use-cabal-install-in-ci
Sep 22, 2020
Merged

Replace Stack with cabal-install on CI#199
utdemir merged 3 commits intomasterfrom
ud/use-cabal-install-in-ci

Conversation

@utdemir
Copy link
Copy Markdown
Contributor

@utdemir utdemir commented Sep 22, 2020

Currently, our stack builds are failing, throwing errors like:

...
2020-09-22 15:05:03.459163: [debug] Failed to decode module interface:
/home/utdemir/workspace/github.com/tweag/linear-base6/.stack-work/dist/x86_64-linux-nix/Cabal-3.3.0.0/build/Data/Vector/Linear.hi
Decoding failure: not enough bytes
2020-09-22 15:05:03.459245: [debug] Failed to decode module interface:
/home/utdemir/workspace/github.com/tweag/linear-base6/.stack-work/dist/x86_64-linux-nix/Cabal-3.3.0.0/build/Data/Vector/Mutable/Linear.hi
Decoding failure: not enough bytes
2020-09-22 15:05:03.459334: [debug] Finished with exception in 18ms: getPackageFiles /home/utdemir/workspace/github.com/tweag/linear-base6/linear-base.cabal
Exception thrown: Prelude.chr: bad argument: 942814208
2020-09-22 15:05:03.459729: [error] Prelude.chr: bad argument: 942814208

More importantly, this affects CI (including master): https://buildkite.com/tweag-1/linear-base/builds/350#2971c76e-2967-45dc-9e8f-4c9f5434afd2

I couldn't pinpoint the exact reason since seemingly minor changes (eg. renaming a function parameter) can make the issue disappear. I suspect they are because an interface format change coming with new GHC. If this is the case, this stack issue is likely relevant.

So, as a workaround, we can just replace stack with cabal-install. I tried modifying our CI configuration in this PR and it seems to be happy.

We lose some determinisim there since now a Hackage update can break our build. But maybe that's good since then we can make sure that we keep up with the ecosystem (or add an upper bound).

The alternatives are:

  • Figure out why Stack doesn't work. This can take an indeterminate amount of time. But if anyone has an idea, I'm happy to try.
  • Move the CI to Nix including the Hackage dependencies. I choose not to do this since I didn't want to add yet another place to override our dependencies (besides cabal.project and stack.yaml).
  • Add a cabal.project.freeze file to freeze the dependencies.

@utdemir utdemir mentioned this pull request Sep 22, 2020
Merged
aspiwack
aspiwack approved these changes Sep 22, 2020
View reviewed changes
Copy link
Copy Markdown
Member

@aspiwack aspiwack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem is probably with stack. But given how unstable the bug that we've been observing is, let's not celebrate quite just yet, until we are sure that this fixes it rather than just works by chance.

In the meantime, let's endeavour spend zero more time on this issue. No lockfile, no upperbound. When 9.0 is released, we will revisit all this.

Comment thread cabal.project
Comment on lines +5 to +7
package linear-base
ghc-options: -Wall -Werror

Copy link
Copy Markdown
Member

@aspiwack aspiwack Sep 22, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should really be only on CI, I think.

Copy link
Copy Markdown
Contributor Author

@utdemir utdemir Sep 22, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree. However setting --ghc-options on command line for some reason applies those options to dependencies too, and they don't play nice with -Werror. I think the cabal issue is here: haskell/cabal#3883 (And coincidently today is its 4th year anniversary 🎉 ).

The workaround is to create a cabal.project.ci file, but I don't want to duplicate that file since it also contains some overrides.

Copy link
Copy Markdown
Member

@aspiwack aspiwack Sep 22, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, let's merge as is, then.

@utdemir utdemir merged commit 8b3c19e into master Sep 22, 2020
@utdemir utdemir deleted the ud/use-cabal-install-in-ci branch September 22, 2020 08:42
@aspiwack
Copy link
Copy Markdown
Member

aspiwack commented Sep 24, 2020

Note: I had the problem appear on my home computer, and I (may have) solved it by upgrading stack from 2.1 to 2.3. In case it's relevant when we explore this again.

@aspiwack
Copy link
Copy Markdown
Member

aspiwack commented Sep 24, 2020

Or not… It showed up again. I have no idea what's happening here.

@Divesh-Otwani
Copy link
Copy Markdown
Contributor

Divesh-Otwani commented Sep 24, 2020

@aspiwack @utdemir I've been using cabal in nix-shell --pure since stack is not working.

@utdemir
Copy link
Copy Markdown
Contributor Author

utdemir commented Sep 25, 2020 •
edited
Loading

I got a similar error on cabal-install too: https://buildkite.com/tweag-1/linear-base/builds/398#08cf4d79-5c5c-4a66-b00b-116be492f1f8

However, I think it is slightly different than stack one, because this one happens while the tests are running; but stack was failing to even compile the library. But since they both seem to be encoding related, they possibly have a related cause.

Edit: It seems like cabal issue only happens when cabal tries to print a test failure which contains unicode characters.

@Divesh-Otwani Divesh-Otwani mentioned this pull request Nov 11, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@aspiwack aspiwack aspiwack approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@utdemir @aspiwack @Divesh-Otwani