Improve: dunify testsuite#881
Conversation
|
Note: I made those tests pass using |
|
Also, if people were willing to give up on the html output, I think we could just get rid of test.sh and switch everything to dune. |
We have a script |
I missed that one, I tried the various options of Anyway, I pushed some more commits, I removed |
|
The test failure seems to be because I guess I want to pass a dune-workspace file manually in that case. (Because I didn't see how to select a specific context from the command line). |
| regtests: | ||
| @echo running regtests with $(OCAMLFORMAT_EXE) | ||
| @./test.sh --ocamlformat $(OCAMLFORMAT_EXE) | ||
| @dune runtest |
There was a problem hiding this comment.
Shouldn't use --profile=$(MODE) ? Otherwise the echo above will be wrong and the MODE variable useless.
There was a problem hiding this comment.
I agree the echo will be wrong, but I don't think that flag does what you want exactly. I think profile can only be set to release or dev. Which is different from the build context, which afaict one cannot set from the command line.
@rgrinberg: can you confirm?
There's no good way to handle this at the moment. There was a suggestion to add an action that checks for a particular exit code, but no concrete ticket/rfc. If you'd like this feature, then please make a ticket. |
I was a bit confused by the in-repo testsuite when I started hacking on ocamlformat. More specifically, I still don't know how to automatically accept a diff, I currently run
./test.sh --meldfrom the test directory (which I assume is not the most efficient way to do things).Since the dune workflow for diffing and promoting seems particularly adapted, this PR proposes to add some dune files under
test/.I needed some minor modifications to the formatting of
.optsfiles to make dune happy, but apart from that I don't think there is any risk of breaking the preexisting test framework.This is still incomplete (I only add a dune file in
tests/passing), but I figured it was a good point to gather feedback and ask for help.An issue I'm currently having is that there are some tests which are expected to fail (stderr is recorded), and for which ocamlformat will have a non-zero exit code. And I don't know how to tell dune "it's expected to fail, don't worry". @rgrinberg: is that something that dune knows how to handle? Or can I at least ignore the exit code?