ocamltest: add an 'ocamltest' action to test ocamltest#13317
ocamltest: add an 'ocamltest' action to test ocamltest#13317gasche merged 1 commit intoocaml:trunkfrom
Conversation
| Result.skip_with_reason "ocamlobjinfo not available", env | ||
| ) | ||
|
|
||
| let ocamltest_action log env = |
There was a problem hiding this comment.
Note for reviewers: to review this, just compare to the mklib stuff below -- on the github interface this requires expanding the diff a bit.
|
This is a simple PR and I wish something would help by reviewing, as it could unblock further contributions to ocamltest. Maybe @OlivierNicole? |
|
I will review this today. |
nojb
left a comment
There was a problem hiding this comment.
LGTM
Would you mind adding a simple test of the new action just to illustrate concretely how it is used? Thanks!
|
I have decided to restart working on negation in ocamltest (a new version of #13315 with only |
|
Note: #14502 did include a test based on this PR, see https://github.com/gasche/ocaml/blob/71d3bc3c1e970cad17fe218569bc16f6b4f054ef/testsuite/tests/tool-ocamltest/semantics.ml |
This is part of #13315, split out for easier reviewing. It adds an
ocamltest;action in ocamltest, which allows to test ocamltest scripts. This sounds like a generally useful feature that I think we should integrate no matter what happens to #13315.For example, suppose I want to test that the
not <test>;feature from #13315 works properly. I write the following test script:The problem is that if I just add this to the testsuite, I will see that this test script succeeds, but I don't know if the feature works as intended. Maybe
not foo;always skips, and therefore this test passes without doing what I think it does. I can check that this does not happen myself, by tweaking the test to say{ not skip; fail }instead of{ not skip; pass }, and checking that that fails. But I cannot commit this to the testsuite. How do I leave evidence in the testsuite that thepassaction after thenot skipindeed runs?My solution is that instead of writing this as a
negation.mltest, called with the usual driver, I write my test above as anegation.testfile, with asemantics.mldriver test that reads as follows:And then in
negation.referenceI put the result of runningocamltest -e negation.test, which is mostly unreadable but changes if I change the test or thenotimplementation. At this point I'm happy.