Gentle ping: I have split two smaller PRs from this PR. (The commits are included there and also here, so reviewing them first is easier.)

• ocamltest: refactor run_test_tree #13316 , which is purely a refactoring (no behavior change), @MisterDA volunteered to review it
• ocamltest: add an 'ocamltest' action to test ocamltest #13317, which adds an ocamltest action to test ocamltest itself

@shindere gentle ping . This PR still needs a review, and the two "smaller" PRs below it are #13316, which was reviewed by @MisterDA but needs a maintainer approval, and #13317 which needs a review.

Hello, this is on my review list but it’s taking me time to get there. If someone else wants to review these first, feel free, just please let me know so we don’t duplicate work.

I remain interested in having someone review this. @OlivierNicole, if you are not available for an implementation review (which I can very well understand), I would still be interested in your opinion on the proposed design, which I think you could form more easily. (Maybe @lukemaurer has an opinion as well as this is tangentially related to #13259.)

If there is consensus among ocamltest users that this is a reasonable feature to have, I can probably find someone else that cares less about ocamltest and somehow extract an implementation review out of them.

I rephrased the original message of this PR to emphasize what new constructs are proposed, with short usage examples. (The previous description, which is kept after that, is more about the history of changes that lead to this proposal, which makes sense in the context of a specific discussion about a slightly different proposal from @OlivierNicole.)

Hmm! Seems useful, though I can see a couple of pitfalls: You could write

windows;
then { (* win-only tests *) }
else { (* non-win tests *) }

and then someone adds a new condition somewhere higher in the file, and if I understand correctly, if that new condition fails, you're going to end up running the non-Windows tests on Windows. This suggests introducing an explicit if to the syntax to tie to then and else?

Also if someone writes

{ A }
else { B }

it will look like else is sort of like catch: do the things in A and if they fail/skip then do the things in B. Possibly then should be required in this case?

Thanks for the feedback.

One difficulty with introducing an if is that currently it's easy to implement conjunction by just running the two checks in sequence, but I cannot do this with if anymore -- or I have to implement boolean operators, meh. On the other hand, it's possible to scope the conditional part:

{
  windows;
  flambda;
  then { ... } /* windows and flamdba */
  else { ... } /* not windows or not flambda */
}

One difficulty with introducing an if is that currently it's easy to implement conjunction by just running the two checks in sequence, but I cannot do this with if anymore -- or I have to implement boolean operators, meh.

Hmm ... I had been thinking if { windows; flambda; } then { ... } else { ... } but then you have to restrict the grammar of the if part so that it doesn't have branches (or you no longer have a tree structure for your dependencies).

On the other hand, it's possible to scope the conditional part:

Yeah, that seems good. So if the surrounding block is skipped, then the whole then/else is skipped, but once we enter the block, either the then or the else must happen.

Sorry for the delay in checking out this design.

Semantics of then, else blocks:

• if one of the actions before the block skipped, we skip then blocks and execute else blocks (in parallel)

• if none of the actions before the block skipped, we execute then blocks (in parallel) and skip else blocks

I am unsure about what is done with the else blocks. My mental model of a test spec is that each several actions in sequence are like successive section titles of decreasing importance, and blocks are inside the latest section and their order doesn’t matter. With that mental model, I agree with the semantics of then { ... }, but I would have expected else { ... } to be run only if the last action skipped or failed.

Doing so would avoid the behavior that @lukemaurer finds counter-intuitive (I find it odd, too). It seems to me that it would make the semantics of action1; action2; action3; then { ... } else { ... } pretty much like a sequence of two operations that can fail, followed by an ordinary conditional. It makes things easier to read and less error-prone, at least for me.

I guess @lukemaurer’s proposal of having blocks as conditions is even better as it would allow for conjunctions.

I don't mind going with if { <sequence> } ((then | else) { <block> })* as suggested. (There can be several then or else branch, independent of / parallel to each other.) Maybe we should use ( <sequence> ) instead of { <sequence> } to highlight that this is less expressive than a full block -- no parallelism/independence inside.

I started splitting this PR to make progress again, in particular #14502 implements the easy part, which is not <action>. We'll see about conditionals as a second step.