Note that this isn't a bug fix, but a breaking change. The old behavior intentionally disabled running, instead of building the test. We could have a knob to tweak to disable building, but we can't just remove the old behavior. For inline tests, we have knobs both for building and running the tests. We should probably have something similar here.

Can't this be controlled by requiring (lang dune 3.8)? It's really important to be able to disable building tests on platforms where they don't compile.

There's a PR on Eio (ocaml-multicore/eio#416) switching to (* -*- tuareg -*- *) mode to work around this. I've been holding off merging that PR in the hope this would be fixed soon in dune.

If I want to build a test but not run it, there are easy ways to do that.

I still think that it doesn't make sense to favor one over the other. In retrospect, we should have had build_if and run_if parameters. Do you want to revive this PR and add a build_if parameter instead?

Is this behaviour documented anywhere? I assumed enabled_if always controlled whether a stanza was enabled or not. The documentation is very vague about it: tests delegates to executables, which delegates to libraries, which obviously doesn't say anything about what it means for tests. But it does seem like it's supposed to have the same meaning in both places, so "build but don't test" doesn't make sense.

It might make sense to have a skip_if stanza for skipping tests, but usually test systems have their own way of doing that.

You have a point that the correct interpretation of enabled_if doesn't make sense for the tests stanza, but it's not something we can just change in 3.8. Upgrading the dune language version should be mostly seamless, so changing the interpretation to be something else would only be a change we're willing to make in 4.0.