Just a note about the use case/how I found about this.

I have a library (topkg) that I want to be able to use both as executable (script, in which case a main is automatically run) and in library mode to be used by other programs (in which case these other programs have to call a library function prevent_main to inhibit the automatic main).

One mechanism to do this is to have an at_exit function that checks if prevent_main was called and if not to run the main function. However that main function may want to be able to signal an exit code in case of errors. This was previously not possible since calling exit there would result in an inifinite loop.

Here's the piece of code, I worked around by calling main in describe but there's still an error case "nothing useful happened" that is currently only logged on stderr but that I would like to be able to exit with a non-zero error code.

I want to be able to use both as executable (script, in which case a main is automatically run) and in library mode to be used by other programs (in which case these other programs have to call a library function prevent_main to inhibit the automatic main)

I know this is not really the topic of this PR, but running the main program as part of the at_exit hook does not seem the most natural thing in the world, and the opt-out feature can be risky (imagine some toplevel initialization code in the user code raising an exception before the user code had a chance to call prevent_main). The classical approach is to provide a tiny unit (outside the library) that simply calls your main function; when the library is linked as an executable, this unit is added. Doesn't that work in your context?

but running the main program as part of the at_exit hook does not seem the most natural thing in the world,

In fact in my example I don't run main in the at_exit hook. I only check that something useful happened and print an error if that is not the case and want to be able to define the error exit code there so that the user realizes something went wrong (also because ocaml is quite broken w.r.t. to exit codes in case of errors see MPR7202).

The classical approach is to provide a tiny unit (outside the library) that simply calls your main function; when the library is linked as an executable, this unit is added. Doesn't that work in your context?

In this context I didn't want to prevent myself from being able to reuse the same file both as an executable and be able to compile it and link it in a larger program so that wouldn't work.

The doc for [exit] should mention that the control flow can now return from the function (through an exception). Users need to be aware of that if they have any kind of catch-all exception handlers above the call to exit (e.g. a proper try-finally idiom that release some resource, or some generic exception logger).

Did you explore alternative ways to prevent the infinite loop (e.g. forcing termination on the second nested call; or keeping a list of pending at_exit functions and continuing directly with the next one when one of them calls [exit])? And with ways to have at_exit hooks change the exit code directly without calling [exit]? (E.g. val at_exit_with_code: (int -> int) -> unit)

The doc for [exit] should mention that the control flow can now return from the function (through an exception). Users need to be aware of that if they have any kind of catch-all exception handlers above the call to exit (e.g. a proper try-finally idiom that release some resource, or some generic exception logger).

Didn't think about that. In my opinion this makes the fix unsuitable. I think I'll close this for now and think about a way that also allows to fix MPR7253 at the same time.

What about using C's longjmp to jump back to a main loop?

Along the lines of what Alain suggests I think we should rather do this:

1. Calling exit in an at_exit function raises Invalid_argument (i.e. is a programming error). This seems better than an infinite loop. However this needs a fix of MPR7253 otherwise it gets confusing.
2. Introduce a function exit_at_exit : int -> unit (note the unit in contrast to 'a for exit) that allows to set the ret code in an at_exit function with the semantics that the last exit_at_exit executed with non-zero exit code defines the retcode of the program.