This is the second of three PRs which implement Relocatable OCaml as proposed in ocaml/RFCs#53. The series of changes in this PR combine to allow the absolute location of the Standard Library (e.g. /usr/lib/ocaml) to be removed from both the C runtime (ocamlrun and libcamlrun.a, etc.) and also from the Config module in the ocamlcommon compiler-libs library. The patches address sections 3 & 4 of the RFC.

The key changes are:

• A new primitive %standard_library_default allows an OCaml program to determine the default of the Standard Library as a compile-time constant. In particular, it means that rather than Config.standard_library_default being fixed once when config.ml itself is compiled, it allows the compiler to determine its value each time a program is linked.
• A new command line option, -set-runtime-default allows the calculated value for the %standard_library_default primitive to be overridden when linking an executable.
• A configure option, --with-relative-libdir, which allows the compiler to be configured to expect to find the Standard Library in a location specified relative to where the compiler is running from. When ./configure is run with no arguments, the default location of the Standard Library is /usr/local/lib/ocaml and binaries are installed to /usr/local/bin. The equivalent relative compiler would be configured with --with-relative-libdir=../lib/ocaml.
• Utilisation of both the BUILD_PATH_PREFIX_MAP support added in Honor BUILD_PATH_PREFIX_MAP #1515 and usage of -fdebug-prefix-map-style options to the C compiler to make the compiler's artefacts considerably more reproducible when --with-relative-libdir has been specified.

These changes necessitate considerable churn in the start-up routines for the bytecode runtime, and in the way argv[0] is being processed. While this code is in motion, there are several additional changes which aren't a strict requirement of the main change, but are here because this is all being shaken up:

• Building the mingw-w64 port using the MSYS2 improves slightly, in that configure now correctly recognises that it is a Cygwin-like environment, and uses cygpath et al as necessary.
• On Windows, if backslashes are included in --prefix (e.g. ./configure --prefix='C:\OCaml') then these are preserved in the resulting compiler (this implements a slightly more sensible version of Display paths using backslashes on Windows #658). Apart from grinding my own axe where this is concerned, it prevents "mixed" slash paths (which look particularly amateur) from being generated when the compiler is configured --with-relative-libdir.
• There is an obscure, but nonetheless present, bug in the order in which the bytecode runtime processes the various options for starting up. In particular, if an executable has been compiled with -custom, it is possible to direct the resulting executable to load a different bytecode image by manipulating argv[0]. This issue is fixed here, and on normal systems (where caml_executable_name is implemented), an executable compiled with -custom only loads the bytecode image it was compiled with and can no longer be directed to a load a different one.
• As part of improving reproducibility, the Cygwin version of ocamlopt now calls as directly, just as Linux does, rather than going via gcc.

  if (stdlib == NULL) stdlib = "/usr/local/lib/ocaml";

let standard_library_default = {|/usr/local/lib/ocaml|}

Printf.printf "Config.standard_library = %S\n" Config.standard_library

$ pwd
/home/dra/work
$ command -v ocamlopt
/usr/local/bin/ocamlopt
$ ocamlopt -o display -I +compiler-libs ocamlcommon.cmxa display.ml
$ ocamlopt -where
/usr/local/lib/ocaml
$ ./display
/usr/local/lib/ocaml

Unix.realpath Config.standard_library
|> Printf.printf "Config.standard_library = %S\n"

$ /usr/local/bin/ocamlc -o display -I +unix -I +compiler-libs unix.cma ocamlcommon.cma display.ml

$ /usr/local/bin/ocamlrun -config
standard_library_default: /usr/local/lib/ocaml
...
shared_libs_path:
  /usr/local/lib/ocaml/stublibs

$ /usr/local/bin/ocamlc -where
/usr/local/lib/ocaml

$ ~/.opam/default/bin/ocamlrun -config
standard_library_default: /home/dra/.opam/default/lib/ocaml
...
shared_libs_path:
  /home/dra/.opam/default/lib/ocaml/stublibs

$ /home/dra/.opam/default/bin/ocamlc -where
/home/dra/.opam/default/lib/ocaml

$ /usr/local/bin/ocamlrun ./display
/usr/local/lib/ocaml

$ ~/.opam/default/bin/ocamlrun ./display
/usr/local/lib/ocaml

The changeset is best reviewed commit-by-commit (and, I'm afraid, armed with the "Technical background" explanation...):

• enum caml_byte_program_mode is augmented with a new APPENDED option which is used for -custom. caml_main then uses caml_byte_program_mode to differentiate between a #!-style "standalone" bytecode image running via ocamlrun and a -custom executable. While moving the code around, I renamed the COMPLETE_EXE enumeration constant to EMBEDDED as the mode is also used with -output-obj.
• The runtime-launch-info is one of two places where the binary directory, rather than library directory is embedded in a file. The format for runtime-launch-info is trivially extended to recognise . as referring to the directory containing the compiler (note that runtime-launch-info is not a generally-configurable file - supporting arbitrary relative paths here would be a hypothetical installation where the runtime executables are installed to a different location from the compiler, which isn't supported or needed at present).
• A symbol caml_runtime_standard_library_default for the runtime default value, principally so that OCAML_STDLIB_DIR is only referred to in once place.
• %standard_library_default is the most involved change, introducing a compile-time constant to retrieve the mutator default value. In native code, the Standard Library location is only present if the Config module is linked. ocamlopt therefore has to create the string as part of linking an executable, which is done using a similar mechanism to the caml_apply, caml_curry and caml_send functions - a new field in the cmx header records that the compilation unit uses the %standard_library_default primitive. At link-time, if any of the compilation units has set this flag, the linker creates caml_standard_library_nat containing the location of the standard library and synthesises references to it. For bytecode, a similar technique is used, except that the linker already has the full list of %-primitives which are used by the program, so there is no need for cmo format to be changed. There are various strategies on offer for exactly where and how the value is stored. It is needed to link the bytecode runtime (i.e. the value has to be included in the same places where C tables of primitives and so forth are generated). It is also has to be included in bytecode images where no C is being produced. Given that -output-complete-exe and so forth share the same value for both runtime and mutator, for bytecode images I've opted to add a new OSLD bytecode section containing the mutator value, which is read to caml_standard_library_default (not to caml_runtime_standard_library_default). Then, as with the other compile-time constants, it's just a matter of adding another caml_sys_const_ primitive in runtime/sys.s which returns an OCaml copy of that string. I've proposed this as a %-primitive, rather than a "known" C primitive for two reasons. Firstly, the behaviour of this is a compile-time constant (i.e. something which the compiler must work out when linking the program), and the other compile-time constants are % primitives. Secondly, the standard library location only wants to be embedded when it's actually needed; if the native runtime used a C primitive, there would have to be a sentinel value or a default for caml_standard_library_nat - it seems worse to have a C primitive sat in the runtime which could return an invalid value (i.e. caml_sys_const_standard_library_default always returns a correct value in bytecode, but if exposed in native code, it would be possible to end up with a program which called it, but got a default empty string back). Given that these primitives are never intended to be called by user-code (because of the difference in linking), unlike in Always define caml_startup et al in bytecode, as already done in native code #13465, I think it's better to have the C primitive for bytecode only, and the completely synthesise the function in ocamlopt only when needed.
• %standard_library_default provides a mechanism which means that the string constant carved into utils/config.ml when the compiler was built can now be determined when the compiler is run. The default value determined by the compiler is simply that same absolute path. The -set-runtime-default provides a mechanism to change that default value when linking a specific program.
• These two features together combine to allow Config.standard_library_default to be changed to be the result of %standard_library_default. This mechanism is a necessary consequence of %standard_library_default for cross-compilation. When linking a cross-compiling version of ocamlopt, that compiler is built with a compiler which uses a host standard library but the resulting cross-compiler it's linking should default to a different target standard library (see the change in Makefile.cross).
• At this point, %standard_library_default means that "/usr/local/lib/ocaml" has now moved the Config module to the executables themselves. Where before, ocamlopt and ocamlc.byte both had the string location linked in code via Config.standard_library, it's now instead embedded in caml_standard_library_nat for ocamlopt (and ocamlc.opt) and in an OSLD section for ocamlc.byte. The next commit then allows explicit-relative values of this path to be interpreted by both the runtime and compiler. This is then activated using -set-runtime-default, so that caml_standard_library_nat is changed to be "../lib/ocaml". Most of the complexity here arises from the fact that the bytecode C runtime needs to be able to do this. Especially in the light of the work on ld.conf in Relocatable OCaml - explicit-relative paths in ld.conf #14243, rather than implementing the logic in both C and OCaml, the logic is implemented just in C and the Config module uses a new caml_sys_get_stdlib_dirs primitive, which is passed the result of %standard_library_default and returns both the effective value and also the directory containing the executable (which is used to implement Config.bindir for ocamlmklib). Note, in passing, that ocamlmklib on Windows now searches for tools in the same way as on Unix, as there's no need for the PATH-search previously there. The C implementation itself is in caml_locate_standard_library which is implemented separately for Unix and Windows in runtime/unix.c and runtime/win32.c. The tools used to implement this (dirname, realpath, etc.) are not equivalently available on Windows, by which I mean that the functions don't have exactly equivalent semantics. In particular, an exact dirname is not available on Windows, but GetFullPathName has the required semantics for this specific use and actually has an option to return the dirname (albeit indirectly). On this occasion, it therefore seemed easier to make the entire "work out the location of the Standard Library" operation platform-specific, than to go to more effort to construct platform-specific building blocks for a generic version of this function.
• The CI matrix for pull requests is altered to test a mix of absolute/relative builds by default. An additional test is added to deal with the "cross-runtime" example the Technical background above. Specifically, after building the compiler, it is duplicated to a new location, and a check is added to ensure ocamlc.byte -where produces the expected result regardless of whether the original or copied ocamlrun is used. If the CI: Full matrix label of GHA: add an optional wider test matrix (Cygwin, static, minimal, etc.) #14013 is added, then each of the jobs builds an additional compiler with the alternate configuration (i.e. the jobs which by default are --without-relative-libdir then have an additional --with-relative-libdir build created, and vice versa). These two compilers are likewise used for the "cross-runtime" ocamlc.byte -where test.
• Finally, there's a little bit of additional plumbing added to go from "Relocatable" to "Reproducible" by utilising BUILD_PATH_PREFIX_MAP and directly adding -ffile-prefix-map to our internal C flags. Ignoring #! lines and RNTM sections (which are addressed in the final PR), when building with --with-relative-libdir, none of the binaries in the build contain either the build path or the installation prefix on Windows (with mingw-w64) or Linux.