The BUILD_PATH_PREFIX_MAP specification tells how to use that
environment variable to achieve reproducible build, i.e. builds of
products that do not leak absolute paths from the build
environment. See
https://reproducible-builds.org/specs/build-path-prefix-map.

However, that specification only covers half of the story.  Let us
call the building of reproducible products the "Build Phase". That is
the phase covered by the existing specification.

Let us defined the "Deployment phase" as the phase where you accept a
built reproducible product and make use of it, i.e. deploy it. An
example would be the debugger taking a reproducible binary and letting
the user debug it, showing the user the source code, etc. These
changes were originally being made for the benefit of the debugger,
but I realized they are more generally applicable and that a
generalization of the spec was desirable.

I have a proposed revision of that specification.  I have capture the
original specification on github, and then created a pull requests
with my proposed changes. See

richardlford/build-path-prefix-map-spec#1

This PR implements the revised specification.

Also, this is part of a larger PR,
ocaml#12085, that is being split up
because it was too large. In addition to these changes, that PR
includes ocamltest enhancements plus the tests for
these changes. See it for prior discussion and the other changes.

I will now describe the essence of the changes. See the
revised document for motivation.

1. utils/build_path_prefix_map.{ml,mli}

The essential change to the spec is that left-hand side of
a mapping pair (formerly the target) now may have a
list of targets, separated by ';'. Because ';' is
now used as a delimiter, it is escaped when
a path is encoded (to "%,"). Because of the encoding,
there is no restriction on characters in a path.

A list of targets that may be returned is called a
"search list". The API is extended with functions
that return search lists.

During the build phase we expect that the maps will only
have a single target. An open issue is what to do
if there is more than one target during the
build phase. Currently we ignore the mapping.

2. parsing/location.{ml, mli}

This has convenience functions for using the above
mapping API. It reads and caches the environment
variable so the end-user does not need to.

a. absolute_path was modified so that
BUILD_PATH_PREFIX_MAP rewriting is done for both
absolute and relative paths. Relative paths
are made absolute by appending to the cwd.
Previously only relative paths were rewritten.
One discovery during testing was that if the
compiler is given an absolute path as the
input source, the debug event information
had that absolute path (see below).

b. rewrite_to_search_list
New function which returns search list from the mapping.

c. search_list_find
New function that uses rewrite_to_search_list to get
a list of paths and then looks for the first one that
exists.

3. bytecomp/emitcode.ml
Added sanitizing of paths in the debug events.
If the compiler is given absolute source paths
this was leaking absolute build paths.

4. bytecomp/bytelink.ml
Never rewrite when producing the path for a shebang.
The value written is from the user "-use-runtime"
option, and I do not believe that should be rewritten.

5. utils/load_path.ml
Modified so that a directory is not added to the load path
if it is already in the load path. I was concerned that
failing to do this might not preserve the desired order.

6. debugger/command_line.ml

A debugger variable, "mapping", was added, which is tied to the
BUILD_PATH_PREFIX_MAP environment variable.  So "set mapping value"
will set BUILD_PATH_PREFIX_MAP, and "show mapping" will show the value
of BUILD_PATH_PREFIX_MAP. This allows the user to set it
programmaticaly, possibly from the ".ocamldebug" file.

7. debugger/program_management.ml
Reverse the list of directories so the ones with highest
priority are first.

8. debugger/source.ml
In source_of_module, use the Location.search_list_find API.

9. debugger/symbols.{ml, mli}
a. Take care to preserve the order of directories, so the
highest priority will come first.
b. Expose bppm_expand_path and get_load_path so they
can be called from a fake printer which, when
loaded and installed in a test, can do unit testing
of ocamldebug.

10. utils/misc.ml
When printing mapping, take into account multiple targets.

11. Man page and manual updated.

Fixes ocaml#12083

This is part 2 of a larger PR, ocaml#12085, which includes a part 1
(compiler and debugger changes), this part, and a third part which is
tests of part 1 that make use of (and hence motivate and test) these
changes. Please see that PR for example uses of these changes.

1. Add the ability to have scripts or files with ocamltest variables
references expanded. Added a new "expand" action to ocamltest. It is
mostly like the "copy" action, but the source file is read a line at a
time and the lines are expanded. It does not support source
directories, but the destination can be a directory.

2. Enhanced ocamltest to have a facility for making builtin
functions. These are like ocamltest variables but have a function
attached to them, and when the variable is expanded, the arguments are
first expanded and then the function is called and its result is
returned. Currently they only take one parameter, but it should not be
too hard to add the ability to have multiply arguments.

3. Added a new "dumpenv_expanded" action which not
only shows the value of each variable, but also what
they expand to. When variables are originally
assigned, their RHS is not expanded, but only
later when the variable itself is looked up.

4. Two builtin functions were defined:

- bppm_decode does BUILD_PATH_PREFIX_MAP decoding

- bppm_encode does BUILD_PATH_PREFIX_MAP encoding

See https://reproducible-builds.org/specs/build-path-prefix-map/
and richardlford/build-path-prefix-map-spec#1.

5. Prepare to be able to do Dune tests:

5a. Added a action helper for testing if a program
is available in PATH.

5b. Use the helper to make an action, has_dune,
that detects whether dune is available.

The BUILD_PATH_PREFIX_MAP specification tells how to use that
environment variable to achieve reproducible build, i.e. builds of
products that do not leak absolute paths from the build
environment. See
https://reproducible-builds.org/specs/build-path-prefix-map.

However, that specification only describes half of the story.  Let us
call the building of reproducible products the "Build Phase". That is
the phase covered by the existing specification.

Let us defined the "Deployment phase" as the phase where you accept a
built reproducible product and make use of it, i.e. deploy it. An
example would be the debugger taking a reproducible binary and letting
the user debug it, showing the user the source code, etc.

We use the same mechanism in the deployment phase. Then
the BUILD_PATH_PREFIX_MAP must be setup with the logical
inverse of the mapping used during the build phase.

Also, this is part of a larger PR,
ocaml#12085, that is being split up
because it was too large. In addition to these changes, that PR
includes ocamltest enhancements plus the tests for
these changes. See it for prior discussion and the other changes.

I will now describe the essence of the changes. See the
revised document for motivation.

1. parsing/location.{ml, mli}

This has convenience functions for using the above
mapping API. It reads and caches the environment
variable so the end-user does not need to.

Location.absolute_path was modified so that
BUILD_PATH_PREFIX_MAP rewriting is done for both
absolute and relative paths. Relative paths
are made absolute by appending to the cwd.
Previously only relative paths were rewritten.
One discovery during testing was that if the
compiler is given an absolute path as the
input source, the debug event information
had that absolute path (see below).

2. bytecomp/emitcode.ml

Added sanitizing of paths in the debug events.
If the compiler is given absolute source paths
this was leaking absolute build paths.

3. bytecomp/bytelink.ml

Rewrite when producing the path for a shebang.
The value written is from the user "-use-runtime"
option. The one setting BUILD_PATH_PREFIX_MAP
will have control whether this does anything or not.

4. debugger/command_line.ml

A debugger variable, "mapping", was added, which is tied to the
BUILD_PATH_PREFIX_MAP environment variable.  So "set mapping value"
will set BUILD_PATH_PREFIX_MAP, and "show mapping" will show the value
of BUILD_PATH_PREFIX_MAP. This allows the user to set it
programmaticaly, possibly from the ".ocamldebug" file.

5. debugger/source.ml

In source_of_module, use the Location.rewrite_absolute_path API.

6. debugger/symbols.{ml, mli}

a. Avoid adding directories redundantly

b. Expose bppm_expand_path and get_load_path so they
can be called from a fake printer which, when
loaded and installed in a test, can do unit testing
of ocamldebug.

7. Man page and manual updated.

Fixes ocaml#12083

The BUILD_PATH_PREFIX_MAP specification tells how to use that
environment variable to achieve reproducible build, i.e. builds of
products that do not leak absolute paths from the build
environment. See
https://reproducible-builds.org/specs/build-path-prefix-map.

However, that specification only describes half of the story.  Let us
call the building of reproducible products the "Build Phase". That is
the phase covered by the existing specification.

Let us defined the "Deployment phase" as the phase where you accept a
built reproducible product and make use of it, i.e. deploy it. An
example would be the debugger taking a reproducible binary and letting
the user debug it, showing the user the source code, etc.

We use the same mechanism in the deployment phase. Then
the BUILD_PATH_PREFIX_MAP must be setup with the logical
inverse of the mapping used during the build phase.

Also, this is part of a larger PR,
ocaml#12085, that is being split up
because it was too large. In addition to these changes, that PR
includes ocamltest enhancements plus the tests for
these changes. See it for prior discussion and the other changes.

I will now describe the essence of the changes. See the
revised document for motivation.

1. parsing/location.{ml, mli}

This has convenience functions for using the above
mapping API. It reads and caches the environment
variable so the end-user does not need to.

Location.absolute_path was modified so that
BUILD_PATH_PREFIX_MAP rewriting is done for both
absolute and relative paths. Relative paths
are made absolute by appending to the cwd.
Previously only relative paths were rewritten.
One discovery during testing was that if the
compiler is given an absolute path as the
input source, the debug event information
had that absolute path (see below).

2. bytecomp/emitcode.ml

Added sanitizing of paths in the debug events.
If the compiler is given absolute source paths
this was leaking absolute build paths.

3. bytecomp/bytelink.ml

Rewrite when producing the path for a shebang.
The value written is from the user "-use-runtime"
option. The one setting BUILD_PATH_PREFIX_MAP
will have control whether this does anything or not.

4. debugger/command_line.ml

A debugger variable, "mapping", was added, which is tied to the
BUILD_PATH_PREFIX_MAP environment variable.  So "set mapping value"
will set BUILD_PATH_PREFIX_MAP, and "show mapping" will show the value
of BUILD_PATH_PREFIX_MAP. This allows the user to set it
programmaticaly, possibly from the ".ocamldebug" file.

5. debugger/source.ml

In source_of_module, use the Location.rewrite_absolute_path API.

6. debugger/symbols.{ml, mli}

a. Avoid adding directories redundantly

b. Expose bppm_expand_path and get_load_path so they
can be called from a fake printer which, when
loaded and installed in a test, can do unit testing
of ocamldebug.

7. Man page and manual updated.

Fixes ocaml#12083

The BUILD_PATH_PREFIX_MAP specification tells how to use that
environment variable to achieve reproducible build, i.e. builds of
products that do not leak absolute paths from the build
environment. See
https://reproducible-builds.org/specs/build-path-prefix-map.

However, that specification only describes half of the story.  Let us
call the building of reproducible products the "Build Phase". That is
the phase covered by the existing specification.

Let us defined the "Deployment phase" as the phase where you accept a
built reproducible product and make use of it, i.e. deploy it. An
example would be the debugger taking a reproducible binary and letting
the user debug it, showing the user the source code, etc.

We use the same mechanism in the deployment phase. Then
the BUILD_PATH_PREFIX_MAP must be setup with the logical
inverse of the mapping used during the build phase.

Also, this is part of a larger PR,
ocaml#12085, that is being split up
because it was too large. In addition to these changes, that PR
includes ocamltest enhancements plus the tests for
these changes. See it for prior discussion and the other changes.

I will now describe the essence of the changes.

1. utils/build_path_prefix_map.{ml,mli}

Added these functions to aid in use of the mapping
during the deployment phase. They allow to skip
mapping entries that are not applicable (find_rewrite).
find_rewrite is then used to implement rewrite_exists and
matching_dirs.

val find_rewrite : map -> (path -> bool) -> path -> path option
(** [rewrite_opt map pred path] tries to find a source in [map]
    that is a prefix of the input [path] that produces a result
    that satisfies predicate [pred]. If it succeeds,
    it replaces this prefix with the corresponding target.
    If it fails, it just returns [None]. *)

val rewrite_exists : map -> path -> path option
(** [rewrite_exists map path] tries to find a source in [map]
    that maps to a result that exists in the file system.
    If so, returns Some result. Otherwise, is there was
    at least one source that was a prefix of path, it is
    assumed that the map is deficient and Not_found is raised.
    If no source prefixes were found, None is returned.
*)

val matching_dirs : map -> path -> path list
(** [matching_dirs map absdir] accumulates a list of existing directories,
  [dirs], that are the result of mapping abstract directory, [absdir],
  over all the mapping pairs in [map]. The list [dirs] will be in
  priority order (head as highest priority). If there is success,
  returns [dirs]. If no source in the map was a prefix of [absdir],
  returns [[]]. If at least one source in the map was a prefix of
  [absdir], but none of the mappings were existing directories,
  raise Not_found.
*)

2. parsing/location.{ml, mli}

This has convenience functions for using the above
mapping API. It reads and caches the environment
variable so the end-user does not need to.

Location.absolute_path was modified so that BUILD_PATH_PREFIX_MAP
rewriting is done for both absolute and relative paths. Relative paths
are made absolute by appending to the cwd.  Previously only relative
paths were rewritten.  One discovery during testing was that if the
compiler is given an absolute path as the input source, the debug
event information had that absolute path (see below).

New functions are:

val rewrite_exists: string -> string option option
(** [rewrite_exists path] uses a BUILD_PATH_PREFIX_MAP mapping
    (https://reproducible-builds.org/specs/build-path-prefix-map/)
    and tries to find a source in mapping
    that maps to a result that exists in the file system.
    There are the following return values:
    - None, means BUILD_PATH_PREFIX_MAP is not set.
    - Some None, no source prefixes of [path] in the mapping were found,
    - Some (Some target), means target is the first file (in priority
      order) that [path] mapped to that exists in the file system.
    - Not_found raised, means some source prefixes in the map
      were found that matched [path], but none of them existed
      in the file system. *)

val matching_dirs: string -> string list option
(** [matching_dirs absdir] accumulates a list of existing directories,
    [dirs], that are the result of mapping abstract directory, [absdir],
    over all the mapping pairs in the BUILD_PATH_PREFIX_MAP environment
    variable. The list [dirs] will be in priority order (head as highest
    priority). The possible results are:
    - None, means BUILD_PATH_PREFIX_MAP is not set.
    - Some [], no source prefixes in the mapping were found,
    - Some dirs, means dirs are the directories found
    - Not_found raised, means some source prefixes in the map
      were found that matched [path], but none of mapping results
      were existing directories.
    See the BUILD_PATH_PREFIX_MAP spec at
    (https://reproducible-builds.org/specs/build-path-prefix-map/)
    *)

3. bytecomp/emitcode.ml

Added sanitizing of paths in the debug events.
If the compiler is given absolute source paths
this was leaking absolute build paths.

4. bytecomp/bytelink.ml

Rewrite when producing the path for a shebang.
The value written is from the user "-use-runtime"
option. The one setting BUILD_PATH_PREFIX_MAP
will have control whether this does anything or not.

5. debugger/command_line.ml

A debugger variable, "mapping", was added, which is tied to the
BUILD_PATH_PREFIX_MAP environment variable.  So "set mapping value"
will set BUILD_PATH_PREFIX_MAP, and "show mapping" will show the value
of BUILD_PATH_PREFIX_MAP. This allows the user to set it
programmaticaly, possibly from the ".ocamldebug" file.

6. debugger/source.ml

In source_of_module, use the Location.rewrite_exists.

6. debugger/symbols.{ml, mli}

a. Avoid adding directories redundantly

b. Expose bppm_expand_path and get_load_path so they
can be called from a fake printer which, when
loaded and installed in a test, can do unit testing
of ocamldebug.

c. Use Location.matching_dirs to expand directories.

7. Man page and manual updated.

Fixes ocaml#12083

The BUILD_PATH_PREFIX_MAP specification tells how to use that
environment variable to achieve reproducible build, i.e. builds of
products that do not leak absolute paths from the build
environment. See
https://reproducible-builds.org/specs/build-path-prefix-map.

However, that specification only describes half of the story.  Let us
call the building of reproducible products the "Build Phase". That is
the phase covered by the existing specification.

Let us defined the "Deployment phase" as the phase where you accept a
built reproducible product and make use of it, i.e. deploy it. An
example would be the debugger taking a reproducible binary and letting
the user debug it, showing the user the source code, etc.

We use the same mechanism in the deployment phase. Then
the BUILD_PATH_PREFIX_MAP must be setup with the logical
inverse of the mapping used during the build phase.

Also, this is part of a larger PR,
ocaml#12085, that is being split up
because it was too large. In addition to these changes, that PR
includes ocamltest enhancements plus the tests for
these changes. See it for prior discussion and the other changes.

I will now describe the essence of the changes.

1. utils/build_path_prefix_map.{ml,mli}

Added these functions to aid in use of the mapping
during the deployment phase. They allow to skip
mapping entries that are not applicable (find_rewrite).
find_rewrite is then used to implement rewrite_exists and
matching_dirs.

val find_rewrite : map -> (path -> bool) -> path -> path option
(** [rewrite_opt map pred path] tries to find a source in [map]
    that is a prefix of the input [path] that produces a result
    that satisfies predicate [pred]. If it succeeds,
    it replaces this prefix with the corresponding target.
    If it fails, it just returns [None]. *)

val rewrite_exists : map -> path -> path option
(** [rewrite_exists map path] tries to find a source in [map]
    that maps to a result that exists in the file system.
    If so, returns Some result. Otherwise, is there was
    at least one source that was a prefix of path, it is
    assumed that the map is deficient and Not_found is raised.
    If no source prefixes were found, None is returned.
*)

val matching_dirs : map -> path -> path list
(** [matching_dirs map absdir] accumulates a list of existing directories,
  [dirs], that are the result of mapping abstract directory, [absdir],
  over all the mapping pairs in [map]. The list [dirs] will be in
  priority order (head as highest priority). If there is success,
  returns [dirs]. If no source in the map was a prefix of [absdir],
  returns [[]]. If at least one source in the map was a prefix of
  [absdir], but none of the mappings were existing directories,
  raise Not_found.
*)

2. parsing/location.{ml, mli}

This has convenience functions for using the above
mapping API. It reads and caches the environment
variable so the end-user does not need to.

Location.absolute_path was modified so that BUILD_PATH_PREFIX_MAP
rewriting is done for both absolute and relative paths. Relative paths
are made absolute by appending to the cwd.  Previously only relative
paths were rewritten.  One discovery during testing was that if the
compiler is given an absolute path as the input source, the debug
event information had that absolute path (see below).

New functions are:

val rewrite_exists: string -> string option option
(** [rewrite_exists path] uses a BUILD_PATH_PREFIX_MAP mapping
    (https://reproducible-builds.org/specs/build-path-prefix-map/)
    and tries to find a source in mapping
    that maps to a result that exists in the file system.
    There are the following return values:
    - None, means BUILD_PATH_PREFIX_MAP is not set.
    - Some None, no source prefixes of [path] in the mapping were found,
    - Some (Some target), means target is the first file (in priority
      order) that [path] mapped to that exists in the file system.
    - Not_found raised, means some source prefixes in the map
      were found that matched [path], but none of them existed
      in the file system. *)

val matching_dirs: string -> string list option
(** [matching_dirs absdir] accumulates a list of existing directories,
    [dirs], that are the result of mapping abstract directory, [absdir],
    over all the mapping pairs in the BUILD_PATH_PREFIX_MAP environment
    variable. The list [dirs] will be in priority order (head as highest
    priority). The possible results are:
    - None, means BUILD_PATH_PREFIX_MAP is not set.
    - Some [], no source prefixes in the mapping were found,
    - Some dirs, means dirs are the directories found
    - Not_found raised, means some source prefixes in the map
      were found that matched [path], but none of mapping results
      were existing directories.
    See the BUILD_PATH_PREFIX_MAP spec at
    (https://reproducible-builds.org/specs/build-path-prefix-map/)
    *)

3. bytecomp/emitcode.ml

Added sanitizing of paths in the debug events.
If the compiler is given absolute source paths
this was leaking absolute build paths.

4. bytecomp/bytelink.ml

Rewrite when producing the path for a shebang.
The value written is from the user "-use-runtime"
option. The one setting BUILD_PATH_PREFIX_MAP
will have control whether this does anything or not.

5. debugger/command_line.ml

A debugger variable, "mapping", was added, which is tied to the
BUILD_PATH_PREFIX_MAP environment variable.  So "set mapping value"
will set BUILD_PATH_PREFIX_MAP, and "show mapping" will show the value
of BUILD_PATH_PREFIX_MAP. This allows the user to set it
programmaticaly, possibly from the ".ocamldebug" file.

6. debugger/source.ml

In source_of_module, use the Location.rewrite_exists.

6. debugger/symbols.{ml, mli}

a. Avoid adding directories redundantly

b. Expose bppm_expand_path and get_load_path so they
can be called from a fake printer which, when
loaded and installed in a test, can do unit testing
of ocamldebug.

c. Use Location.matching_dirs to expand directories.

7. Man page and manual updated.

Fixes ocaml#12083

The BUILD_PATH_PREFIX_MAP specification tells how to use that
environment variable to achieve reproducible build, i.e. builds of
products that do not leak absolute paths from the build
environment. See
https://reproducible-builds.org/specs/build-path-prefix-map.

However, that specification only describes half of the story.  Let us
call the building of reproducible products the "Build Phase". That is
the phase covered by the existing specification.

Let us defined the "Deployment phase" as the phase where you accept a
built reproducible product and make use of it, i.e. deploy it. An
example would be the debugger taking a reproducible binary and letting
the user debug it, showing the user the source code, etc.

We use the same mechanism in the deployment phase. Then
the BUILD_PATH_PREFIX_MAP must be setup with the logical
inverse of the mapping used during the build phase.

Also, this is part of a larger PR,
ocaml#12085, that is being split up
because it was too large. In addition to these changes, that PR
includes ocamltest enhancements plus the tests for
these changes. See it for prior discussion and the other changes.

I will now describe the essence of the changes.

1. utils/build_path_prefix_map.{ml,mli}

Added these functions to aid in use of the mapping
during the deployment phase. They allow to skip
mapping entries that are not applicable (find_rewrite).
find_rewrite is then used to implement rewrite_exists and
matching_dirs.

val find_rewrite : map -> (path -> bool) -> path -> path option
(** [rewrite_opt map pred path] tries to find a source in [map]
    that is a prefix of the input [path] that produces a result
    that satisfies predicate [pred]. If it succeeds,
    it replaces this prefix with the corresponding target.
    If it fails, it just returns [None]. *)

val rewrite_exists : map -> path -> path option
(** [rewrite_exists map path] tries to find a source in [map]
    that maps to a result that exists in the file system.
    If so, returns Some result. Otherwise, is there was
    at least one source that was a prefix of path, it is
    assumed that the map is deficient and Not_found is raised.
    If no source prefixes were found, None is returned.
*)

val matching_dirs : map -> path -> path list
(** [matching_dirs map absdir] accumulates a list of existing directories,
  [dirs], that are the result of mapping abstract directory, [absdir],
  over all the mapping pairs in [map]. The list [dirs] will be in
  priority order (head as highest priority). If there is success,
  returns [dirs]. If no source in the map was a prefix of [absdir],
  returns [[]]. If at least one source in the map was a prefix of
  [absdir], but none of the mappings were existing directories,
  raise Not_found.
*)

2. parsing/location.{ml, mli}

This has convenience functions for using the above
mapping API. It reads and caches the environment
variable so the end-user does not need to.

Location.absolute_path was modified so that BUILD_PATH_PREFIX_MAP
rewriting is done for both absolute and relative paths. Relative paths
are made absolute by appending to the cwd.  Previously only relative
paths were rewritten.  One discovery during testing was that if the
compiler is given an absolute path as the input source, the debug
event information had that absolute path (see below).

New functions are:

val rewrite_exists: string -> string option option
(** [rewrite_exists path] uses a BUILD_PATH_PREFIX_MAP mapping
    (https://reproducible-builds.org/specs/build-path-prefix-map/)
    and tries to find a source in mapping
    that maps to a result that exists in the file system.
    There are the following return values:
    - None, means BUILD_PATH_PREFIX_MAP is not set.
    - Some None, no source prefixes of [path] in the mapping were found,
    - Some (Some target), means target is the first file (in priority
      order) that [path] mapped to that exists in the file system.
    - Not_found raised, means some source prefixes in the map
      were found that matched [path], but none of them existed
      in the file system. *)

val matching_dirs: string -> string list option
(** [matching_dirs absdir] accumulates a list of existing directories,
    [dirs], that are the result of mapping abstract directory, [absdir],
    over all the mapping pairs in the BUILD_PATH_PREFIX_MAP environment
    variable. The list [dirs] will be in priority order (head as highest
    priority). The possible results are:
    - None, means BUILD_PATH_PREFIX_MAP is not set.
    - Some [], no source prefixes in the mapping were found,
    - Some dirs, means dirs are the directories found
    - Not_found raised, means some source prefixes in the map
      were found that matched [path], but none of mapping results
      were existing directories.
    See the BUILD_PATH_PREFIX_MAP spec at
    (https://reproducible-builds.org/specs/build-path-prefix-map/)
    *)

3. bytecomp/emitcode.ml

Added sanitizing of paths in the debug events.
If the compiler is given absolute source paths
this was leaking absolute build paths.

4. bytecomp/bytelink.ml

Rewrite when producing the path for a shebang.
The value written is from the user "-use-runtime"
option. The one setting BUILD_PATH_PREFIX_MAP
will have control whether this does anything or not.

5. debugger/command_line.ml

A debugger variable, "mapping", was added, which is tied to the
BUILD_PATH_PREFIX_MAP environment variable.  So "set mapping value"
will set BUILD_PATH_PREFIX_MAP, and "show mapping" will show the value
of BUILD_PATH_PREFIX_MAP. This allows the user to set it
programmaticaly, possibly from the ".ocamldebug" file.

6. debugger/source.ml

In source_of_module, use the Location.rewrite_exists.

6. debugger/symbols.{ml, mli}

a. Avoid adding directories redundantly

b. Expose bppm_expand_path and get_load_path so they
can be called from a fake printer which, when
loaded and installed in a test, can do unit testing
of ocamldebug.

c. Use Location.matching_dirs to expand directories.

7. Man page and manual updated.

Fixes ocaml#12083

The BUILD_PATH_PREFIX_MAP specification tells how to use that
environment variable to achieve reproducible build, i.e. builds of
products that do not leak absolute paths from the build
environment. See
https://reproducible-builds.org/specs/build-path-prefix-map.

However, that specification only describes half of the story.  Let us
call the building of reproducible products the "Build Phase". That is
the phase covered by the existing specification.

Let us defined the "Deployment phase" as the phase where you accept a
built reproducible product and make use of it, i.e. deploy it. An
example would be the debugger taking a reproducible binary and letting
the user debug it, showing the user the source code, etc.

We use the same mechanism in the deployment phase. Then
the BUILD_PATH_PREFIX_MAP must be setup with the logical
inverse of the mapping used during the build phase.

This PR generalized the functions in Build_path_prefix_map
and Location to facility the inverse map.

Also, this is part of a larger PR,
ocaml#12126, which itself
was part of
ocaml#12085, that is being split up
because it was too large. In addition to these changes, that PR
includes ocamltest enhancements plus the tests for
these changes. See it for prior discussion and the other changes.

I will now describe the essence of the changes.

1. utils/build_path_prefix_map.{ml,mli}

The API is now:

    val rewrite_first : map -> path -> path option
    (** [rewrite_first map path] tries to find a source in [map]
	that is a prefix of the input [path]. If it succeeds,
	it replaces this prefix with the corresponding target.
	If it fails, it just returns [None]. *)

    val rewrite_all : map -> path -> path list
    (** [rewrite_all map path] finds all sources in [map]
	that are a prefix of the input [path]. For each matching
	source, in priority order, it replaces this prefix with
	the corresponding target and adds the result to
	the returned list.
	If there are no matches, it just returns [[]]. *)

    val rewrite : map -> path -> path

2. parsing/location.{ml,mli}

Use the new API and renamed the rewriting functions.

    val rewrite_find_first_existing: string -> string option
    (** [rewrite_find_first_existing path] uses a BUILD_PATH_PREFIX_MAP mapping
	(https://reproducible-builds.org/specs/build-path-prefix-map/)
	and tries to find a source in mapping
	that maps to a result that exists in the file system.
	There are the following return values:
	- None, means BUILD_PATH_PREFIX_MAP is not set.
	- Some None, no source prefixes of [path] in the mapping were found,
	- Some (Some target), means target is the first file (in priority
	  order) that [path] mapped to that exists in the file system.
	- Not_found raised, means some source prefixes in the map
	  were found that matched [path], but none of them existed
	  in the file system. *)

    val rewrite_find_all_existing_dirs: string -> string list
    (** [rewrite_find_all_existing_dirs absdir] accumulates a list of existing
	directories, [dirs], that are the result of mapping abstract directory,
	[absdir], over all the mapping pairs in the BUILD_PATH_PREFIX_MAP
	environment variable, if any. The list [dirs] will be in priority order
	(head as highest priority). The possible results are:
	- [], means BUILD_PATH_PREFIX_MAP is not set, or if set, then
	  there were no matching prefixes of path.
	- Some dirs, means dirs are the directories found. A possibility
	  is that there was no mapping, but the input path is an existing
	  directory and the result if [[path]].
	- Not_found raised, means some source prefixes in the map
	  were found that matched [path], but none of mapping results
	  were existing directories (possibly due to misconfiguration)

	See the BUILD_PATH_PREFIX_MAP spec at
	(https://reproducible-builds.org/specs/build-path-prefix-map/)
	*)

In addition to those new APIs, Location.absolute_path was modified
to do rewriting on absolute paths (in addition to relative paths
which it was already doing).

At the present time their is no code depending on these APIs.
They are used in the other parts of the PRs mentioned.

The BUILD_PATH_PREFIX_MAP specification tells how to use that
environment variable to achieve reproducible build, i.e. builds of
products that do not leak absolute paths from the build
environment. See
https://reproducible-builds.org/specs/build-path-prefix-map.

However, that specification only describes half of the story.  Let us
call the building of reproducible products the "Build Phase". That is
the phase covered by the existing specification.

Let us defined the "Deployment phase" as the phase where you accept a
built reproducible product and make use of it, i.e. deploy it. An
example would be the debugger taking a reproducible binary and letting
the user debug it, showing the user the source code, etc.

We use the same mechanism in the deployment phase. Then
the BUILD_PATH_PREFIX_MAP must be setup with the logical
inverse of the mapping used during the build phase.

This PR generalized the functions in Build_path_prefix_map
and Location to facility the inverse map.

Also, this is part of a larger PR,
ocaml#12126, which itself
was part of
ocaml#12085, that is being split up
because it was too large. In addition to these changes, that PR
includes ocamltest enhancements plus the tests for
these changes. See it for prior discussion and the other changes.

I will now describe the essence of the changes.

1. utils/build_path_prefix_map.{ml,mli}

The API is now:

    val rewrite_first : map -> path -> path option
    (** [rewrite_first map path] tries to find a source in [map]
	that is a prefix of the input [path]. If it succeeds,
	it replaces this prefix with the corresponding target.
	If it fails, it just returns [None]. *)

    val rewrite_all : map -> path -> path list
    (** [rewrite_all map path] finds all sources in [map]
	that are a prefix of the input [path]. For each matching
	source, in priority order, it replaces this prefix with
	the corresponding target and adds the result to
	the returned list.
	If there are no matches, it just returns [[]]. *)

    val rewrite : map -> path -> path

2. parsing/location.{ml,mli}

Use the new API and renamed the rewriting functions.

    val rewrite_find_first_existing: string -> string option
    (** [rewrite_find_first_existing path] uses a BUILD_PATH_PREFIX_MAP mapping
	(https://reproducible-builds.org/specs/build-path-prefix-map/)
	and tries to find a source in mapping
	that maps to a result that exists in the file system.
	There are the following return values:
	- None, means BUILD_PATH_PREFIX_MAP is not set.
	- Some None, no source prefixes of [path] in the mapping were found,
	- Some (Some target), means target is the first file (in priority
	  order) that [path] mapped to that exists in the file system.
	- Not_found raised, means some source prefixes in the map
	  were found that matched [path], but none of them existed
	  in the file system. *)

    val rewrite_find_all_existing_dirs: string -> string list
    (** [rewrite_find_all_existing_dirs absdir] accumulates a list of existing
	directories, [dirs], that are the result of mapping abstract directory,
	[absdir], over all the mapping pairs in the BUILD_PATH_PREFIX_MAP
	environment variable, if any. The list [dirs] will be in priority order
	(head as highest priority). The possible results are:
	- [], means BUILD_PATH_PREFIX_MAP is not set, or if set, then
	  there were no matching prefixes of path.
	- Some dirs, means dirs are the directories found. A possibility
	  is that there was no mapping, but the input path is an existing
	  directory and the result if [[path]].
	- Not_found raised, means some source prefixes in the map
	  were found that matched [path], but none of mapping results
	  were existing directories (possibly due to misconfiguration)

	See the BUILD_PATH_PREFIX_MAP spec at
	(https://reproducible-builds.org/specs/build-path-prefix-map/)
	*)

In addition to those new APIs, Location.absolute_path was modified
to do rewriting on absolute paths (in addition to relative paths
which it was already doing).

At the present time their is no code depending on these new APIs,
so the only functional change is the change to Location.absolute_path

They new API functions are used in the other parts of the PRs mentioned.

The BUILD_PATH_PREFIX_MAP specification tells how to use that
environment variable to achieve reproducible build, i.e. builds of
products that do not leak absolute paths from the build
environment. See
https://reproducible-builds.org/specs/build-path-prefix-map.

However, that specification only describes half of the story.  Let us
call the building of reproducible products the "Build Phase". That is
the phase covered by the existing specification.

Let us defined the "Deployment phase" as the phase where you accept a
built reproducible product and make use of it, i.e. deploy it. An
example would be the debugger taking a reproducible binary and letting
the user debug it, showing the user the source code, etc.

We use the same mechanism in the deployment phase. Then
the BUILD_PATH_PREFIX_MAP must be setup with the logical
inverse of the mapping used during the build phase.

This PR generalized the functions in Build_path_prefix_map
and Location to facility the inverse map.

Also, this is part of a larger PR,
ocaml#12126, which itself
was part of
ocaml#12085, that is being split up
because it was too large. In addition to these changes, that PR
includes ocamltest enhancements plus the tests for
these changes. See it for prior discussion and the other changes.

I will now describe the essence of the changes.

1. utils/build_path_prefix_map.{ml,mli}

The API is now:

    val rewrite_first : map -> path -> path option
    (** [rewrite_first map path] tries to find a source in [map]
	that is a prefix of the input [path]. If it succeeds,
	it replaces this prefix with the corresponding target.
	If it fails, it just returns [None]. *)

    val rewrite_all : map -> path -> path list
    (** [rewrite_all map path] finds all sources in [map]
	that are a prefix of the input [path]. For each matching
	source, in priority order, it replaces this prefix with
	the corresponding target and adds the result to
	the returned list.
	If there are no matches, it just returns [[]]. *)

    val rewrite : map -> path -> path

2. parsing/location.{ml,mli}

Use the new API and renamed the rewriting functions.

    val rewrite_find_first_existing: string -> string option
    (** [rewrite_find_first_existing path] uses a BUILD_PATH_PREFIX_MAP mapping
	(https://reproducible-builds.org/specs/build-path-prefix-map/)
	and tries to find a source in mapping
	that maps to a result that exists in the file system.
	There are the following return values:
	- None, means BUILD_PATH_PREFIX_MAP is not set.
	- Some None, no source prefixes of [path] in the mapping were found,
	- Some (Some target), means target is the first file (in priority
	  order) that [path] mapped to that exists in the file system.
	- Not_found raised, means some source prefixes in the map
	  were found that matched [path], but none of them existed
	  in the file system. *)

    val rewrite_find_all_existing_dirs: string -> string list
    (** [rewrite_find_all_existing_dirs absdir] accumulates a list of existing
	directories, [dirs], that are the result of mapping abstract directory,
	[absdir], over all the mapping pairs in the BUILD_PATH_PREFIX_MAP
	environment variable, if any. The list [dirs] will be in priority order
	(head as highest priority). The possible results are:
	- [], means BUILD_PATH_PREFIX_MAP is not set, or if set, then
	  there were no matching prefixes of path.
	- Some dirs, means dirs are the directories found. A possibility
	  is that there was no mapping, but the input path is an existing
	  directory and the result if [[path]].
	- Not_found raised, means some source prefixes in the map
	  were found that matched [path], but none of mapping results
	  were existing directories (possibly due to misconfiguration)

	See the BUILD_PATH_PREFIX_MAP spec at
	(https://reproducible-builds.org/specs/build-path-prefix-map/)
	*)

In addition to those new APIs, Location.absolute_path was modified
to do rewriting on absolute paths (in addition to relative paths
which it was already doing).

At the present time their is no code depending on these new APIs,
so the only functional change is the change to Location.absolute_path

They new API functions are used in the other parts of the PRs mentioned.

The BUILD_PATH_PREFIX_MAP specification tells how to use that
environment variable to achieve reproducible build, i.e. builds of
products that do not leak absolute paths from the build
environment. See
https://reproducible-builds.org/specs/build-path-prefix-map.

However, that specification only describes half of the story.  Let us
call the building of reproducible products the "Build Phase". That is
the phase covered by the existing specification.

Let us defined the "Deployment phase" as the phase where you accept a
built reproducible product and make use of it, i.e. deploy it. An
example would be the debugger taking a reproducible binary and letting
the user debug it, showing the user the source code, etc.

We use the same mechanism in the deployment phase. Then
the BUILD_PATH_PREFIX_MAP must be setup with the logical
inverse of the mapping used during the build phase.

This PR generalized the functions in Build_path_prefix_map
and Location to facility the inverse map.

Also, this is part of a larger PR,
ocaml#12126, which itself
was part of
ocaml#12085, that is being split up
because it was too large. In addition to these changes, that PR
includes ocamltest enhancements plus the tests for
these changes. See it for prior discussion and the other changes.

For the new functions, see the '.mli' file for details.

1. utils/build_path_prefix_map.{ml,mli}

The new api functions are:

    val rewrite_first : map -> path -> path option
    val rewrite_all : map -> path -> path list

2. parsing/location.{ml,mli}

The new api functions are:

    val rewrite_find_first_existing: string -> string option
    val rewrite_find_all_existing_dirs: string -> string list

In addition to those new APIs, Location.absolute_path was modified
to do rewriting on absolute paths (in addition to relative paths
which it was already doing).

At the present time their is no code depending on these new APIs,
so the only functional change is the change to Location.absolute_path

The new API functions are used in the other parts of the PRs mentioned.

This is part 2 of a larger PR, ocaml#12085, which includes a part 1
(compiler and debugger changes), this part, and a third part which is
tests of part 1 that make use of (and hence motivate and test) these
changes. Please see that PR for example uses of these changes.

1. Add the ability to have scripts or files with ocamltest variables
references expanded. Added a new "expand" action to ocamltest. It is
mostly like the "copy" action, but the source file is read a line at a
time and the lines are expanded. It does not support source
directories, but the destination can be a directory.

2. Enhanced ocamltest to have a facility for making builtin
functions. These are like ocamltest variables but have a function
attached to them, and when the variable is expanded, the arguments are
first expanded and then the function is called and its result is
returned. Currently they only take one parameter, but it should not be
too hard to add the ability to have multiply arguments.

3. Added a new "dumpenv_expanded" action which not
only shows the value of each variable, but also what
they expand to. When variables are originally
assigned, their RHS is not expanded, but only
later when the variable itself is looked up.

4. Two builtin functions were defined:

- bppm_decode does BUILD_PATH_PREFIX_MAP decoding

- bppm_encode does BUILD_PATH_PREFIX_MAP encoding

See https://reproducible-builds.org/specs/build-path-prefix-map/

5. Prepare to be able to do Dune tests:

5a. Added a action helper for testing if a program
is available in PATH.

5b. Use the helper to make an action, has_dune,
that detects whether dune is available.

The BUILD_PATH_PREFIX_MAP specification tells how to use that
environment variable to achieve reproducible build, i.e. builds of
products that do not leak absolute paths from the build
environment. See
https://reproducible-builds.org/specs/build-path-prefix-map.

However, that specification only describes half of the story.  Let us
call the building of reproducible products the "Build Phase". That is
the phase covered by the existing specification.

Let us defined the "Deployment phase" as the phase where you accept a
built reproducible product and make use of it, i.e. deploy it. An
example would be the debugger taking a reproducible binary and letting
the user debug it, showing the user the source code, etc.

We use the same mechanism in the deployment phase, but use a different
variable, DEPLOY_PATH_PREFIX_MAP.  It must be setup with the logical
inverse of the mapping used during the build phase.

Also, this is part of a larger PR,
ocaml#12085, that is being split up
because it was too large. In addition to these changes, that PR
includes ocamltest enhancements plus the tests for these changes. See
it for prior discussion and the other changes.

I will now describe the essence of the changes.

1. bytecomp/emitcode.ml

Added sanitizing of paths in the debug events.
If the compiler is given absolute source paths
this was leaking absolute build paths.

2. debugger/command_line.ml

A debugger variable, "mapping", was added, which is tied to the
DEPLOY_PATH_PREFIX_MAP environment variable.  So "set mapping value"
will set DEPLOY_PATH_PREFIX_MAP, and "show mapping" will show the value
of DEPLOY_PATH_PREFIX_MAP. This allows the user to set it
programmaticaly, possibly from the ".ocamldebug" file.

3. debugger/source.ml

In source_of_module, use the Location.rewrite_exists.

4. debugger/symbols.{ml, mli}

a. Avoid adding directories redundantly

b. Use Location.matching_dirs to expand directories.

5. Man page and manual updated.

Fixes ocaml#12083

The BUILD_PATH_PREFIX_MAP specification tells how to use that
environment variable to achieve reproducible build, i.e. builds of
products that do not leak absolute paths from the build
environment. See
https://reproducible-builds.org/specs/build-path-prefix-map.

However, that specification only describes half of the story.  Let us
call the building of reproducible products the "Build Phase". That is
the phase covered by the existing specification.

Let us defined the "Deployment phase" as the phase where you accept a
built reproducible product and make use of it, i.e. deploy it. An
example would be the debugger taking a reproducible binary and letting
the user debug it, showing the user the source code, etc.

We use the same mechanism in the deployment phase, but use a different
variable, DEPLOY_PATH_PREFIX_MAP.  It must be setup with the logical
inverse of the mapping used during the build phase.

Also, this is part of a larger PR,
ocaml#12085, that is being split up
because it was too large. In addition to these changes, that PR
includes ocamltest enhancements plus the tests for these changes. See
it for prior discussion and the other changes.

I will now describe the essence of the changes.

1. bytecomp/emitcode.ml

Added sanitizing of paths in the debug events.
If the compiler is given absolute source paths
this was leaking absolute build paths.

2. debugger/command_line.ml

A debugger variable, "mapping", was added, which is tied to the
DEPLOY_PATH_PREFIX_MAP environment variable.  So "set mapping value"
will set DEPLOY_PATH_PREFIX_MAP, and "show mapping" will show the value
of DEPLOY_PATH_PREFIX_MAP. This allows the user to set it
programmaticaly, possibly from the ".ocamldebug" file.

3. debugger/source.ml

In source_of_module, use the Location.rewrite_exists.

4. debugger/symbols.{ml, mli}

a. Avoid adding directories redundantly

b. Use Location.matching_dirs to expand directories.

5. Man page and manual updated.

Updated new tests to take into account:

1. The new ocamltest syntax that is C-like rather than
org-mode like.

2. The debugger now uses DEPLOY_PATH_PREFIX_MAP for its
inverse mapping, rather than BUILD_PATH_PREFIX_MAP.

Fixes ocaml#12083

This is part 2 of a larger PR, ocaml#12085, which includes a part 1
(compiler and debugger changes), this part, and a third part which is
tests of part 1 that make use of (and hence motivate and test) these
changes. Please see that PR for example uses of these changes.

1. Add the ability to have scripts or files with ocamltest variables
references expanded. Added a new "expand" action to ocamltest. It is
mostly like the "copy" action, but the source file is read a line at a
time and the lines are expanded. It does not support source
directories, but the destination can be a directory.

2. Enhanced ocamltest to have a facility for making builtin
functions. These are like ocamltest variables but have a function
attached to them, and when the variable is expanded, the arguments are
first expanded and then the function is called and its result is
returned. Currently they only take one parameter, but it should not be
too hard to add the ability to have multiply arguments.

3. Added a new "dumpenv_expanded" action which not
only shows the value of each variable, but also what
they expand to. When variables are originally
assigned, their RHS is not expanded, but only
later when the variable itself is looked up.

4. Two builtin functions were defined:

- bppm_decode does BUILD_PATH_PREFIX_MAP decoding

- bppm_encode does BUILD_PATH_PREFIX_MAP encoding

See https://reproducible-builds.org/specs/build-path-prefix-map/

5. Prepare to be able to do Dune tests:

5a. Added a action helper for testing if a program
is available in PATH.

5b. Use the helper to make an action, has_dune,
that detects whether dune is available.