-
Notifications
You must be signed in to change notification settings - Fork 1.2k
Towards an OCAMLPATH #8898
Description
Most programming languages have a notion of include path to allow to specify relative file lookups in an ordered sequence of absolute install directories. For example there is LIBRARY_PATH and C_INCLUDE_PATH for C, CLASSPATH for Java, the include_path directive of php.ini files, etc.
OCaml has this but in a very limited form via the -I +DIR notation which allows to specify the DIR directory relative to one absolute directory (ocamlc -where) which itself can be set via the CAMLLIB or OCAMLLIB environment variable.
I would like to propose to extend the semantics of this notation in a backward compatible way to support lookup in a proper OCAMLPATH variable. That is:
- If
OCAMLPATHis undefined nothing changes. - If
OCAMLPATH=D1:D2..., the notation-I +DIRindicates for the compiler to lookup for objects inD1/DIRthen if not found inD2/DIR, etc.
This would be a first step towards eventually providing better library lookup in the compiler and for the eco-system it formalizes a general file lookup procedure that build systems and infrastructure tools can follow to lookup compilation objects and attribute them short root names under the form of these +DIR relative file path (which you can see as "package" names, see below).
The odig and omod tools have been exploring the idea to move towards suppressing the ocamlfind indirection from the eco-system, something it seems upstream is not entirely cold about it.
However at the moment dealing with opam system compilers and/or mixed opam and system OCaml package installs is messy and done entirely in ad-hoc fashion by these tools -- see the first two points here. I think it would be beneficial for the eco-system if upstream helps in formalizing this. Doing so should have no impact on the ocamlfind system itself which relies on META files to do its own lookups.
The ideas behind this move are the following:
- We want to be able to use the file system to determine root (or "package") names and the files that are associated to them. A simple convention for this is given an installation directory
$LIBDIRto attribute the files in$LIBDIR/PKGto the package namePKG(and those of$LIBDIR/PKG/SUBto namesPKG/SUBorPKG.SUB, etc.). This directly translates into the existing-I +PKGand-I +PKG/SUB. - We want to support multiple package install sources, that is lookup in more than one
$LIBDIR, hence have a notion of left-leaning packagePATHfor lookup: this isOCAMLPATH. - We want this to have as little impact or need for change as possible on the existing world. This proposal disturbs nothing. The
ocamlfindworld continues to exist untouched. Users of the-I +DIRnotation are not affected as long as they don't defineOCAMLPATHor define it but start it with$(ocamlc -where). Finally it almost entirely works in the world as it exists:opaminstalls,ocamlc -whereinstalls. Except for one package:ocamlitself (see below).
The idea is that in opam switches OCAMLPATH will be set to $(ocamlc -where):$(opam var lib). And if you have other packages installed somewhere else by another entity you can simply add them at the front or the end of the OCAMLPATH as you see it fit.
Note that all of this does absolutely not touch the problem of looking up objects/libraries and their dependencies and constitutes in no way an ocamlfind replacement, it just lays down the ground to define root names and subnames based on directories existing on the file system.
This is mostly similar to @lpw25 OCAMLNAMESPACES environment variable used by his namespace proposal. One difference with OCAMLPATH is that you can't bind toplevel root (or "package") names to directories directly. The root names are defined by the toplevel directory names of the directories in OCAMLPATH.
I argue the proposed scheme is not less powerful than the one @lpw25 proposes. In practice you can always rename by creating a directory with appropriate symlinks that you put in front of OCAMLPATH. It also more trivial to understand the naming structure for the end users and other consumers of the variable, an ls of OCAMLPATH directories will do and a stat in each of these dirs can be used to test for the existence of a given name.
One issue that needs to be touched with this proposal, is that ocamlc -where is what should be added in practice to OCAMLPATH that's e.g. where debian installs its packages. However we also want to have ocaml as the root name for the stdlib and compiler-libs "subpackages". Unfortunately ocaml installs the compilation objects of these directly in ocamlc -where. Fundamentally we would like those to be installed in $(ocamlc -where)/ocaml. So that setting OCAMLPATH to $(ocamlc -where) the ocaml upstream libraries are seen in the ocaml "package". So maybe a configure option should be added to allow this install structure, which should eventually be enabled by both system packagers and opam.
Note except for the "where is the ocaml package" problem. The new -I +dir semantics doesn't even have to be implemented to be useful for the eco-system, the tools I mention perform the lookups by themselves. I just think it's good if we can agree with that convention and that tools that need to lookup ocaml installs can do so in a similar and principled manner using the same file-system derived names and an environment variable (OCAMLPATH) blessed by upstream.