Add new API classes with the purpose to represent configuration data that may contain paths. All paths, even when embedded in options or text, are represented as os.Path type and can be therefore handled in a type-safe manner. In case, paths need to be to relocated, no guessing or vague String.replace logic is needed. For example, PR #6031 contains a path mapping mechanism, with which all Opts would be automatically covered, too. That means if a Opts or it's companion types are used for javacOptions, scalacOptions and other configuration tasks, embedded paths are no longer an issue for relocatable Mill caches or remote caching in general.

In contrast to the initial proposal of this idea (#6057) the main classes are named Opt/Opts, not Arg/Args, since we already have a mill.api.Args class used for other purposes.

New API classes

Opt is a holder of configuration data. Opt can be constructed from one or multiple Strings or os.Paths. When rendered as string, all parts will be concatenated.

Opts is a Seq-like container to be used instead of Seq[String].

OptMap is a type alias to Map[String, Opt] to provide some convenience to be used instead of Map[String, String].

Both container types, Opts and OptMap can be easily converted to a String-based container with .toStringSeq or .toStringMap.

OptGroup is a group of Opts that are tightly coupled, and should never be split. This is useful to model semantically bound CLI args with multi-arity like --file <file> or -classpath jar2:jar2:...:jarN. It can help to communicate intent and eases post-processing of multi-part opts.

JSON serialization

The JSON structure is more complex than the previous Seq[String] equivalent, because it needs to support options, option groups and multi-parts options, which may itself consist of many strings and paths.

A rather complex Opts JSON serialization may look like this:

Opts(
  "-deprecated", 
  OptGroup("-classpath", Opt.mkPath(classpath, sep = ":"))
)

[
  "-deprecated",
  [
    "-classpath",
    [
      { "path": "/path/to/jar1.jar" },
      ":",
      { "path": "/path/to/jar2.jar" },
      ":",
      { "path": "/path/to/jar3.jar" }
    ]
  ]
]

Since the JSON serialization for paths re-use the respective upickle-serializers, the root path mapping of PR #6031 may also work here (Resulting is paths like $MILL_OUT/task/to/jar.dest/out.jar).

YAML convenience

In addition, the json reader supports reading of flat json arrays, which means, that all Seq[String] can easily be read as Opts. This was done to make working with Opts as convenient as Seq[String] not only in Scala code but also in YAML configuration code.

This means, build.mill YAML frontmatter and build.mill.yaml don't need any changes.

mill-build:
  scalacOptions: ["-verbose"]
  forkEnv: { "MY_CUSOTM_ENV": "my-env-value" }

Although the full encoding is also supported but seldomly required.

mill-build:
  forkArgs: [[["-javaagent:", {"path": "$WORKSPACE/agent.jar"}]]]
  forkEnv: { "MY_WORK_DIR": [{"path": "$HOME/workDir"}] }

Realistic example

In POC PR #6800, you can see a refactoring of Mill to use the new mill.api.opt API in all relevant tasks like javacOptions, scalacOptions, forkArgs, forkEnv and some more.

Caveats / Feedback wanted

In this current POC, we need to add import mill.api.opt.*. I'm looking for ideas, how to improve this and make it a single import.

Related issues and discussions

• Make out/ folder contents (more) reproducible and filesystem layout agnostic #3660 - Tracker issue
• Dynamically map known root paths in cache files #6031 - Stores relocatable paths
• new `Args` class for relocateable configurations #6057 - Idea that drives this PR about how to store relocatable configuration options

Pull request: #6129