In #25851 it was proposed to add a command-line option --persist-stage=package1,package2,... to improve the performance of building a package if another version of the package had been built in a previous spack run. I think that specific option would make sense to provide in either the command-line (if you only want to use it for a specific spack invocation, but otherwise do clean builds) or in a yaml config (if you want to speed up the rebuilds of a package for all spack invocations of a user or site).

Separately, it's always been frustrating to me that I need to modify a config file to add config:debug:true when I want to make spack spit out a stacktrace, or to use the other concretizer for just a single run.

This dilemma does not exist in the pants build tool, which allows setting all options (in order of increasing precedence):

• in a config file
• in the process environment
• on the command line

See https://v1.pantsbuild.org/options.html#setting-option-values for more details.

Proposal

Since spack has already decided (very reasonably) to separate the concepts of command-line and config options, we can't and likely shouldn't attempt to make every option perfectly fungible in a config vs on the command line. However, I had an idea about how to achieve the same result without needing to mess around too much with our existing methods of declaring options:

1. Make a global command-line option to slurp in a YAML config file for the duration of a single spack command.
   • The config options specified in this way would take precedence over the static filesystem config, and there would be a command-line option to enable this for each config scope.
   • The option could be repeated to add more config values to the global config dict before executing the spack command.
2. Make a cli config section which specifies defaults for command-line options either globally or for specific subcommands.
   • Before we pass the command-line options to the ArgumentParser and execute the spack command, we can first check the content of the cli config section and apply any argument values to it directly. This means explicitly-specified command-line options would still take precedence over anything in the cli config section.
   • Currently, the default value of command-line arguments is either hardcoded in the ArgumentParser, or implemented via checking by hand whether the option is set, and setting it to some default value if not. Adding values to the spack repo's default cli.yaml would be a declarative way to set these defaults that makes changes easier to audit than scanning the code implementing a spack subcommand.
3. (tentative) Check the process environment to see whether any variables SPACK_<CONFIG_SCOPE>_<CONFIG_PATH_COMPONENT>_<CONFIG_OPTION_NAME> are set, and apply the value of that environment variable to the appropriate config.
   • This seems less useful, since spack config and command-line options cover the main use cases of "I want this to be set by default for all my spack commands" and "I want this to be set for just this spack command".
   • I personally wouldn't want to add this, but I've provided it here for the sake of completeness.