Steps to reproduce

Kudos to @albestro for reporting the issue and helping debug it

Summary
Config scopes cannot be trusted when -C and -e are used together on the command line. The issue has been introduced in #50853 and is really nasty for two reasons:

1. When -C and -e are used together, Spack silently shuffles the config scopes in the middle of execution. The environment scope might be swapped with some other custom scope. Due to the mechanics involved this is also difficult to debug, since the output of spack config get/blame cannot be trusted.
2. The documentation was never updated in config: CLI scopes should override environments #50853. The priority for the CUSTOM config scope in the code is higher than the priority of ENVIRONMENT. The docs say otherwise.

Why are the scopes shuffled?

Since #50853 was merged, environments added from the command line are considered at CUSTOM priority:

When the environment is loaded, we use a context manager:

that adds the environment at ENVIRONMENT priority:

The result is that config scopes are swapped in the middle of execution, and user might see unexpected behaviors.

Minimal reproducer

This can be reproduced easily with the following setup:

$ tree
.
├── custom
│   └── config.yaml
├── env1
│   └── spack.yaml
└── script.py

2 directories, 3 files

The files are the following:

# script.py
import spack.config
import spack.environment

KEY = "config:install_tree:root"

print(f"[Before context manager] {spack.config.CONFIG.get(KEY)}")
with spack.environment.active_environment().manifest.use_config():
    print(f"[Within context manager] {spack.config.CONFIG.get(KEY)}")
print(f"[After context manager] {spack.config.CONFIG.get(KEY)}")

# spack.yaml
spack:
  specs: []
  config:
    install_tree:
      root: /tmp/env

# config.yaml
config:
  install_tree:
    root: /tmp/custom

Then we can run the script like:

$ spack -C custom -e env1 python script.py 
[Before context manager] /tmp/env
[Within context manager] /tmp/custom
[After context manager] /tmp/custom

and see the swapping in action 😞

Error message

There is no clear error message, just weird observed behavior that doesn't match:

1. the documentation
2. the output of spack config blame

The report above has been obtained with trial and error + a debugger. The original issue was reported because some modifications in a custom repository added at environment scope were not taken into account correctly.

Information on your system

• Spack: 1.1.0.dev0 (e250ec0)
• Builtin repo: spack/spack-packages@8f612a2
• Python: 3.13.2
• Platform: linux-ubuntu20.04-icelake

@tgamblin @kwryankrattiger

General information

• I have run spack debug report and reported the version of Spack/Python/Platform
• I have searched the issues of this repo and believe this is not a duplicate
• I have run the failing commands in debug mode and reported the output