Tested this change on a few concretizations that don't involve package preferences, and there is no impact. I used the same configuration for both and develop at d0d5526:

radiuss_develop.csv
radiuss_packages.csv
radiuss.txt

Here is a plot of the results:

I have a couple thoughts on this due to running into one of the things this involves.

On the modifications

Modifications:

* [x]  `version` can only be set under each specific package (and not under `all:`) 

* [x]  Modified documentation to reflect the change in `packages.yaml`

* [x]  Added warnings to guide users if they have configurations which are not valid anymore

These changes are very sensible. The first one is kind of a surprise that it was this way at all. And the last one is the best approach when what is allowed in a config file changes.

* [x]  `target`, `compiler` and `providers` soft-preferences can only be set under `all:` in `packages.yaml`

I don't know about providers, but have some thoughts on target and compiler. My run in has been with target.

It is definitely true that forcing compiler and target to be global vastly simplifies things.

And with compiler, it certainly can help avoid some sticky issues for C++ based packages where unlike languages like C, built objects are not that compatible with each other due to different name mangling schemes, how exceptions are implemented, etc. with the exception of compilers that put serious work into being compatible with each other. Very easy to have a configuration that cannot be built. C++ isn't the only language in this situation, but it is the biggest offender. Fortran is as well, but to a much lesser degree, especially for legacy Fortran.

But, on compiler, one major issue is that if one wants to use compiler X for most things, but compiler X is incompatible with package A that is a dependency (say someone wants to use the Intel compiler for most things, but there are a few things that require GCC); then the user is completely out of luck unless they explicitly add that dependency to their package specs and configure concretization to reuse (which a user may not want for other reasons, but now their choice would be forced). For one package, this isn't too bad, but it doesn't scale that well compared to having this in the packages.yaml for the packages that apply, which has the added bonus of it ideally automatically applying when the package is installed.

On target, there are less sticky issues I think than on compiler. If the CPU can run all the targets, it probably doesn't matter much if the targets differ for different packages even if they are dependencies of each other so long as a function in one package isn't trying to say pass AVX512 registers to a function in another package that was compiled for an architecture that doesn't have them. But aren't those sorts of things usually passed on the stack or as pointers instead on x86? Though, even if so, that might not be true for non-x86 architectures and more sticky issues come up (spack does support many architectures after all, so choices need to keep them all in mind). I have a use case for having mismatched target, which is my horse in this race. But as it stands, the weights aren't sufficient to make my use case work at present anyways and I might be the only user who wants this (spack is used by many people and thus has to keep more in mind than just a single user, after all).

* [x]  ASP code has a single set of weights for providers, targets and compilers (as opposed to different weights per package)

I actually have a somewhat naive question on this. If the user specifically asks for a specific config, whether in their specs or in their packages.yaml, why consider any other option other than what was specifically requested and just warn the user if it looks like this will cause something that might not work (essentially trusting the user even if they might want something that will fail to build and/or run)? Is it that doing this means concretization outright fails all the time or something? Essentially, what problem/s is this scheme trying to solve?

It is definitely true that forcing compiler and target to be global vastly simplifies things.

Just to be sure there are no misconceptions: I am forcing target and compiler preferences to be global. The DAG can be mixed as it was before. For instance:

$ spack spec hdf5 %clang target=icelake ^zlib %gcc target=x86_64

is still perfectly valid. Users can mix and match as they want, either using spec literals or the requirements in the configuration file.

If the user specifically asks for a specific config, whether in their specs [ ... ]

If the users asks for a specific configuration, i.e. they use spec literals or requirements, Spack either provides that or errors out. We have also a less strict "preferences" mechanism, and that's what I am touching here.

It is definitely true that forcing compiler and target to be global vastly simplifies things.

Just to be sure there are no misconceptions: I am forcing target and compiler preferences to be global. The DAG can be mixed as it was before. For instance:

$ spack spec hdf5 %clang target=icelake ^zlib %gcc target=x86_64

is still perfectly valid. Users can mix and match as they want, either using spec literals or the requirements in the configuration file.

If the user specifically asks for a specific config, whether in their specs [ ... ]

If the users asks for a specific configuration, i.e. they use spec literals or requirements, Spack either provides that or errors out. We have also a less strict "preferences" mechanism, and that's what I am touching here.

I don't know how I missed or misunderstood that feature. So, something like require: "target=X" would work and that is stronger than putting it the way that this PR would remove? If that is the case, then this PR basically gets rid of the weak path that doesn't always work while still keeping the much stronger path. In this case, then, I don't see any particular problems for this PR.

Benchmarked on:

• Spack: 0.21.0.dev0 (fc49a02)
• Python: 3.11.5
• Platform: linux-ubuntu20.04-icelake
• Concretizer: clingo

radiuss-pr.csv
radiuss-develop.csv
radiuss.txt

There is no impact in performance.

This PR has no impact on the solver performance. Tested on:

• Spack: 0.21.0.dev0 (5ca20d2)
• Python: 3.11.5
• Platform: linux-ubuntu20.04-icelake
• Concretizer: clingo

develop: a2f0088
PR: 5ca20d2

radiuss-develop.csv
radiuss-pr.csv
radiuss.txt

If we use the following packages.yaml:

packages:
  all:
    version: ["1.0", "2.0"]
  hdf5:
    target: x86_64
    compiler: [gcc]
  zlib-ng:
    target: x86_64
    compiler: [clang]

we get these warnings:

I'd get rid of the Currently, in both error messages but otherwise I think this looks great.

@alecbcs Done, see 1c9ccea