Steps to reproduce
I have found an issue with the concretizer not picking up the correct providers when described in packages.yaml. If providers are aded to the command line, e.g. ^intel-mpi ^intel-mkl, all works as expected. Here is one example of intel-mpi not being picked up.
Tagging @tgamblin as discussed yesterday.
The default MPI provider after pulling the develop branch is Open MPI:
[ec2-user@ip-172-31-1-92 spack]$ spack solve gromacs
==> Best of 9 considered solutions.
==> Optimization Criteria:
Priority Criterion Value
1 deprecated versions used 0
2 version weight 0
3 number of non-default variants (roots) 0
4 preferred providers for roots 0
5 number of values in multi valued variants (root) 0
6 number of non-default variants (non-roots) 0
7 preferred providers (non-roots) 0
8 compiler mismatches 0
9 version badness 2
10 number of values in multi valued variants (non-root) -8
11 non-preferred compilers 0
12 target mismatches 0
13 non-preferred targets 0
gromacs@2021.3%gcc@7.3.1~blas~cuda~cycle_subcounters~double+hwloc~ipo~lapack~mdrun_only+mpi~nosuffix~opencl+openmp~plumed~relaxed_double_precision+shared~sycl build_type=RelWithDebInfo arch=linux-amzn2-skylake_avx512
^cmake@3.21.4%gcc@7.3.1~doc+ncurses+openssl+ownlibs~qt build_type=Release arch=linux-amzn2-skylake_avx512
^ncurses@6.2%gcc@7.3.1~symlinks+termlib abi=none arch=linux-amzn2-skylake_avx512
^pkgconf@1.8.0%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
^openssl@1.1.1l%gcc@7.3.1~docs certs=system arch=linux-amzn2-skylake_avx512
^perl@5.34.0%gcc@7.3.1+cpanm+shared+threads arch=linux-amzn2-skylake_avx512
^berkeley-db@18.1.40%gcc@7.3.1+cxx~docs+stl patches=b231fcc4d5cff05e5c3a4814f6a5af0e9a966428dc2176540d2c05aff41de522 arch=linux-amzn2-skylake_avx512
^bzip2@1.0.8%gcc@7.3.1~debug~pic+shared arch=linux-amzn2-skylake_avx512
^diffutils@3.8%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
^libiconv@1.16%gcc@7.3.1 libs=shared,static arch=linux-amzn2-skylake_avx512
^gdbm@1.19%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
^readline@8.1%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
^zlib@1.2.11%gcc@7.3.1+optimize+pic+shared arch=linux-amzn2-skylake_avx512
^fftw@3.3.10%gcc@7.3.1+mpi~openmp~pfft_patches precision=double,float arch=linux-amzn2-skylake_avx512
^openmpi@4.1.1%gcc@7.3.1~atomics~cuda~cxx~cxx_exceptions+gpfs~internal-hwloc~java~legacylaunchers~lustre~memchecker~pmi~pmix~singularity~sqlite3+static~thread_multiple+vt+wrapper-rpath fabrics=none schedulers=none arch=linux-amzn2-skylake_avx512
^hwloc@2.5.0%gcc@7.3.1~cairo~cuda~gl~libudev+libxml2~netloc~nvml~opencl+pci~rocm+shared arch=linux-amzn2-skylake_avx512
^libpciaccess@0.16%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
^libtool@2.4.6%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
^m4@1.4.19%gcc@7.3.1+sigsegv patches=9dc5fbd0d5cb1037ab1e6d0ecc74a30df218d0a94bdd5a02759a97f62daca573,bfdffa7c2eb01021d5849b36972c069693654ad826c1a20b53534009a4ec7a89 arch=linux-amzn2-skylake_avx512
^libsigsegv@2.13%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
^util-macros@1.19.3%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
^libxml2@2.9.12%gcc@7.3.1~python arch=linux-amzn2-skylake_avx512
^xz@5.2.5%gcc@7.3.1~pic libs=shared,static arch=linux-amzn2-skylake_avx512
^libevent@2.1.12%gcc@7.3.1+openssl arch=linux-amzn2-skylake_avx512
^numactl@2.0.14%gcc@7.3.1 patches=4e1d78cbbb85de625bad28705e748856033eaafab92a66dffd383a3d7e00cc94,62fc8a8bf7665a60e8f4c93ebbd535647cebf74198f7afafec4c085a8825c006,ff37630df599cfabf0740518b91ec8daaf18e8f288b19adaae5364dc1f6b2296 arch=linux-amzn2-skylake_avx512
^autoconf@2.69%gcc@7.3.1 patches=35c449281546376449766f92d49fc121ca50e330e60fefcfc9be2af3253082c2,7793209b33013dc0f81208718c68440c5aae80e7a1c4b8d336e382525af791a7,a49dd5bac3b62daa0ff688ab4d508d71dbd2f4f8d7e2a02321926346161bf3ee arch=linux-amzn2-skylake_avx512
^automake@1.16.3%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
^openssh@8.7p1%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
^libedit@3.1-20210216%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
The default provider can be overwritten by adding a dependency to a different MPI on the command line:
[ec2-user@ip-172-31-1-92 spack]$ spack solve gromacs ^intel-mpi
==> Bootstrapping clingo from pre-built binaries
==> Best of 9 considered solutions.
==> Optimization Criteria:
Priority Criterion Value
1 deprecated versions used 0
2 version weight 0
3 number of non-default variants (roots) 0
4 preferred providers for roots 0
5 number of values in multi valued variants (root) 0
6 number of non-default variants (non-roots) 0
7 preferred providers (non-roots) 100
8 compiler mismatches 0
9 version badness 2
10 number of values in multi valued variants (non-root) -6
11 non-preferred compilers 0
12 target mismatches 0
13 non-preferred targets 0
gromacs@2021.3%gcc@7.3.1~blas~cuda~cycle_subcounters~double+hwloc~ipo~lapack~mdrun_only+mpi~nosuffix~opencl+openmp~plumed~relaxed_double_precision+shared~sycl build_type=RelWithDebInfo arch=linux-amzn2-skylake_avx512
^cmake@3.21.4%gcc@7.3.1~doc+ncurses+openssl+ownlibs~qt build_type=Release arch=linux-amzn2-skylake_avx512
^ncurses@6.2%gcc@7.3.1~symlinks+termlib abi=none arch=linux-amzn2-skylake_avx512
^pkgconf@1.8.0%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
^openssl@1.1.1l%gcc@7.3.1~docs certs=system arch=linux-amzn2-skylake_avx512
^perl@5.34.0%gcc@7.3.1+cpanm+shared+threads arch=linux-amzn2-skylake_avx512
^berkeley-db@18.1.40%gcc@7.3.1+cxx~docs+stl patches=b231fcc4d5cff05e5c3a4814f6a5af0e9a966428dc2176540d2c05aff41de522 arch=linux-amzn2-skylake_avx512
^bzip2@1.0.8%gcc@7.3.1~debug~pic+shared arch=linux-amzn2-skylake_avx512
^diffutils@3.8%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
^libiconv@1.16%gcc@7.3.1 libs=shared,static arch=linux-amzn2-skylake_avx512
^gdbm@1.19%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
^readline@8.1%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
^zlib@1.2.11%gcc@7.3.1+optimize+pic+shared arch=linux-amzn2-skylake_avx512
^fftw@3.3.10%gcc@7.3.1+mpi~openmp~pfft_patches precision=double,float arch=linux-amzn2-skylake_avx512
^intel-mpi@2019.10.317%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
^hwloc@2.5.0%gcc@7.3.1~cairo~cuda~gl~libudev+libxml2~netloc~nvml~opencl+pci~rocm+shared arch=linux-amzn2-skylake_avx512
^libpciaccess@0.16%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
^libtool@2.4.6%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
^m4@1.4.19%gcc@7.3.1+sigsegv patches=9dc5fbd0d5cb1037ab1e6d0ecc74a30df218d0a94bdd5a02759a97f62daca573,bfdffa7c2eb01021d5849b36972c069693654ad826c1a20b53534009a4ec7a89 arch=linux-amzn2-skylake_avx512
^libsigsegv@2.13%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
^util-macros@1.19.3%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
^libxml2@2.9.12%gcc@7.3.1~python arch=linux-amzn2-skylake_avx512
^xz@5.2.5%gcc@7.3.1~pic libs=shared,static arch=linux-amzn2-skylake_avx512However, if I try ot overwrite the provider in packages.yaml it is not picked up:
[ec2-user@ip-172-31-1-92 spack]$ echo "packages:
> gromacs:
> providers:
> mpi: [intel-mpi]
> fftw:
> providers:
> mpi: [intel-mpi]
> " > etc/spack/packages.yaml
[ec2-user@ip-172-31-1-92 spack]$ spack solve gromacs
==> Best of 9 considered solutions.
==> Optimization Criteria:
Priority Criterion Value
1 deprecated versions used 0
2 version weight 0
3 number of non-default variants (roots) 0
4 preferred providers for roots 0
5 number of values in multi valued variants (root) 0
6 number of non-default variants (non-roots) 0
7 preferred providers (non-roots) 0
8 compiler mismatches 0
9 version badness 2
10 number of values in multi valued variants (non-root) -8
11 non-preferred compilers 0
12 target mismatches 0
13 non-preferred targets 0
gromacs@2021.3%gcc@7.3.1~blas~cuda~cycle_subcounters~double+hwloc~ipo~lapack~mdrun_only+mpi~nosuffix~opencl+openmp~plumed~relaxed_double_precision+shared~sycl build_type=RelWithDebInfo arch=linux-amzn2-skylake_avx512
^cmake@3.21.4%gcc@7.3.1~doc+ncurses+openssl+ownlibs~qt build_type=Release arch=linux-amzn2-skylake_avx512
^ncurses@6.2%gcc@7.3.1~symlinks+termlib abi=none arch=linux-amzn2-skylake_avx512
^pkgconf@1.8.0%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
^openssl@1.1.1l%gcc@7.3.1~docs certs=system arch=linux-amzn2-skylake_avx512
^perl@5.34.0%gcc@7.3.1+cpanm+shared+threads arch=linux-amzn2-skylake_avx512
^berkeley-db@18.1.40%gcc@7.3.1+cxx~docs+stl patches=b231fcc4d5cff05e5c3a4814f6a5af0e9a966428dc2176540d2c05aff41de522 arch=linux-amzn2-skylake_avx512
^bzip2@1.0.8%gcc@7.3.1~debug~pic+shared arch=linux-amzn2-skylake_avx512
^diffutils@3.8%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
^libiconv@1.16%gcc@7.3.1 libs=shared,static arch=linux-amzn2-skylake_avx512
^gdbm@1.19%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
^readline@8.1%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
^zlib@1.2.11%gcc@7.3.1+optimize+pic+shared arch=linux-amzn2-skylake_avx512
^fftw@3.3.10%gcc@7.3.1+mpi~openmp~pfft_patches precision=double,float arch=linux-amzn2-skylake_avx512
^openmpi@4.1.1%gcc@7.3.1~atomics~cuda~cxx~cxx_exceptions+gpfs~internal-hwloc~java~legacylaunchers~lustre~memchecker~pmi~pmix~singularity~sqlite3+static~thread_multiple+vt+wrapper-rpath fabrics=none schedulers=none arch=linux-amzn2-skylake_avx512
^hwloc@2.5.0%gcc@7.3.1~cairo~cuda~gl~libudev+libxml2~netloc~nvml~opencl+pci~rocm+shared arch=linux-amzn2-skylake_avx512
^libpciaccess@0.16%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
^libtool@2.4.6%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
^m4@1.4.19%gcc@7.3.1+sigsegv patches=9dc5fbd0d5cb1037ab1e6d0ecc74a30df218d0a94bdd5a02759a97f62daca573,bfdffa7c2eb01021d5849b36972c069693654ad826c1a20b53534009a4ec7a89 arch=linux-amzn2-skylake_avx512
^libsigsegv@2.13%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
^util-macros@1.19.3%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
^libxml2@2.9.12%gcc@7.3.1~python arch=linux-amzn2-skylake_avx512
^xz@5.2.5%gcc@7.3.1~pic libs=shared,static arch=linux-amzn2-skylake_avx512
^libevent@2.1.12%gcc@7.3.1+openssl arch=linux-amzn2-skylake_avx512
^numactl@2.0.14%gcc@7.3.1 patches=4e1d78cbbb85de625bad28705e748856033eaafab92a66dffd383a3d7e00cc94,62fc8a8bf7665a60e8f4c93ebbd535647cebf74198f7afafec4c085a8825c006,ff37630df599cfabf0740518b91ec8daaf18e8f288b19adaae5364dc1f6b2296 arch=linux-amzn2-skylake_avx512
^autoconf@2.69%gcc@7.3.1 patches=35c449281546376449766f92d49fc121ca50e330e60fefcfc9be2af3253082c2,7793209b33013dc0f81208718c68440c5aae80e7a1c4b8d336e382525af791a7,a49dd5bac3b62daa0ff688ab4d508d71dbd2f4f8d7e2a02321926346161bf3ee arch=linux-amzn2-skylake_avx512
^automake@1.16.3%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
^openssh@8.7p1%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
^libedit@3.1-20210216%gcc@7.3.1 arch=linux-amzn2-skylake_avx512The weird part is that for some packages it does pick up the correct MPI and I cannot find out when. One working example is fftw:
[ec2-user@ip-172-31-1-92 spack]$ echo "packages:
gromacs:
providers:
mpi: [intel-mpi]
fftw:
providers:
mpi: [intel-mpi]
" > etc/spack/packages.yaml
[ec2-user@ip-172-31-1-92 spack]$ spack solve fftw
==> Best of 8 considered solutions.
==> Optimization Criteria:
Priority Criterion Value
1 deprecated versions used 0
2 version weight 0
3 number of non-default variants (roots) 0
4 preferred providers for roots 0
5 number of values in multi valued variants (root) -2
6 number of non-default variants (non-roots) 0
7 preferred providers (non-roots) 0
8 compiler mismatches 0
9 version badness 0
10 number of values in multi valued variants (non-root) 0
11 non-preferred compilers 0
12 target mismatches 0
13 non-preferred targets 0
fftw@3.3.10%gcc@7.3.1+mpi~openmp~pfft_patches precision=double,float arch=linux-amzn2-skylake_avx512
^intel-mpi@2019.10.317%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
[ec2-user@ip-172-31-1-92 spack]$ rm etc/spack/packages.yaml
[ec2-user@ip-172-31-1-92 spack]$ spack solve fftw
==> Best of 9 considered solutions.
==> Optimization Criteria:
Priority Criterion Value
1 deprecated versions used 0
2 version weight 0
3 number of non-default variants (roots) 0
4 preferred providers for roots 0
5 number of values in multi valued variants (root) -2
6 number of non-default variants (non-roots) 0
7 preferred providers (non-roots) 0
8 compiler mismatches 0
9 version badness 2
10 number of values in multi valued variants (non-root) -6
11 non-preferred compilers 0
12 target mismatches 0
13 non-preferred targets 0
fftw@3.3.10%gcc@7.3.1+mpi~openmp~pfft_patches precision=double,float arch=linux-amzn2-skylake_avx512
^openmpi@4.1.1%gcc@7.3.1~atomics~cuda~cxx~cxx_exceptions+gpfs~internal-hwloc~java~legacylaunchers~lustre~memchecker~pmi~pmix~singularity~sqlite3+static~thread_multiple+vt+wrapper-rpath fabrics=none schedulers=none arch=linux-amzn2-skylake_avx512
^hwloc@2.5.0%gcc@7.3.1~cairo~cuda~gl~libudev+libxml2~netloc~nvml~opencl+pci~rocm+shared arch=linux-amzn2-skylake_avx512
^libpciaccess@0.16%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
^libtool@2.4.6%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
^m4@1.4.19%gcc@7.3.1+sigsegv patches=9dc5fbd0d5cb1037ab1e6d0ecc74a30df218d0a94bdd5a02759a97f62daca573,bfdffa7c2eb01021d5849b36972c069693654ad826c1a20b53534009a4ec7a89 arch=linux-amzn2-skylake_avx512
^libsigsegv@2.13%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
^pkgconf@1.8.0%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
^util-macros@1.19.3%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
^libxml2@2.9.12%gcc@7.3.1~python arch=linux-amzn2-skylake_avx512
^libiconv@1.16%gcc@7.3.1 libs=shared,static arch=linux-amzn2-skylake_avx512
^xz@5.2.5%gcc@7.3.1~pic libs=shared,static arch=linux-amzn2-skylake_avx512
^zlib@1.2.11%gcc@7.3.1+optimize+pic+shared arch=linux-amzn2-skylake_avx512
^ncurses@6.2%gcc@7.3.1~symlinks+termlib abi=none arch=linux-amzn2-skylake_avx512
^libevent@2.1.12%gcc@7.3.1+openssl arch=linux-amzn2-skylake_avx512
^openssl@1.1.1l%gcc@7.3.1~docs certs=system arch=linux-amzn2-skylake_avx512
^perl@5.34.0%gcc@7.3.1+cpanm+shared+threads arch=linux-amzn2-skylake_avx512
^berkeley-db@18.1.40%gcc@7.3.1+cxx~docs+stl patches=b231fcc4d5cff05e5c3a4814f6a5af0e9a966428dc2176540d2c05aff41de522 arch=linux-amzn2-skylake_avx512
^bzip2@1.0.8%gcc@7.3.1~debug~pic+shared arch=linux-amzn2-skylake_avx512
^diffutils@3.8%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
^gdbm@1.19%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
^readline@8.1%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
^numactl@2.0.14%gcc@7.3.1 patches=4e1d78cbbb85de625bad28705e748856033eaafab92a66dffd383a3d7e00cc94,62fc8a8bf7665a60e8f4c93ebbd535647cebf74198f7afafec4c085a8825c006,ff37630df599cfabf0740518b91ec8daaf18e8f288b19adaae5364dc1f6b2296 arch=linux-amzn2-skylake_avx512
^autoconf@2.69%gcc@7.3.1 patches=35c449281546376449766f92d49fc121ca50e330e60fefcfc9be2af3253082c2,7793209b33013dc0f81208718c68440c5aae80e7a1c4b8d336e382525af791a7,a49dd5bac3b62daa0ff688ab4d508d71dbd2f4f8d7e2a02321926346161bf3ee arch=linux-amzn2-skylake_avx512
^automake@1.16.3%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
^openssh@8.7p1%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
^libedit@3.1-20210216%gcc@7.3.1 arch=linux-amzn2-skylake_avx512
Error message
No response
Information on your system
[ec2-user@ip-172-31-1-92 spack]$ spack debug report
- Spack: 0.16.3
- Python: 3.7.10
- Platform: linux-amzn2-skylake_avx512
- Concretizer: clingo
General information
Steps to reproduce
I have found an issue with the concretizer not picking up the correct providers when described in
packages.yaml. If providers are aded to the command line, e.g.^intel-mpi ^intel-mkl, all works as expected. Here is one example of intel-mpi not being picked up.Tagging @tgamblin as discussed yesterday.
The default MPI provider after pulling the
developbranch is Open MPI:The default provider can be overwritten by adding a dependency to a different MPI on the command line:
However, if I try ot overwrite the provider in
packages.yamlit is not picked up:The weird part is that for some packages it does pick up the correct MPI and I cannot find out when. One working example is fftw:
Error message
No response
Information on your system
[ec2-user@ip-172-31-1-92 spack]$ spack debug report
General information
spack debug reportand reported the version of Spack/Python/Platform