I believe this to be a bug, rather than behavior just not matching my expectations.
When a package has variants, and you have multiple externals defined to match those variants - spack is having troubles picking the correct external - if the full variants list isnt specified.
This means it will pick an external for concretization which does not match the spec asked for.
Steps to reproduce the issue
Lets define a simple dummy package ext-spec, with one variant (debug default false):
from spack import *
class ExtSpec(Package):
"""FIXME: Put a proper description of your package here."""
# FIXME: Add a proper url for your package's homepage here.
homepage = "https://www.example.com"
url = "ext-spec"
# FIXME: Add proper versions and checksums here.
version('1.2.3')
variant('debug', default=False, description='debug build')
def install(self, spec, prefix):
# FIXME: Unknown build system
make()
make('install')Now we can spec the package with the variant:
$ spack spec ext-spec
Input spec
--------------------------------
ext-spec
Concretized
--------------------------------
ext-spec@1.2.3%gcc@10.3.0~debug arch=linux-amzn2-graviton2
$ spack spec ext-spec+debug
Input spec
--------------------------------
ext-spec+debug
Concretized
--------------------------------
ext-spec@1.2.3%gcc@10.3.0+debug arch=linux-amzn2-graviton2
Great, those concretized specs look perfect.
Now lets define an externals for the package, with two installs - default and with debug enabled:
packages:
ext-spec:
externals:
- spec: ext-spec
prefix: /software/ext-spec/
- spec: ext-spec+debug
prefix: /software/ext-spec_debug/Now we can repeat our spec queries:
$ spack spec ext-spec
Input spec
--------------------------------
ext-spec
Concretized
--------------------------------
ext-spec@1.2.3%gcc@10.3.0~debug arch=linux-amzn2-graviton2
$ spack spec ext-spec+debug
Input spec
--------------------------------
ext-spec+debug
Concretized
--------------------------------
ext-spec@1.2.3%gcc@10.3.0~debug arch=linux-amzn2-graviton2
So even though we have asked for ext-spec+debug it is concretizing to ext-spec~debug, even though we have an external defined for that exact spec.
I believe this is because the externals selection is too permissive, and taking the 'default' spec and accepting that.
We can correct this behavior by formalizing the spec of the 'default' install:
packages:
ext-spec:
externals:
- spec: ext-spec~debug
prefix: /software/ext-spec/
- spec: ext-spec+debug
prefix: /software/ext-spec_debug/
Now our spec concretize correctly:
$ spack spec ext-spec
Input spec
--------------------------------
ext-spec
Concretized
--------------------------------
ext-spec@1.2.3%gcc@10.3.0~debug arch=linux-amzn2-graviton2
$ spack spec ext-spec+debug
Input spec
--------------------------------
ext-spec+debug
Concretized
--------------------------------
ext-spec@1.2.3%gcc@10.3.0+debug arch=linux-amzn2-graviton2
Note: This format is 'reproduced' from the externals example
Information on your system
$ spack debug report
* **Spack:** 0.16.2-3221-8e249c03de
* **Python:** 3.7.10
* **Platform:** linux-amzn2-graviton2
* **Concretizer:** original
Debug output, just in case.
$ spack --debug spec ext-spec+debug
==> [2021-06-24-10:04:18.498824] Imported spec from built-in commands
==> [2021-06-24-10:04:18.499689] Imported spec from built-in commands
Input spec
--------------------------------
ext-spec+debug
Concretized
--------------------------------
==> [2021-06-24-10:04:18.501474] Reading config file /software/spack/etc/spack/defaults/config.yaml
==> [2021-06-24-10:04:18.524815] Reading config file /software/spack/etc/spack/config.yaml
==> [2021-06-24-10:04:18.528468] Reading config file /software/spack/etc/spack/defaults/repos.yaml
==> [2021-06-24-10:04:18.540288] Reading config file /software/spack/etc/spack/defaults/packages.yaml
==> [2021-06-24-10:04:18.573902] Reading config file /software/spack/etc/spack/packages.yaml
==> [2021-06-24-10:04:18.585097] Reading config file /home/ec2-user/.spack/packages.yaml
==> [2021-06-24-10:04:18.590828] Reading config file /software/spack/etc/spack/compilers.yaml
==> [2021-06-24-10:04:18.615132] DATABASE LOCK TIMEOUT: 3s
==> [2021-06-24-10:04:18.615196] PACKAGE LOCK TIMEOUT: No timeout
ext-spec@1.2.3%gcc@10.3.0~debug arch=linux-amzn2-graviton2
Additional information
I believe this to be a bug, rather than behavior just not matching my expectations.
When a package has variants, and you have multiple externals defined to match those variants - spack is having troubles picking the correct external - if the full variants list isnt specified.
This means it will pick an external for concretization which does not match the spec asked for.
Steps to reproduce the issue
Lets define a simple dummy package ext-spec, with one variant (debug default false):
Now we can spec the package with the variant:
Great, those concretized specs look perfect.
Now lets define an externals for the package, with two installs - default and with debug enabled:
Now we can repeat our spec queries:
So even though we have asked for
ext-spec+debugit is concretizing toext-spec~debug, even though we have an external defined for that exact spec.I believe this is because the externals selection is too permissive, and taking the 'default' spec and accepting that.
We can correct this behavior by formalizing the spec of the 'default' install:
Now our spec concretize correctly:
Note: This format is 'reproduced' from the externals example
Information on your system
Debug output, just in case.
Additional information
spack debug reportand reported the version of Spack/Python/Platform