Steps to reproduce
I'm seeing some strange differences in concretization where a package is being concretized differently. The only difference being an external pkg-config. Sometimes the external pkg-config is used and for other packages pkgconf is being built from scratch resulting in the same packages being built twice with different hashes.
I was able to reduce this behavior with a simple example.
I have a Spack environment with an external pkg-config:
packages:
pkg-config:
externals:
- spec: pkg-config@0.27.1
prefix: /usr
A bundle with a single package:
class NcioBundle(BundlePackage):
version('1.0.0')
depends_on('ncio')
And a simple spec list:
specs:
- ncio-bundle
- ncio
You can see the full concretization below but the only difference is pkgconf and pkg-config.
I have my environment activated and I run spack concretize and the same spec concretizes differently:
^pkgconf@1.8.0%intel@2021.3.0 arch=linux-centos7-cascadelake
^pkg-config@0.27.1%intel@2021.3.0+internal_glib
You can see when part of a bundle it's using the external pkg-config, but if not it's building pkgconf.
In my real use-case I have several bundles being built and two contain this ncio package and they being concretized differently exactly like my example.
I can solve this issue by setting pkgconfig buildable: False, or if I modify the built-in packages.yaml to prefer pkg-config, or override it in my own packages.yaml
|
pkgconfig: [pkgconf, pkg-config] |
But why would curl concretize differently because one is in a bundle? Why isn't my external pkg-config always being used?
Error message
Full concretization below, notice how the only difference is pkg-config and pkgconf.
==> Concretized ncio-bundle
- 7isq7nk ncio-bundle@1.0.0%intel@2021.3.0 arch=linux-centos7-cascadelake
[+] giuwrww ^ncio@1.0.0%intel@2021.3.0~ipo build_type=RelWithDebInfo arch=linux-centos7-cascadelake
[+] zq5gb6h ^cmake@3.22.1%intel@2021.3.0~doc+ncurses+openssl+ownlibs~qt build_type=Release arch=linux-centos7-cascadelake
[+] qzmzrqg ^ncurses@6.3.20211021%intel@2021.3.0~symlinks+termlib abi=6 arch=linux-centos7-cascadelake
[+] wqpzmd4 ^openssl@1.1.1n%intel@2021.3.0~docs certs=system arch=linux-centos7-cascadelake
[+] di4fmna ^intel-oneapi-mpi@2021.3.0%intel@2021.3.0~external-libfabric~ilp64 arch=linux-centos7-cascadelake
[+] qjwlh4i ^netcdf-fortran@4.5.3%intel@2021.3.0~doc+pic+shared arch=linux-centos7-cascadelake
[+] jzc2wgy ^netcdf-c@4.7.4%intel@2021.3.0+dap~fsync~hdf4~jna+mpi+parallel-netcdf+pic+shared patches=2c88dfbd6d339a0336a43b14a65a1d1df995b853b645e4af612617612a642a53 arch=linux-centos7-cascadelake
[+] 3u4dxij ^autoconf@2.69%intel@2021.3.0 patches=7793209b33013dc0f81208718c68440c5aae80e7a1c4b8d336e382525af791a7 arch=linux-centos7-cascadelake
[+] y4uegic ^m4@1.4.16%intel@2021.3.0+sigsegv arch=linux-centos7-cascadelake
[+] vnjncy4 ^perl@5.16.3%intel@2021.3.0~cpanm+shared+threads patches=0eac10ed90aeb0459ad8851f88081d439a4e41978e586ec743069e8b059370ac,3bbd7d6f9933d80b9571533867b444c6f8f5a1ba0575bfba1fba4db9d885a71a arch=linux-centos7-cascadelake
[+] cn4xdnq ^automake@1.16.5%intel@2021.3.0 arch=linux-centos7-cascadelake
[+] m62w6ev ^curl@7.80.0%intel@2021.3.0~gssapi~ldap~libidn2~librtmp~libssh~libssh2~nghttp2 tls=openssl arch=linux-centos7-cascadelake
[+] h3n2aln ^pkg-config@0.27.1%intel@2021.3.0+internal_glib patches=49ffcd644e190dc5efcb2fab491177811ea746c1a526f75d77118c2706574358 arch=linux-centos7-cascadelake
[+] ioczwxv ^zlib@1.2.11%intel@2021.3.0+optimize+pic+shared arch=linux-centos7-cascadelake
[+] j5nqr2n ^hdf5@1.12.1%intel@2021.3.0~cxx+fortran+hl~ipo~java+mpi+shared~szip+threadsafe+tools api=default build_type=RelWithDebInfo arch=linux-centos7-cascadelake
[+] b7fa37y ^numactl@2.0.14%intel@2021.3.0 patches=4e1d78cbbb85de625bad28705e748856033eaafab92a66dffd383a3d7e00cc94,62fc8a8bf7665a60e8f4c93ebbd535647cebf74198f7afafec4c085a8825c006,ff37630df599cfabf0740518b91ec8daaf18e8f288b19adaae5364dc1f6b2296 arch=linux-centos7-cascadelake
[+] ycssfzg ^libtool@2.4.6%intel@2021.3.0 arch=linux-centos7-cascadelake
[+] at325hc ^parallel-netcdf@1.12.2%intel@2021.3.0~burstbuffer+cxx+fortran+pic+shared arch=linux-centos7-cascadelake
==> Concretized ncio
- xfpfe3x ncio@1.0.0%intel@2021.3.0~ipo build_type=RelWithDebInfo arch=linux-centos7-cascadelake
[+] zq5gb6h ^cmake@3.22.1%intel@2021.3.0~doc+ncurses+openssl+ownlibs~qt build_type=Release arch=linux-centos7-cascadelake
[+] qzmzrqg ^ncurses@6.3.20211021%intel@2021.3.0~symlinks+termlib abi=6 arch=linux-centos7-cascadelake
[+] wqpzmd4 ^openssl@1.1.1n%intel@2021.3.0~docs certs=system arch=linux-centos7-cascadelake
[+] di4fmna ^intel-oneapi-mpi@2021.3.0%intel@2021.3.0~external-libfabric~ilp64 arch=linux-centos7-cascadelake
[+] nitel6p ^netcdf-fortran@4.5.3%intel@2021.3.0~doc+pic+shared arch=linux-centos7-cascadelake
[+] jueep3f ^netcdf-c@4.7.4%intel@2021.3.0+dap~fsync~hdf4~jna+mpi+parallel-netcdf+pic+shared patches=2c88dfbd6d339a0336a43b14a65a1d1df995b853b645e4af612617612a642a53 arch=linux-centos7-cascadelake
[+] 3u4dxij ^autoconf@2.69%intel@2021.3.0 patches=7793209b33013dc0f81208718c68440c5aae80e7a1c4b8d336e382525af791a7 arch=linux-centos7-cascadelake
[+] y4uegic ^m4@1.4.16%intel@2021.3.0+sigsegv arch=linux-centos7-cascadelake
[+] vnjncy4 ^perl@5.16.3%intel@2021.3.0~cpanm+shared+threads patches=0eac10ed90aeb0459ad8851f88081d439a4e41978e586ec743069e8b059370ac,3bbd7d6f9933d80b9571533867b444c6f8f5a1ba0575bfba1fba4db9d885a71a arch=linux-centos7-cascadelake
[+] cn4xdnq ^automake@1.16.5%intel@2021.3.0 arch=linux-centos7-cascadelake
[+] m62w6ev ^curl@7.80.0%intel@2021.3.0~gssapi~ldap~libidn2~librtmp~libssh~libssh2~nghttp2 tls=openssl arch=linux-centos7-cascadelake
[+] 5j7y6ul ^pkgconf@1.8.0%intel@2021.3.0 arch=linux-centos7-cascadelake
[+] ioczwxv ^zlib@1.2.11%intel@2021.3.0+optimize+pic+shared arch=linux-centos7-cascadelake
[+] uxvdtkv ^hdf5@1.12.1%intel@2021.3.0~cxx+fortran+hl~ipo~java+mpi+shared~szip+threadsafe+tools api=default build_type=RelWithDebInfo arch=linux-centos7-cascadelake
[+] b7fa37y ^numactl@2.0.14%intel@2021.3.0 patches=4e1d78cbbb85de625bad28705e748856033eaafab92a66dffd383a3d7e00cc94,62fc8a8bf7665a60e8f4c93ebbd535647cebf74198f7afafec4c085a8825c006,ff37630df599cfabf0740518b91ec8daaf18e8f288b19adaae5364dc1f6b2296 arch=linux-centos7-cascadelake
[+] ycssfzg ^libtool@2.4.6%intel@2021.3.0 arch=linux-centos7-cascadelake
[+] at325hc ^parallel-netcdf@1.12.2%intel@2021.3.0~burstbuffer+cxx+fortran+pic+shared arch=linux-centos7-cascadelake
Information on your system
- Spack: 0.16.1-6587-6f2df33
- Python: 3.9.7
- Platform: linux-centos7-cascadelake
- Concretizer: clingo
General information
Steps to reproduce
I'm seeing some strange differences in concretization where a package is being concretized differently. The only difference being an external
pkg-config. Sometimes the externalpkg-configis used and for other packagespkgconfis being built from scratch resulting in the same packages being built twice with different hashes.I was able to reduce this behavior with a simple example.
I have a Spack environment with an external
pkg-config:A bundle with a single package:
And a simple spec list:
You can see the full concretization below but the only difference is
pkgconfandpkg-config.I have my environment activated and I run
spack concretizeand the same spec concretizes differently:You can see when part of a bundle it's using the external
pkg-config, but if not it's buildingpkgconf.In my real use-case I have several bundles being built and two contain this
nciopackage and they being concretized differently exactly like my example.I can solve this issue by setting
pkgconfigbuildable: False, or if I modify the built-inpackages.yamlto preferpkg-config, or override it in my ownpackages.yamlspack/etc/spack/defaults/packages.yaml
Line 49 in 57f5c92
But why would
curlconcretize differently because one is in a bundle? Why isn't my externalpkg-configalways being used?Error message
Full concretization below, notice how the only difference is
pkg-configandpkgconf.Information on your system
General information
spack debug reportand reported the version of Spack/Python/Platform