Hi @haampie! I noticed that the following package(s) don't yet have maintainers:

• musl

Are you interested in adopting any of these package(s)? If so, simply add the following to the package class:

    maintainers("haampie")

If not, could you contact the developers of this package and see if they are interested? You can quickly see who has worked on a package with spack blame:

$ spack blame musl

Thank you for your help! Please don't add maintainers without their consent.

You don't have to be a Spack expert or package developer in order to be a "maintainer," it just gives us a list of users willing to review PRs or debug issues relating to this package. A package can have multiple maintainers; just add a list of GitHub handles of anyone who wants to volunteer.

With respect to:

fix some bug in the concretizer that prevents this (circular dep with gcc-runtime?)

This is working, provided we add / inject glibc as an external package. Otherwise there will be circular dependencies due to glibc dependency on e.g. linux-kernel that depends on gcc-runtime and glibc itself.

For instance, on:

• Spack: 0.22.0.dev0 (9a17670)
• Python: 3.11.5
• Platform: linux-ubuntu20.04-icelake
• Concretizer: clingo

and using:

packages:
  glibc:                                                                                                                                                                                                                                                                         
    externals:                                                                                                                                                                                                                                                                   
    - spec: "glibc@=2.31"                                                                                                                                                                                                                                                        
      prefix: /usr          

I get the following for:

$ spack solve zlib-ng

Right, my original idea was to make it an automatic external

The last commit (19d2c2c) gives a fully working GCC, but we still need some tightening for other compilers. Anyhow, to try the new feature:

$ spack mirror add devtools https://raw.githubusercontent.com/alalazo/test-buildcaches/main/devtools
$ spack buildcache keys -it 
$ spack env activate --temp
$ spack add hdf5~mpi
$ spack concretize --reuse -f

On my platform:

• Spack: 0.22.0.dev0 (19d2c2c)
• Python: 3.11.5
• Platform: linux-ubuntu20.04-icelake
• Concretizer: clingo

This reuses a bunch of nodes from the buildcache that has almalinux8 builds:

$ spack spec -L zlib-ng
Input spec
--------------------------------
 -   zlib-ng

Concretized
--------------------------------
 -   gaxnrttfo2lk7yzdd2nsm5yxf55zadsv  zlib-ng@2.1.6%gcc@13.2.0+compat+new_strategies+opt+pic+shared build_system=autotools arch=linux-ubuntu23.10-zen2
[+]  tofiopdeqtfqrjqrfnclgnvx2d47yraq      ^gcc-runtime@13.2.0%gcc@13.2.0 build_system=generic arch=linux-ubuntu23.10-zen2
[e]  gx2zlxihjlhzfkcz47rjr2iir5hrogi7          ^glibc@2.38%gcc@13.2.0 build_system=autotools arch=linux-ubuntu23.10-zen2
[e]  gx2zlxihjlhzfkcz47rjr2iir5hrogi7      ^glibc@2.38%gcc@13.2.0 build_system=autotools arch=linux-ubuntu23.10-zen2
[+]  boxbsmc7a5ycptkstw777ao5xecdo34h      ^gmake@4.4.1%gcc@13.2.0~guile build_system=generic arch=linux-ubuntu23.10-zen2

not sure what that is about.

One glibc reused, one "fresh" resulting in two instances of the same object?

(Have a hard time reducing it to mwe...)

not sure what that is about.

The issue was that the libc was not "reused". Fixed in b049375

To sum up the relocation struggles:

1. The interpreter path is on most distros "out of prefix", meaning default prefix -> prefix relocation is insufficient
2. On almost all distros it's out of prefix exactly such that relocation is unnecessary
3. But we do care about that small fraction of distros that is different (EESSI users w/ Gentoo Prefix, and NixOS)
4. We preferably don't wanna model in terms of the system libc, there can be multiple
5. It's most robust to determine the interpreter path from the compiler's link line
6. But we also want spack install gcc to work without a compiler installed: there may not be a compiler to query where libc is.
7. If no configured compiler: we can determine the interpreter path from the current Python process's PT_INTERP (except if Python is static, which is a valid use case when bootstrapping Spack on systems without Python)
8. Alternatively we could run /usr/bin/ldd or ldd from PATH, or scan /usr for a libc.so with arch compatible with the currently running Python process.
9. Finally there's an annoying issue that after concretization we just have libc with say prefix=/usr, but not the interpreter path, so we'd have to query the compiler again to get it.

One way or another we get the interpreter path, and add a replacement <old interpreter path> -> <new interpreter path>.

@spackbot run pipeline

I've started that pipeline for you!

One test in pipelines took more than 6 hrs. I guess we need to put a timeout on tests.

We decided to stick with exact prefix match for now so that reuse between nix / gentoo prefix and classical distros isn't possible. That makes the scope of this pr a bit smaller.