Skip to content

Add depends_on([c,cxx,fortran])#45217

Merged
haampie merged 10 commits intodevelopfrom
hs/ecosystem/c-cxx-fortran-deps
Jul 17, 2024
Merged

Add depends_on([c,cxx,fortran])#45217
haampie merged 10 commits intodevelopfrom
hs/ecosystem/c-cxx-fortran-deps

Conversation

@haampie
Copy link
Copy Markdown
Member

@haampie haampie commented Jul 12, 2024

generated with https://github.com/haampie/spack-language-to-package-py: based on file extensions in archives w/o looking at file contents or build systems in detail.

the idea is people remove # generated when it's verified.

@haampie haampie marked this pull request as draft July 12, 2024 15:49
@alalazo alalazo marked this pull request as ready for review July 12, 2024 16:42
@alalazo
Copy link
Copy Markdown
Member

alalazo commented Jul 12, 2024

Removed from being a draft to check concretization in pipelines

@alalazo alalazo added this to the v0.23.0 milestone Jul 13, 2024
@haampie haampie force-pushed the hs/ecosystem/c-cxx-fortran-deps branch 3 times, most recently from 326c081 to 9d19d3a Compare July 15, 2024 15:25
@tgamblin
Copy link
Copy Markdown
Member

@alalazo can you time concretization on this?

@tgamblin
Copy link
Copy Markdown
Member

Based on this:

C_EXT = {".c"}
CXX_EXT = {".cpp", ".cc", ".cxx", ".c++"}
FORTRAN_EXT = {".f", ".f77", ".f90", ".f95", ".f03", ".f08"}

I think we're missing a few suffixes -- these are what cloc uses:

> cloc --show-lang=c
C                          (c, cats, ec, idc, pgc)
> cloc --show-lang=c++
C++                        (C, c++, cc, CPP, cpp, cxx, h++, inl, ipp, pcc, tcc, tpp)
> cloc --show-lang="fortran 77"
Fortran 77                 (F, F77, f77, FOR, FTN, ftn, pfo)
> cloc --show-lang="fortran 90"
Fortran 90                 (F90, f90)
> cloc --show-lang="fortran 95"
Fortran 95                 (F95, f95)

And I guess I would also add hpp for C++ as I see that frequently. Once @alecbcs gets the source mirror fully downloaded here, can we go and re-run your script and do some sampling to check it?

@haampie haampie force-pushed the hs/ecosystem/c-cxx-fortran-deps branch from 9d19d3a to 88eab7f Compare July 16, 2024 08:40
@haampie
Copy link
Copy Markdown
Member Author

haampie commented Jul 16, 2024

Once @alecbcs gets the source mirror fully downloaded here, can we go and re-run your script and do some sampling to check it?

Don't see the point of waiting for that. The # generated comment is enough to update packages at a later stage, I doubt it can be improved.

I've force pushed the PR with the patch generated by https://github.com/haampie/spack-language-to-package-py/actions/runs/9952936512/job/27495303284, the number of changed files is 89, half of which is about .C classified as C++ now instead of C, so that leaves about 1.5% of package changes due to flaky downloads compared to last run. That's likely already below the false positive threshold, we don't need 100% accuracy on 90% reliable data...

IMO let's merge this now, it's easy enough to fix things later. I don't want to rebase this.

@haampie haampie force-pushed the hs/ecosystem/c-cxx-fortran-deps branch 2 times, most recently from c266ae4 to 3fe4830 Compare July 16, 2024 10:37
@spackbot-app spackbot-app bot added the core PR affects Spack core functionality label Jul 16, 2024
@haampie haampie force-pushed the hs/ecosystem/c-cxx-fortran-deps branch from 3fe4830 to 57e017d Compare July 16, 2024 11:14
@spackbot-app spackbot-app bot added the gitlab Issues related to gitlab integration label Jul 16, 2024
@alalazo
Copy link
Copy Markdown
Member

alalazo commented Jul 17, 2024

I confirm that I see no significant differences in concretization times:

  • Spack: 0.23.0.dev0 (89c0b4a)
  • Python: 3.11.5
  • Platform: linux-ubuntu20.04-icelake
  • Concretizer: clingo

radiuss.pr.csv
radiuss.develop.csv
radiuss.txt

radiuss

A few specs seems slower, a few faster. Notably, py-deephyper (which is kind of unstable in concretization time) is 30 sec. faster in this branch using gcc. Maybe adding information on fortran/cxx virtuals helped concretizaton.

I would be for merging this asap,

@haampie
Copy link
Copy Markdown
Member Author

haampie commented Jul 17, 2024

apart from the significant differences no significant differences?

@alalazo
Copy link
Copy Markdown
Member

alalazo commented Jul 17, 2024

apart from the significant differences no significant differences?

I mean, there are no trends for things getting generally slower or faster due to this PR 😉

@haampie
Copy link
Copy Markdown
Member Author

haampie commented Jul 17, 2024

I see tons of rebuilds. Non-determinism?

Edit: nope it was 40b3909

@haampie
Copy link
Copy Markdown
Member Author

haampie commented Jul 17, 2024

FWIW, there is non-determinism, but not due to this PR :)

These two specs

warpx ^openpmd-api@0.14
warpx ^openpmd-api@0.15

have the same "version badness", because

  • openpmd-api@0.14 is 3 versions from latest but does not depend on toml11
  • openpmd-api@0.15 is latest, but depends toml11 3 versions from latest

🎉

not a blocker here, just happened to be that warpx ^openpmd-api@0.14 was picked and that exposed a missing compat bound.

@haampie
Copy link
Copy Markdown
Member Author

haampie commented Jul 17, 2024

cairo failure is also unrelated and also caused non-deterministic concretization. Fix in #45272

@haampie haampie disabled auto-merge July 17, 2024 14:04
@haampie haampie merged commit 7b9f8ab into develop Jul 17, 2024
@haampie haampie deleted the hs/ecosystem/c-cxx-fortran-deps branch July 17, 2024 14:07
@tgamblin tgamblin changed the title Add depends_on([c,cxx,fortran]) (#45217) Add depends_on([c,cxx,fortran]) Jul 17, 2024
@ax3l
Copy link
Copy Markdown
Member

ax3l commented Jul 18, 2024

Thank you, @haampie - both for the excellent detection (confirming in #45251) and for fixing the WarpX / openPMD-api constraint.

This was referenced Jul 18, 2024
diehlpk pushed a commit to diehlpk/spack that referenced this pull request Aug 14, 2024
Add language dependencies `c`, `cxx`, and `fortran`.

These `depends_on` statements are auto-generated based on file extensions found
in source tarballs/zipfiles.

The `# generated` comment can be removed by package maintainers after
validating correctness.
FrederickDeny pushed a commit to FrederickDeny/spack that referenced this pull request Aug 26, 2024
Add language dependencies `c`, `cxx`, and `fortran`.

These `depends_on` statements are auto-generated based on file extensions found
in source tarballs/zipfiles.

The `# generated` comment can be removed by package maintainers after
validating correctness.
@alalazo alalazo mentioned this pull request Dec 4, 2024
15 tasks
haampie added a commit to spack/spack-packages that referenced this pull request Nov 7, 2025
This was missed back in July 2024 likely because the R packages failed
to download. That's because in the script I used a direct fetch without
Spack's mirror, which failed, because R's default URL property is
incorrect.

spack/spack#45217

Signed-off-by: Harmen Stoppels <me@harmenstoppels.nl>
bollig pushed a commit to bollig/spack-packages that referenced this pull request Nov 7, 2025
This was missed back in July 2024 likely because the R packages failed
to download. That's because in the script I used a direct fetch without
Spack's mirror, which failed, because R's default URL property is
incorrect.

spack/spack#45217

Signed-off-by: Harmen Stoppels <me@harmenstoppels.nl>
alalazo pushed a commit to spack/spack-packages that referenced this pull request Nov 11, 2025
This was missed back in July 2024 likely because the R packages failed
to download. That's because in the script I used a direct fetch without
Spack's mirror, which failed, because R's default URL property is
incorrect.

spack/spack#45217

Signed-off-by: Harmen Stoppels <me@harmenstoppels.nl>
mkrack pushed a commit to mkrack/spack-packages that referenced this pull request Dec 23, 2025
This was missed back in July 2024 likely because the R packages failed
to download. That's because in the script I used a direct fetch without
Spack's mirror, which failed, because R's default URL property is
incorrect.

spack/spack#45217

Signed-off-by: Harmen Stoppels <me@harmenstoppels.nl>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core PR affects Spack core functionality dependencies gitlab Issues related to gitlab integration intel python R update-package

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants