Improve hit-rate on buildcaches#43272
Merged
alalazo merged 9 commits intospack:developfrom Apr 5, 2024
Merged
Conversation
Member
Smart. On second thought: does it work with externals? |
06c0c69 to
db6f8f9
Compare
haampie
reviewed
Mar 20, 2024
db6f8f9 to
2379050
Compare
Member
Author
|
Example of the current state (the occasional |
630a5fe to
5c5de4d
Compare
Member
Author
|
The changes here give a much better hit rate on store / buildcaches. I tried to benchmark on a baseline without any mirror and it's mostly the same. There's one consistent spike I need to investigate further: radiuss.pr.nomirror.csv
|
5c5de4d to
ed006df
Compare
alalazo
commented
Mar 25, 2024
ed006df to
f550c55
Compare
The mismatch occurs on an edge. Previously it was assigned the parent priority, now it is assigned to the child priority. This should make reuse from buildcaches or store more likely, since most mismatches will be counted with "reused" priority.
We don't want to e.g. switch other attributes because we cannot reuse an old installed runtime.
This is such that the version of the runtime would not influence whether we should reuse a spec. Compiler mismatches are considered for runtimes, to avoid situations where compiling foo%gcc@9 brings in gcc-runtime%gcc@13 if gcc@13 is among the available compilers
This should ensure that we do not reuse specs that could be broken, as they expect the compiler to be installed in a specific place.
This is to avoid differences on m1 with respect to x86_64 machines
This is to avoid differences on m1 with respect to x86_64
5376bef to
523c704
Compare
Member
Author
|
I just tried to re-measure the benchmark above. There must be something in the
radiuss.pr.nomirror.csv
Footnotes
|
haampie
reviewed
Apr 5, 2024
haampie
reviewed
Apr 5, 2024
Member
|
I think this is fine. As per discussion in dm, users can always But we have to document it (in high impact changes for instance). |
haampie
approved these changes
Apr 5, 2024
tldahlgren
pushed a commit
to tldahlgren/spack
that referenced
this pull request
Apr 23, 2024
* Relax compiler and target mismatches The mismatch occurs on an edge. Previously it was assigned the parent priority, now it is assigned the child priority. This should make reuse from buildcaches or store more likely, since most mismatches will be counted with "reused" priority. * Optimize version badness for runtimes at very low priority We don't want to e.g. switch other attributes because we cannot reuse an old installed runtime. * Optimize runtime attributes at very low priority This is such that the version of the runtime would not influence whether we should reuse a spec. Compiler mismatches are considered for runtimes, to avoid situations where compiling foo%gcc@9 brings in gcc-runtime%gcc@13 if gcc@13 is among the available compilers * Exclude specs without runtimes from reuse This should ensure that we do not reuse specs that could be broken, as they expect the compiler to be installed in a specific place.
teaguesterling
pushed a commit
to teaguesterling/spack
that referenced
this pull request
Jun 15, 2024
* Relax compiler and target mismatches The mismatch occurs on an edge. Previously it was assigned the parent priority, now it is assigned the child priority. This should make reuse from buildcaches or store more likely, since most mismatches will be counted with "reused" priority. * Optimize version badness for runtimes at very low priority We don't want to e.g. switch other attributes because we cannot reuse an old installed runtime. * Optimize runtime attributes at very low priority This is such that the version of the runtime would not influence whether we should reuse a spec. Compiler mismatches are considered for runtimes, to avoid situations where compiling foo%gcc@9 brings in gcc-runtime%gcc@13 if gcc@13 is among the available compilers * Exclude specs without runtimes from reuse This should ensure that we do not reuse specs that could be broken, as they expect the compiler to be installed in a specific place.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR tries to modify the optimization criteria to have more hits on buildcaches by default.
Modifications:
gcc-runtimeto be reused (they'll be mostly broken at runtime)