Environments: add run deps to shell modifications#23485
Merged
scheibelp merged 8 commits intospack:developfrom May 11, 2021
Merged
Environments: add run deps to shell modifications#23485scheibelp merged 8 commits intospack:developfrom
scheibelp merged 8 commits intospack:developfrom
Conversation
…oot specs. This now includes run dependencies of root specs
alalazo
previously approved these changes
May 7, 2021
Member
alalazo
left a comment
There was a problem hiding this comment.
It would be good to have a test for this, but otherwise LGTM
…environment modifications from deps are applied
Member
Author
|
@alalazo does the new test look reasonable? |
alalazo
previously approved these changes
May 11, 2021
Member
alalazo
left a comment
There was a problem hiding this comment.
@scheibelp Test looks good to me, and I am fine merging the PR as is. I just left a minor comment, so I'll wait for your opinion on that before merging.
alalazo
approved these changes
May 11, 2021
Member
|
@vchuravy FYI |
RikkiButler20
pushed a commit
to RikkiButler20/spack
that referenced
this pull request
Jun 9, 2021
When adding an Environment to a user's shell, Spack was only adding root specs. This now includes run dependencies of root specs.
haampie
added a commit
to haampie/spack
that referenced
this pull request
Sep 2, 2021
Partially undoes spack#23485 The current logic of `spack env activate` is as follows: 1. collect root specs 2. add root specs + transient run type deps of root specs to a list 3. call `environment_modifications_for_spec` for every spec of this list However, `environment_modifications_for_spec` also processes run type dependencies of the spec, so we're doing a lot of redundant work, resulting in O(n^2) calls to `modifications_from_dependencies` if we have a chain of deps `a` <- `b` <- ... <- `z` of deptype `run`. This PR drops step 2, so that we call `environment_modifications_for_spec` only on the root specs, and this function will process the run type for us anyways. Given an environment like this: ```yaml spack: specs: - py-flake8 - py-matplotlib - py-isort - py-sphinx - py-six - py-ipython ``` This is the time spent on `activate` before and after: **Before**: ``` In [1]: from spack.main import SpackCommand In [2]: env = SpackCommand('env') In [3]: %timeit env('activate', '--sh', '.') 2.56 s ± 11.4 ms per loop (mean ± std. dev. of 7 runs, 1 loop each) ``` **After**: ``` In [1]: from spack.main import SpackCommand In [2]: env = SpackCommand('env') In [3]: %timeit env('activate', '--sh', '.') 761 ms ± 4.36 ms per loop (mean ± std. dev. of 7 runs, 1 loop each) ``` which is 70% less.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #23368
When adding an Environment to a user's shell, Spack was only adding root specs. This now includes run dependencies of root specs.