Skip to content

Environments: add run deps to shell modifications#23485

Merged
scheibelp merged 8 commits intospack:developfrom
scheibelp:bugfix/add-run-deps-to-environment-shell-mods
May 11, 2021
Merged

Environments: add run deps to shell modifications#23485
scheibelp merged 8 commits intospack:developfrom
scheibelp:bugfix/add-run-deps-to-environment-shell-mods

Conversation

@scheibelp
Copy link
Copy Markdown
Member

Fixes #23368

When adding an Environment to a user's shell, Spack was only adding root specs. This now includes run dependencies of root specs.

…oot specs. This now includes run dependencies of root specs
@scheibelp scheibelp added the WIP label May 6, 2021
alalazo
alalazo previously approved these changes May 7, 2021
Copy link
Copy Markdown
Member

@alalazo alalazo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be good to have a test for this, but otherwise LGTM

@scheibelp
Copy link
Copy Markdown
Member Author

@alalazo does the new test look reasonable?

alalazo
alalazo previously approved these changes May 11, 2021
Copy link
Copy Markdown
Member

@alalazo alalazo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@scheibelp Test looks good to me, and I am fine merging the PR as is. I just left a minor comment, so I'll wait for your opinion on that before merging.

Comment thread lib/spack/spack/environment.py
@alalazo alalazo added bugfix Something wasn't working, here's a fix environments and removed WIP labels May 11, 2021
@scheibelp scheibelp merged commit 5230730 into spack:develop May 11, 2021
@tgamblin
Copy link
Copy Markdown
Member

@vchuravy FYI

RikkiButler20 pushed a commit to RikkiButler20/spack that referenced this pull request Jun 9, 2021
When adding an Environment to a user's shell, Spack was only adding
root specs. This now includes run dependencies of root specs.
haampie added a commit to haampie/spack that referenced this pull request Sep 2, 2021
Partially undoes spack#23485

The current logic of `spack env activate` is as follows:

1. collect root specs
2. add root specs + transient run type deps of root specs to a list
3. call `environment_modifications_for_spec` for every spec of this list

However, `environment_modifications_for_spec` also processes run type
dependencies of the spec, so we're doing a lot of redundant work,
resulting in O(n^2) calls to `modifications_from_dependencies` if we
have a chain of deps `a` <- `b` <- ... <- `z` of deptype `run`.

This PR drops step 2, so that we call
`environment_modifications_for_spec` only on the root specs, and this
function will process the run type for us anyways.

Given an environment like this:

```yaml
spack:
  specs:
  - py-flake8
  - py-matplotlib
  - py-isort
  - py-sphinx
  - py-six
  - py-ipython
```

This is the time spent on `activate` before and after:

**Before**:

```
In [1]: from spack.main import SpackCommand
In [2]: env = SpackCommand('env')
In [3]: %timeit env('activate', '--sh', '.')
2.56 s ± 11.4 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
```

**After**:

```
In [1]: from spack.main import SpackCommand
In [2]: env = SpackCommand('env')
In [3]: %timeit env('activate', '--sh', '.')
761 ms ± 4.36 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
```

which is 70% less.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugfix Something wasn't working, here's a fix environments

Projects

None yet

Development

Successfully merging this pull request may close these issues.

spack env activate does not traverse run dependencies for environment modifications

3 participants