Skip to content

bugfix: spack shouldn't fail in an incomplete environment#16473

Merged
tgamblin merged 1 commit intodevelopfrom
bugfix/no-fail-on-no-link
May 7, 2020
Merged

bugfix: spack shouldn't fail in an incomplete environment#16473
tgamblin merged 1 commit intodevelopfrom
bugfix/no-fail-on-no-link

Conversation

@tgamblin
Copy link
Copy Markdown
Member

@tgamblin tgamblin commented May 5, 2020

Fixed #15884.

Spack asks every package linked into an environment to tell us how environment variables should be modified when a spack environment is activated. As part of this, specs in an environment are symlinked into the environment's view (see #13249), and the package calculates environment modifications with the default view as the prefix.

All of this works nicely for pointing the user's environment at the view if every package is successfully linked. Unfortunately, right now we only track what specs "should" be in a view, not which specs actually are. So we end up calculating environment modifications on things that aren't linked into thee view, and the exception isn't caught, so lots of spack commands end up failing.

This fixes the issue by ignoring and warning about specs where calculating environment modifications fails. So we can still keep using Spack even if the current environment is incomplete.

We should probably also just avoid computing env modifications entirely for unlinked packages, but right now that is a slow operation (requires a lot of YAML parsing). We should revisit that when we have some better state management for views, but the fix adopted here will still be necessary, as we want spack commands to be resilient to other types of bugs in setup_run_environment() and friends. That code is in packages and we have to assume it could be buggy when we call it outside of builds (as it might fail more than just the build).

  • add try/catch around calculation of each package's env modifications
  • tests

@tgamblin tgamblin requested review from becker33 and scheibelp May 5, 2020 18:08
@tgamblin
Copy link
Copy Markdown
Member Author

tgamblin commented May 5, 2020

@mdorier @hartzell @ax3l: does this fix your issues?

@tgamblin tgamblin added bugfix Something wasn't working, here's a fix environments labels May 5, 2020
Copy link
Copy Markdown
Member

@becker33 becker33 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I probably would have just issued the warnings as they occur, but I don't have a problem with batching them in the return statement either.

@tgamblin
Copy link
Copy Markdown
Member Author

tgamblin commented May 5, 2020

The issue that is addressing is that both add and rm are called on deactivate, so you get the messages twice. Note that only add prints the warnings.

@tgamblin tgamblin force-pushed the bugfix/no-fail-on-no-link branch from 39f3c5d to 1894ba3 Compare May 7, 2020 08:43
Fixed #15884.

Spack asks every package linked into an environment to tell us how
environment variables should be modified when a spack environment is
activated. As part of this, specs in an environment are symlinked into
the environment's view (see #13249), and the package calculates
environment modifications with *the default view as the prefix*.

All of this works nicely for pointing the user's environment at the view
*if* every package is successfully linked. Unfortunately, right now we
only track what specs "should" be in a view, not which specs actually
are. So we end up calculating environment modifications on things that
aren't linked into thee view, and the exception isn't caught, so lots of
spack commands end up failing.

This fixes the issue by ignoring and warning about specs where
calculating environment modifications fails. So we can still keep using
Spack even if the current environment is incomplete.

We should probably also just avoid computing env modifications *entirely*
for unlinked packages, but right now that is a slow operation (requires a
lot of YAML parsing). We should revisit that when we have some better
state management for views, but the fix adopted here will still be
necessary, as we want spack commands to be resilient to other types of
bugs in `setup_run_environment()` and friends. That code is in packages
and we have to assume it could be buggy when we call it outside of builds
(as it might fail more than just the build).
@tgamblin tgamblin force-pushed the bugfix/no-fail-on-no-link branch from 1894ba3 to 3446fe0 Compare May 7, 2020 08:46
@tgamblin tgamblin merged commit b196e43 into develop May 7, 2020
@alalazo alalazo deleted the bugfix/no-fail-on-no-link branch May 7, 2020 10:50
@ax3l
Copy link
Copy Markdown
Member

ax3l commented Jun 10, 2020

Late confirmation: fixes the reported issue for me, thanks a lot!

becker33 pushed a commit that referenced this pull request Jul 11, 2020
Fixed #15884.

Spack asks every package linked into an environment to tell us how
environment variables should be modified when a spack environment is
activated. As part of this, specs in an environment are symlinked into
the environment's view (see #13249), and the package calculates
environment modifications with *the default view as the prefix*.

All of this works nicely for pointing the user's environment at the view
*if* every package is successfully linked. Unfortunately, right now we
only track what specs "should" be in a view, not which specs actually
are. So we end up calculating environment modifications on things that
aren't linked into thee view, and the exception isn't caught, so lots of
spack commands end up failing.

This fixes the issue by ignoring and warning about specs where
calculating environment modifications fails. So we can still keep using
Spack even if the current environment is incomplete.

We should probably also just avoid computing env modifications *entirely*
for unlinked packages, but right now that is a slow operation (requires a
lot of YAML parsing). We should revisit that when we have some better
state management for views, but the fix adopted here will still be
necessary, as we want spack commands to be resilient to other types of
bugs in `setup_run_environment()` and friends. That code is in packages
and we have to assume it could be buggy when we call it outside of builds
(as it might fail more than just the build).
vjranagit pushed a commit to vjranagit/spack that referenced this pull request Jan 18, 2026
Fixed spack#15884.

Spack asks every package linked into an environment to tell us how
environment variables should be modified when a spack environment is
activated. As part of this, specs in an environment are symlinked into
the environment's view (see spack#13249), and the package calculates
environment modifications with *the default view as the prefix*.

All of this works nicely for pointing the user's environment at the view
*if* every package is successfully linked. Unfortunately, right now we
only track what specs "should" be in a view, not which specs actually
are. So we end up calculating environment modifications on things that
aren't linked into thee view, and the exception isn't caught, so lots of
spack commands end up failing.

This fixes the issue by ignoring and warning about specs where
calculating environment modifications fails. So we can still keep using
Spack even if the current environment is incomplete.

We should probably also just avoid computing env modifications *entirely*
for unlinked packages, but right now that is a slow operation (requires a
lot of YAML parsing). We should revisit that when we have some better
state management for views, but the fix adopted here will still be
necessary, as we want spack commands to be resilient to other types of
bugs in `setup_run_environment()` and friends. That code is in packages
and we have to assume it could be buggy when we call it outside of builds
(as it might fail more than just the build).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugfix Something wasn't working, here's a fix environments

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Spack breaking when installing Python in an environment

3 participants