Skip to content

Fix use of sys.executable for module/env commands#14496

Merged
adamjstewart merged 4 commits intospack:developfrom
adamjstewart:fixes/sys-executable
Jan 16, 2020
Merged

Fix use of sys.executable for module/env commands#14496
adamjstewart merged 4 commits intospack:developfrom
adamjstewart:fixes/sys-executable

Conversation

@adamjstewart
Copy link
Copy Markdown
Member

@zzotta can you see if this fixes #14491 for you?

I'm not sure whether the quotes are necessary or not. I wanted to be robust in case python is installed in a directory containing spaces. Honestly, we should probably rewrite all of this stuff so that it doesn't use shell=True, and then it shouldn't be necessary anymore.

@s-sajid-ali @chissg

@adamjstewart adamjstewart added python bugfix Something wasn't working, here's a fix labels Jan 14, 2020
@zzotta
Copy link
Copy Markdown
Contributor

zzotta commented Jan 15, 2020

@zzotta can you see if this fixes #14491 for you?

Works for me.

Copy link
Copy Markdown
Member

@greenc-FNAL greenc-FNAL left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Subject to #14252 (review) which I believe is still valid, I think this is good.

@adamjstewart
Copy link
Copy Markdown
Member Author

@chissg I completely agree, we should be running this without shell=True. I think I could do this pretty easily in util/environment.py, but util/module_cmd.py will be much harder, as it involves running multiple commands in the same shell and piping some of the output to a different location. If you want to take a stab at this in a separate PR, please do, but I don't feel comfortable enough with subprocess to do it myself.

@adamjstewart adamjstewart merged commit 808c80d into spack:develop Jan 16, 2020
@adamjstewart adamjstewart deleted the fixes/sys-executable branch January 16, 2020 21:46
alalazo added a commit to alalazo/spack that referenced this pull request Jan 20, 2020
tgamblin pushed a commit that referenced this pull request Jan 25, 2020
Using `sys.executable` to run Python in a sub-shell doesn't always work in a virtual environment as the `sys.executable` Python is not necessarily compatible with any loaded spack/other virtual environment.

- revert use of sys.executable to print out subshell environment (#14496)
- try instead to use an available python, then if there *is not* one, use `sys.executable`
- this addresses RHEL8 (where there is no `python` and `PYTHONHOME` issue in a simpler way
tgamblin pushed a commit that referenced this pull request Feb 7, 2020
* Fix use of sys.executable for module/env commands

* Fix unit tests

* More consistent quotation, less duplication

* Fix import syntax
tgamblin pushed a commit that referenced this pull request Feb 7, 2020
Using `sys.executable` to run Python in a sub-shell doesn't always work in a virtual environment as the `sys.executable` Python is not necessarily compatible with any loaded spack/other virtual environment.

- revert use of sys.executable to print out subshell environment (#14496)
- try instead to use an available python, then if there *is not* one, use `sys.executable`
- this addresses RHEL8 (where there is no `python` and `PYTHONHOME` issue in a simpler way
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugfix Something wasn't working, here's a fix python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

EnvironmentModifications does not work if the package depends on a specific python package

4 participants