Skip to content

Install lint deps at user level#10051

Closed
feerrenrut wants to merge 1 commit intomasterfrom
flake8InstallNoAdmin
Closed

Install lint deps at user level#10051
feerrenrut wants to merge 1 commit intomasterfrom
flake8InstallNoAdmin

Conversation

@feerrenrut
Copy link
Copy Markdown
Contributor

@feerrenrut feerrenrut commented Aug 8, 2019

For users without an admin account, deps must be installed at user level.
This does not harm our project, so prefer this.

Note To test this, remove tests/lint/lintInstall/_executed_requirements.txt between installs, this is used as a mechanism to ensure pip isn't run when requirements haven't changed.

Link to issue number:

None

Summary of the issue:

Some reports of the dependency install step failing from developers who have a non admin windows account.

Description of how this pull request fixes the issue:

Install the dependencies at user level instead.

Testing performed:

  • the developer test this on their machine, the deps installed correctly and lint was able to be run
  • tested this on my machine with deps already installed at system level, no errors and lint was able to be run

Known issues with pull request:

None

Change log entry:

None.

For users without an admin account, deps must be installed at user level.
This does not harm our project, so prefer this.
@feerrenrut feerrenrut requested a review from jcsteh August 8, 2019 11:48
Copy link
Copy Markdown
Contributor

@jcsteh jcsteh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What this ends up doing is installing the Python dependencies to a directory called "~" in the nvda checkout. That does work, but it's certainly strange. If you remove that directory (which you might well do because git shows it as untracked), scons lint is broken because it can't find the deps.

I think this is probably happening because our scons environment is missing a lot of variables. I'm guessing pip internally uses os.path.expanduser, which depends on either the USERPROFILE or HOMEDRIVE + HOMEPATH environment variables. So we somehow need to copy these across from the real environment.

@feerrenrut
Copy link
Copy Markdown
Contributor Author

installing the Python dependencies to a directory called "~" in the nvda checkout.

Oh yes, I forgot you mentioned that.

So we somehow need to copy these across from the real environment.

I'll take a look at that.

@lukaszgo1
Copy link
Copy Markdown
Contributor

I'm afraid it would fail in the following situation:

  1. Developer has standard user account on which he does most of his work, so he installs dependencies at the user level for that account.
  2. For a particular issue he has to work on an admin account so he writes code there and when he is ready he tries to lint his changes still under this account.
  3. The process fails because Flake8 is not installed however installation isn't triggered, because the _executed_requirements.txt exist.
  4. He tries to install flake8 with scons lintInstall and installation isn't performed.

Would it be possible instead of copying requirements file and using it to determine if dependencies are installed simply attempt importing flake8 and flake8_tabs and if import fails perform installation?

@jcsteh
Copy link
Copy Markdown
Contributor

jcsteh commented Aug 9, 2019 via email

@feerrenrut
Copy link
Copy Markdown
Contributor Author

attempt importing flake8 and flake8_tabs and if import fails perform installation

Couple of downsides to this:

  • We have to maintain a second list of things to test for importing, matching our requirements file.
  • We also want to check for versions, currently the lintInstall step runs again when the requirements file changes.

I have had a quick look at importing environment vars to try to get this into the right place, short of importing the full external env, I think this will be error prone. I am now in a situation on my local system where flake8 is installed to some random folder in my user directory, not the standard --user location. I think i need to use SCons to uninstall 🙄

Importing the full env is not really ideal either, this can easily cause strange outcomes on different systems.

I think the best eventual option is to use virtual environments to handle python deps. Tox automatically sets up virtual envs and runs tests in the chosen environment.

@lukaszgo1
Copy link
Copy Markdown
Contributor

We have to maintain a second list of things to test for importing, matching our requirements file.

It should be possible to parse requirements file in the sconscript, import what is included there and even check for versions. I have no idea if it would be easier than going with virtual environments.

We also want to check for versions, currently the lintInstall step runs again when the requirements file changes.

If the file would be properly parsed with a version check it would also be done.

@lukaszgo1
Copy link
Copy Markdown
Contributor

Editing the same source copy with multiple user accounts seems like a recipe for permission problems. It's also somewhat contrary to how UAC is meant to work.

As long as source code isn't stored in the user directory I don't thing it should be problematic.

Why not just use an admin account and elevate only when needed?

This PR was created precisely because some developers are using standard accounts. It is apparently more secure.

@LeonarddeR
Copy link
Copy Markdown
Collaborator

@lukaszgo1 commented on 9 aug. 2019 14:36 CEST:

We have to maintain a second list of things to test for importing, matching our requirements file.

It should be possible to parse requirements file in the sconscript, import what is included there and even check for versions.

I think this sounds interesting. pip is able to parse these requirement files, so I assume it has some functionality to get the appropriate package names from them.

@lukaszgo1
Copy link
Copy Markdown
Contributor

It looks like calling second python interpreter with subprocess.call installs these dependencies to the right place. It certainly isn't as pretty as a virtual environment, but is a lot simpler.

@XLTechie
Copy link
Copy Markdown
Collaborator

XLTechie commented Aug 9, 2019 via email

@feerrenrut
Copy link
Copy Markdown
Contributor Author

I don't think this is a good approach, I'm closing this PR.

@feerrenrut feerrenrut closed this Aug 22, 2019
@feerrenrut feerrenrut deleted the flake8InstallNoAdmin branch January 17, 2020 08:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants