Environment
- pip version: 19.2.3+
- Python version: python 3.7
- OS: Windows 10
- AntiVirus: CylancePROTECT
Description
pip install or download fails on Windows during cleanup because (real-time) virus protection is still scanning files that pip is trying to remove.
using Process Monitor (procmon), I was able to confirm that CylanceSvc.exe (virus scanner) was reading the files while python was trying to the same files.
I tried both increasing the retry wait (in misc.py) and ignoring the rmtree error (in temp_dir.py). both approaches worked.
This issue indicates that other anti-virus scanners cause similar problems:
aws/aws-cli#2654
Expected behavior
The package should install.
How to Reproduce
pip install or download a large package. I was usually able to reproduce when installing or downloading awscli. If I have difficulty reproducing, I can lower the priority of the Cylance.exe to Background (4), and the symptoms are easier to reproduce consistently.
using install:
pip install awscli
using download:
pip download awscli
pip download is better because it provides a stacktrace.
Output
using pip install, the error message is quite terse.
>pip install awscli
Collecting awscli
Using cached https://files.pythonhosted.org/packages/f2/f5/d682aa32100edc6908a784630aa2903ef7d2735130e6df98a05af6c33096/awscli-1.16.271-py2.py3-none-any.whl
ERROR: Could not install packages due to an EnvironmentError: [WinError 5] Access is denied: 'c:\\tmp\\pip\\pip-unpack-5e4a1ye3\\awscli-1.16.271-py2.py3-none-any.whl'
Consider using the `--user` option or check the permissions.
. . .
using pip download, the stacktrace is more helpful.
>pip download awscli
Collecting awscli
Using cached https://files.pythonhosted.org/packages/f2/f5/d682aa32100edc6908a784630aa2903ef7d2735130e6df98a05af6c33096/awscli-1.16.271-py2.py3-none-any.whl
Saved c:\users\rich9002\tmp\awscli-1.16.271-py2.py3-none-any.whl
ERROR: Exception:
Traceback (most recent call last):
File "c:\sw\python37\lib\shutil.py", line 398, in _rmtree_unsafe
os.unlink(fullname)
PermissionError: [WinError 5] Access is denied: 'c:\\tmp\\pip\\pip-unpack-4gdrskd3\\awscli-1.16.271-py2.py3-none-any.whl'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "c:\sw\python37\lib\site-packages\pip\_internal\cli\base_command.py", line 188, in main
status = self.run(options, args)
File "c:\sw\python37\lib\site-packages\pip\_internal\commands\download.py", line 156, in run
resolver.resolve(requirement_set)
File "c:\sw\python37\lib\site-packages\pip\_internal\legacy_resolve.py", line 196, in resolve
self._resolve_one(requirement_set, req)
File "c:\sw\python37\lib\site-packages\pip\_internal\legacy_resolve.py", line 359, in _resolve_one
abstract_dist = self._get_abstract_dist_for(req_to_install)
File "c:\sw\python37\lib\site-packages\pip\_internal\legacy_resolve.py", line 307, in _get_abstract_dist_for
self.require_hashes
File "c:\sw\python37\lib\site-packages\pip\_internal\operations\prepare.py", line 199, in prepare_linked_requirement
progress_bar=self.progress_bar
File "c:\sw\python37\lib\site-packages\pip\_internal\download.py", line 1064, in unpack_url
progress_bar=progress_bar
File "c:\sw\python37\lib\site-packages\pip\_internal\download.py", line 935, in unpack_http_url
os.unlink(from_path)
File "c:\sw\python37\lib\site-packages\pip\_internal\utils\temp_dir.py", line 60, in __exit__
self.cleanup()
File "c:\sw\python37\lib\site-packages\pip\_internal\utils\temp_dir.py", line 83, in cleanup
rmtree(self.path)
File "c:\sw\python37\lib\site-packages\pip\_vendor\retrying.py", line 49, in wrapped_f
return Retrying(*dargs, **dkw).call(f, *args, **kw)
File "c:\sw\python37\lib\site-packages\pip\_vendor\retrying.py", line 212, in call
raise attempt.get()
File "c:\sw\python37\lib\site-packages\pip\_vendor\retrying.py", line 247, in get
six.reraise(self.value[0], self.value[1], self.value[2])
File "c:\sw\python37\lib\site-packages\pip\_vendor\six.py", line 693, in reraise
raise value
File "c:\sw\python37\lib\site-packages\pip\_vendor\retrying.py", line 200, in call
attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
File "c:\sw\python37\lib\site-packages\pip\_internal\utils\misc.py", line 166, in rmtree
onerror=rmtree_errorhandler)
File "c:\sw\python37\lib\shutil.py", line 516, in rmtree
return _rmtree_unsafe(path, onerror)
File "c:\sw\python37\lib\shutil.py", line 400, in _rmtree_unsafe
onerror(os.unlink, fullname, sys.exc_info())
File "c:\sw\python37\lib\site-packages\pip\_internal\utils\misc.py", line 176, in rmtree_errorhandler
os.chmod(path, stat.S_IWRITE)
PermissionError: [WinError 5] Access is denied: 'c:\\tmp\\pip\\pip-unpack-4gdrskd3\\awscli-1.16.271-py2.py3-none-any.whl'
Environment
Description
pip installordownloadfails on Windows during cleanup because (real-time) virus protection is still scanning files that pip is trying to remove.using Process Monitor (procmon), I was able to confirm that
CylanceSvc.exe(virus scanner) was reading the files while python was trying to the same files.I tried both increasing the retry wait (in
misc.py) and ignoring the rmtree error (in temp_dir.py). both approaches worked.This issue indicates that other anti-virus scanners cause similar problems:
aws/aws-cli#2654
Expected behavior
The package should install.
How to Reproduce
pip installordownloada large package. I was usually able to reproduce when installing or downloadingawscli. If I have difficulty reproducing, I can lower the priority of theCylance.exeto Background (4), and the symptoms are easier to reproduce consistently.using install:
pip install awscliusing download:
pip download awsclipip downloadis better because it provides a stacktrace.Output
using
pip install, the error message is quite terse.using
pip download, the stacktrace is more helpful.