Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.
Sign upbpo-35907, CVE-2019-9948: urllib rejects local_file:// scheme #13474
Conversation
This comment has been minimized.
This comment has been minimized.
|
This PR is very close to PR #11842, I just added the unit test:
|
This comment has been minimized.
This comment has been minimized.
|
@tiran: Would you mind to review this forward port of PR #11842? Fix for https://bugs.python.org/issue35907 |
This comment has been minimized.
This comment has been minimized.
|
@tirkarthi @gpshead: Would you mind to review this HTTP security fix? |
| for url in ('local_file://example', 'local-file://example'): | ||
| self.assertRaises(IOError, urllib.request.urlopen, url) | ||
| self.assertRaises(IOError, urllib.request.URLopener().open, url) | ||
| self.assertRaises(IOError, DummyURLopener().open, url) |
This comment has been minimized.
This comment has been minimized.
tirkarthi
May 22, 2019
Contributor
Are we making this change to the deprecated legacy interface urllib.request.URLopener().retrieve too? If so I would prefer adding below test since it also now throws IOError that used to return a result. I guess Python 2 also has urlretrieve which now forbids local_file
Note : urlretrieve was untested but documented that it used to throw NameError (https://bugs.python.org/issue36948) . Please also check Windows compatibility if urlretrieve tests have to be added since I mistakenly broke two Windows buildbots in the issue.
self.assertRaises(urllib.request.URLopener().rertrieve, url)
This comment has been minimized.
This comment has been minimized.
tirkarthi
May 22, 2019
Contributor
Sorry, my test was wrong. urlretrieve requires a file. Maybe above urlretrieve tests can be used to just check for local-file:// too? Change in behavior as below
Before PR :
cpython git:(master) ✗ touch /tmp/a.txt
cpython git:(master) ✗ ./python.exe -q
>>> import urllib.request
>>> urllib.request.URLopener().retrieve('local_file://localhost/tmp/a.txt')
sys:1: DeprecationWarning: URLopener style of invoking requests is deprecated. Use newer urlopen functions/methods
('/var/folders/2b/mhgtnnpx4z943t4cc9yvw4qw0000gn/T/tmpmx6km86p.txt', <email.message.Message object at 0x1066a2a00>)
After PR :
cpython git:(master) ✗ touch /tmp/a.txt
cpython git:(master) ✗ ./python.exe -q
>>> import urllib.request
>>> urllib.request.URLopener().retrieve('local_file://localhost/tmp/a.txt')
sys:1: DeprecationWarning: URLopener style of invoking requests is deprecated. Use newer urlopen functions/methods
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/urllib/request.py", line 1789, in retrieve
fp = self.open(url, data)
File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/urllib/request.py", line 1752, in open
return self.open_unknown(fullurl, data)
File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/urllib/request.py", line 1766, in open_unknown
raise OSError('url error', 'unknown url type', type)
OSError: [Errno url error] unknown url type: 'local_file'
This comment has been minimized.
This comment has been minimized.
vstinner
May 22, 2019
Author
Member
I didn't know URLopener().retrieve :-) My PR impacts it as well.
I added requested tests.
CVE-2019-9948: Avoid file reading as disallowing the unnecessary URL scheme in URLopener().open() and URLopener().retrieve() of urllib.request. Co-Authored-By: SH <push0ebp@gmail.com>
This comment has been minimized.
This comment has been minimized.
|
I rebased my PR to modify the commit message: mention also |
|
LGTM. Thanks :) |
This comment has been minimized.
This comment has been minimized.
miss-islington
commented
May 22, 2019
|
Thanks @vstinner for the PR |
This comment has been minimized.
This comment has been minimized.
miss-islington
commented
May 22, 2019
|
Sorry, @vstinner, I could not cleanly backport this to |
This comment has been minimized.
This comment has been minimized.
bedevere-bot
commented
May 22, 2019
|
GH-13505 is a backport of this pull request to the 3.7 branch. |
…GH-13474) (pythonGH-13505) CVE-2019-9948: Avoid file reading as disallowing the unnecessary URL scheme in URLopener().open() and URLopener().retrieve() of urllib.request. Co-Authored-By: SH <push0ebp@gmail.com> (cherry picked from commit 0c2b6a3) (cherry picked from commit 34bab21)
…GH-13474) (pythonGH-13505) CVE-2019-9948: Avoid file reading as disallowing the unnecessary URL scheme in URLopener().open() and URLopener().retrieve() of urllib.request. Co-Authored-By: SH <push0ebp@gmail.com> (cherry picked from commit 0c2b6a3) (cherry picked from commit 34bab21)
| @@ -0,0 +1,2 @@ | |||
| CVE-2019-9948: Avoid file reading as disallowing the unnecessary URL scheme in | |||
This comment has been minimized.
This comment has been minimized.
gpshead
May 22, 2019
•
Member
This text doesn't make sense as written. "as disallowing the unnecessary URL scheme" says nothing. WHAT url scheme did we just break our users applications by refusing to support? Those need to be listed in the NEWS entry.
CVE-2019-9948: Drop support for the unintended local_file:// URL scheme in ...
…GH-13474) (pythonGH-13505) CVE-2019-9948: Avoid file reading by disallowing local-file:// and local_file:// URL schemes in URLopener().open() and URLopener().retrieve() of urllib.request. Co-Authored-By: SH <push0ebp@gmail.com> (cherry picked from commit 0c2b6a3) (cherry picked from commit 34bab21)
…GH-13474) (pythonGH-13505) CVE-2019-9948: Avoid file reading by disallowing local-file:// and local_file:// URL schemes in URLopener().open() and URLopener().retrieve() of urllib.request. Co-Authored-By: SH <push0ebp@gmail.com> (cherry picked from commit 0c2b6a3) (cherry picked from commit 34bab21)
vstinner commentedMay 21, 2019
•
edited
CVE-2019-9948: Avoid file reading as disallowing the unnecessary URL
scheme in URLopener().open() of urllib.request.
Co-Authored-By: SH push0ebp@gmail.com
https://bugs.python.org/issue35907