-
-
Notifications
You must be signed in to change notification settings - Fork 217
Use the requests package for all http requests #843
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
now a requests.Response is returned instead of the file-like object from urllib. Fixed all usages of util.urlopen: it simplifies getting json, text encoding detection. In particular feedcore (responsible for fetching feeds) is simplified. This is a first pass and could benefit from better usage of the requests api (Sessions for instance, to keep connection pools) TODO: download.py
|
src/gpodder/feedcore.py:192:56: E226 missing whitespace around arithmetic operator I applied these patches and they seem to work with limited testing, I'll let you know if they break in the next few days. Do you know the requests package has at least 6 dependencies? This one change doubles the minimum dependencies needed to run gpodder. |
|
Thanks for your testing and feedback: it's good to know nothing is broken (yet)! Yes, it adds dependencies but urllib is lagging in features for the core thing in gPodder (http). requests is a thin wrapper around urllib3. It does the right thing for:
It depends on:
Optional dependencies:
I'm not concerned about those dependencies because urllib3 is already used in recent versions of pip, so it's likely already installed on Linux and the size of python deps is dwarfed by the whole gtk3 stack in macOS/Windows packages. The next area I'll port to requests is download.py. |
|
I know this is needed. Gentoo's urllib3 also requires PySocks, pyopenssl and cryptography, is that not the case for the windows/mac builds? |
|
Example internationalized domain name: https://日本語.jp/ |
AFAICT pyopenssl and cryptography are only needed to enable SNI verification in python2 (https://github.com/urllib3/urllib3/blob/eee53a69e1af019da18635d6974f893308db0ada/src/urllib3/contrib/pyopenssl.py and https://github.com/psf/requests/blob/251f73f65d46ff5de64b8ae2e6c6fe38c1a28808/requests/__init__.py#L96). Pysocks should indeed be installed for SOCKS proxy support |
|
I overlooked requests's encoding detection: it doesn't use chardet automatically (one has to call response.apparent_encoding to get it) and falls back to ISO-8859-1 if content-type is So all my happy changes from shady decode to response.text must be reconsidered. |
|
Fixed for feedcore in d59bba5, but escapist.py, youtube.py, etc. need to be fixed |
bytes for xml, utf-8 fallback for html, known encoding if available
|
Downloading youtube episodes works fine, but subscribing fails. |
|
you're right, thanks! |
|
@auouymous I have further work to push (unittests, fix lint, retry rss download) but it's not quite ready yet... |
|
@elelay Okay, all patches are working fine so far. |
main goal is for feedcore but I don't see how it would hurt to retry in other (cover download, etc.) cases.
|
The isort -rc removal causes the following warnings for me. |
|
Hi, please try with isort >= 5.0.0. |
|
Gentoo doesn't have 5.0+ yet and the ebuild for 4.3.21 won't successfully build the 5.0 branch. |
|
@auouymous done, thanks |
No description provided.