Skip to content

Conversation

@vstavrinov
Copy link
Contributor

plugin was broken due to changes on the site (urls and some data). For example:

https://1plus1.video/tvguide/1plus1/online

Though it is geo restricted, but working via proxy

@vstavrinov
Copy link
Contributor Author

The test doesn't passed because it requires html.parser backport for python2 and slimit module for python3 to be installed.

@back-to
Copy link
Collaborator

back-to commented Apr 14, 2019

The way is to complicated, it can be done with less changes.

I will take a look later.

@vstavrinov
Copy link
Contributor Author

vstavrinov commented Apr 14, 2019

The way is to complicated, it can be done with less changes.

It is not about complicated or simple way. The matter is about reliable and systematic way. And concept is to use parser instead of regex.

In this particular precedent one among others reasons caused plugin getting broken was that 'src' attribute of 'iframe' tag moved to the next line. But this is one example only, while the common problem is that inessential, even least change in html or java script formatting may cause regex fail. Such occasional, unpredictable change may not touch content or logic itself of html or java script code and everything look like nothing changes, but requires deep investigation to discover the reason caused the fail.

Yes, there are downside using parser because hml is error tolerant and some errors missing by developers may cause parser fail. But first, such case has less chances to happen and second, some tools exists to fix html that again is regular solution. Though here is not relevant place for exhaustive list of pro and cos, one more is worth to mention. It is not always possible to get unambiguous result with regex and you need to analyze context. That is where parser comes in.

@back-to
Copy link
Collaborator

back-to commented Apr 14, 2019

if you don't want to update the regex, which is the fast way.

you can also use
from streamlink.plugin.api.utils import itertags and
from streamlink.utils.url import update_scheme


adding more dependencies is just pointless and Travis CI / AppVeyor must work

@back-to back-to added the plugin issue A Plugin does not work correctly label Apr 14, 2019
@vstavrinov
Copy link
Contributor Author

if you don't want to update the regex, which is the fast way.

"fast way" is not the first priority in such type of applications. You don't call regex every second. You start once and then watch tv for a while. In fact on the start most of the time is spent on server side and network transaction. At the same time reliable and reproducible operation is more important then "fast way"

you can also use
from streamlink.plugin.api.utils import itertags and

HTMLParser is standard python3 library so we don't need to install it. And writing and using custom replacement like itertags for standard library is not a good idea. Though writing a wrapper around HTMLParser more specific for this application - it may be better idea. But the issue remains about python2 backport. The joke is that HTMLParser is already in use in streamlink.compat - so You should add corresponding dependency independently of requirement of this plugin. Thus you can kill two birds with one stone. On the other hand python2 support will be dropped in some future, so this issue disappear by itself.

from streamlink.utils.url import update_scheme

Really it is not needed in this plugin and correspondent legacy code can be safely removed.

adding more dependencies is just pointless and Travis CI / AppVeyor must work

Using parsers is not "pointless" - it is wright way to do the task.

Finally what remains in our focus is slimit module. This is JavaScript parser. The reasons of using it is the same as for HTML described in the previous post. I believe it should be used and not only in this plugin.

@gravyboat
Copy link
Member

gravyboat commented Apr 15, 2019

@back-to Has already weighed in pretty decisively here, HTMLParser I don't mind so much as there's no extra bloat (still doesn't seem like much of a reason to use it but whatever) but I just want to add that slimit is comically out of date and basically unmaintained. Let's not add anything that isn't absolutely needed. We're trying to keep the application as fast as possible on as many devices as possible. It's already been an issue in the past for people trying to run Streamlink on weaker hardware.

@beardypig
Copy link
Member

I don't have any problem with using HTMLParser here, personally I wouldn't, but that's your choice.

Manipulating JavaScript seems to be cropping up again and again, and probably we should decide on a library that we want to use for that purpose.

@vstavrinov
Copy link
Contributor Author

What about this:

https://github.com/Kronuz/esprima-python ?

it's really better then slimit

@vstavrinov
Copy link
Contributor Author

It's already been an issue in the past for people trying to run Streamlink on weaker hardware.

I run streamlink on my ancient android device without problems. What is more weaker? raspberry ? I hope no. If You wait, You wait usually server and network, but not streamlink itself. Await time is very differ between sources.

@back-to
Copy link
Collaborator

back-to commented Apr 19, 2019

it's not about old devices,

it's about making Streamlink unnecessary harder to repack on some OS with more 3rd party dependencies

https://streamlink.github.io/install.html#linux-and-bsd

@vstavrinov
Copy link
Contributor Author

First, I answered to @gravyboat argument:

It's already been an issue in the past for people trying to run Streamlink on weaker hardware.

That is about running but not packaging that happens on different device.

Second, any dependencies cause no problem as long as they could be satisfied with pip either in pre-installation script or while packaging itself. In our case everything we need is:

pip2  install HTMLParser
pip2  install esprima
pip3  install esprima

@codecov
Copy link

codecov bot commented Aug 9, 2019

Codecov Report

Merging #2425 into master will decrease coverage by 0.04%.
The diff coverage is 45.2%.

@@            Coverage Diff             @@
##           master    #2425      +/-   ##
==========================================
- Coverage   52.77%   52.73%   -0.05%     
==========================================
  Files         240      240              
  Lines       15029    15057      +28     
==========================================
+ Hits         7932     7940       +8     
- Misses       7097     7117      +20

@back-to back-to merged commit 0f011ae into streamlink:master Aug 9, 2019
@vstavrinov vstavrinov deleted the ovvatv branch August 20, 2019 09:40
Billy2011 pushed a commit to Billy2011/streamlink-27 that referenced this pull request May 14, 2020
mkbloke pushed a commit to mkbloke/streamlink that referenced this pull request Aug 18, 2020
resiproxy pushed a commit to resiproxy/streamlink that referenced this pull request Nov 5, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

plugin issue A Plugin does not work correctly

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants