Skip to content

Add some delays when querying NPO.nl#1739

Merged
liiight merged 1 commit intoFlexget:developfrom
mfonville:requests
Mar 17, 2017
Merged

Add some delays when querying NPO.nl#1739
liiight merged 1 commit intoFlexget:developfrom
mfonville:requests

Conversation

@mfonville
Copy link
Copy Markdown
Contributor

@mfonville mfonville commented Mar 17, 2017

Motivation for changes:

NPO.nl blocks if too many requests directly after one another, probably to prevent DDoS attacks.
This spawned errors during the execution of npo_watchlist if having several favorite series

Detailed changes:

  • Added some fixed 5 second delays before requests to NPO.nl

Addressed issues:

Prevents that Flexget randomly gives

  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 501, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/flexget/utils/requests.py", line 240, in request
    result = super(Session, self).request(method, url, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 488, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 609, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 473, in send
    raise ConnectionError(err, request=request)
ConnectionError: ('Connection aborted.', BadStatusLine("''",))

during the execution of the npo_watchlist plugin

@liiight
Copy link
Copy Markdown
Member

liiight commented Mar 17, 2017

We have a specific implementation for that: https://github.com/Flexget/Flexget/blob/develop/flexget/plugins/notifiers/join.py#L17

Also, why two prs for the same plugin? There's no reason not to do them all in one pr

@mfonville
Copy link
Copy Markdown
Contributor Author

mfonville commented Mar 17, 2017 •
edited
Loading

The 2 PRs because two separate issues and I was not sure if this one was done the right way (and apparently I did not); I will look at adding that specific trick

@mfonville
Copy link
Copy Markdown
Contributor Author

mfonville commented Mar 17, 2017

I updated now this PR with the suggested manner of @liiight to tackle this issue

@liiight liiight merged commit 16bc317 into Flexget:develop Mar 17, 2017
@mfonville mfonville deleted the requests branch March 18, 2017 08:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@mfonville @liiight