Many HTTP 520 responses from rarbg #3705
Description
Expected behaviour:
rarbg plugin reliably returns search results.
Actual behaviour:
There seem to be two separate issues:
- Occasionally the timeout between requests is too short (<2s), I believe their request rate is/was 1 req/2s.
- Even when timeout is longer than 2s, rarbg returns 520 which is their way of saying there are too many requests.
Initially I thought that my VPN exit node might be too "popular" but one of the google searches lead me to Sonarr/Sonarr#5206 which outlines much longer timeouts between requests for rarbg.
Log:
(click to expand)
2023-03-02 18:30:16 DEBUG utils.requests download-tv-seasons-4k GETing URL https://torrentapi.org/pubapi_v2.php with args () and kwargs {'params': {'mode': 'search', 'ranked': 1, 'min_seeders': 0, 'min_leechers': 0, 'sort': 'seeders', 'category': '49', 'format': 'json_extended', 'app_id': 'flexget', 'search_string': b'Series 1 (2019) S04', 'token': 'removed'}, 'allow_redirects': True, 'timeout': 30}
2023-03-02 18:30:17 DEBUG rarbg download-tv-seasons-4k requesting: https://torrentapi.org/pubapi_v2.php?mode=search&ranked=1&min_seeders=0&min_leechers=0&sort=seeders&category=49&format=json_extended&app_id=flexget&search_string=removed&token=removed
2023-03-02 18:30:17 DEBUG rarbg download-tv-seasons-4k No results found for b'Series 1 (2019) S04'. Message from rarbg: No results found
2023-03-02 18:30:17 DEBUG utils.requests download-tv-seasons-4k Waiting 1.74 seconds until next request to torrentapi.org
2023-03-02 18:30:19 DEBUG utils.requests download-tv-seasons-4k GETing URL https://torrentapi.org/pubapi_v2.php with args () and kwargs {'params': {'mode': 'search', 'ranked': 1, 'min_seeders': 0, 'min_leechers': 0, 'sort': 'seeders', 'category': '49', 'format': 'json_extended', 'app_id': 'flexget', 'token': 'removed', 'search_string': b'removed'}, 'allow_redirects': True, 'timeout': 30}
2023-03-02 18:30:19 ERROR rarbg download-tv-seasons-4k Rarbg request failed: 520 Server Error: for url: https://torrentapi.org/pubapi_v2.php?mode=search&ranked=1&min_seeders=0&min_leechers=0&sort=seeders&category=49&format=json_extended&app_id=flexget&token=removed&search_string=removed
2023-03-02 18:30:19 DEBUG discover download-tv-seasons-4k No results from rarbg
2023-03-02 18:30:19 VERBOSE discover download-tv-seasons-4k No search results for `Series 1 (2019) S04`
2023-03-02 18:30:19 DEBUG series.db download-tv-seasons-4k latest season pack for series <Series(id=114,name=Series 1 (2019))>, with downloaded set to True and season set to None
2023-03-02 18:30:19 DEBUG series.db download-tv-seasons-4k latest episode for series <Series(id=114,name=Series 1 (2019))>, with downloaded set to True and season set to 4
2023-03-02 18:30:19 VERBOSE discover download-tv-seasons-4k Searching for `Series 2 (2021) S02` with plugin `rarbg` (2 of 21)
2023-03-02 18:30:19 DEBUG utils.requests download-tv-seasons-4k Waiting 2.76 seconds until next request to torrentapi.org
2023-03-02 18:30:22 DEBUG utils.requests download-tv-seasons-4k GETing URL https://torrentapi.org/pubapi_v2.php with args () and kwargs {'params': {'mode': 'search', 'ranked': 1, 'min_seeders': 0, 'min_leechers': 0, 'sort': 'seeders', 'category': '49', 'format': 'json_extended', 'app_id': 'flexget', 'search_string': b'Series 2 (2021) S02', 'token': 'removed'}, 'allow_redirects': True, 'timeout': 30}
2023-03-02 18:30:22 ERROR rarbg download-tv-seasons-4k Rarbg request failed: 520 Server Error: for url: https://torrentapi.org/pubapi_v2.php?mode=search&ranked=1&min_seeders=0&min_leechers=0&sort=seeders&category=49&format=json_extended&app_id=flexget&search_string=removed&token=removed
Additional information:
- FlexGet version: 3.5.27