-
-
Notifications
You must be signed in to change notification settings - Fork 197
Description
We've seen HTTP status client error (429 Too Many Requests) for url for github.com urls intermittently even when using the GITHUB_TOKEN env and reducing concurrency to 10 threads. A few examples of Action runs where this occurred: 1, 2, 3, 4 and here is the workflow itself. Note the workflow uses a cached installation of lychee installed via cargo from a specific commit sha rather than the v0.7.1 marketplace Action because of additional functionality that's been added since that release.
I think the ideal solution would be as described in #348 to avoid hitting the same url repeatedly from different files (which is indeed the case with the urls causing failures in the Actions runs above), but I can see this is complex (#349).
Another theoretical idea could be running lychee twice, once with high concurrency using --exclude github.com and again with minimal concurrency and --include github.com.
A third might be identifying github.com urls and applying different internal handling of how they're requested?
Would love to get your thoughts and also confirm it isn't a usage issue.
Overall though great tool, it's helpful to check both internal and external urls + images / img tags quickly