Skip to content

429 Too Many Requests #367

@dcroote

Description

@dcroote

We've seen HTTP status client error (429 Too Many Requests) for url for github.com urls intermittently even when using the GITHUB_TOKEN env and reducing concurrency to 10 threads. A few examples of Action runs where this occurred: 1, 2, 3, 4 and here is the workflow itself. Note the workflow uses a cached installation of lychee installed via cargo from a specific commit sha rather than the v0.7.1 marketplace Action because of additional functionality that's been added since that release.

I think the ideal solution would be as described in #348 to avoid hitting the same url repeatedly from different files (which is indeed the case with the urls causing failures in the Actions runs above), but I can see this is complex (#349).

Another theoretical idea could be running lychee twice, once with high concurrency using --exclude github.com and again with minimal concurrency and --include github.com.

A third might be identifying github.com urls and applying different internal handling of how they're requested?

Would love to get your thoughts and also confirm it isn't a usage issue.

Overall though great tool, it's helpful to check both internal and external urls + images / img tags quickly

Metadata

Metadata

Assignees

Labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions