Skip to content

Don't hardcode not reporting 503, 504, and 999 errors #77

@SeanTAllen

Description

@SeanTAllen

In bad_url is:

    def bad_url(url_status: int) -> bool:
        if url_status == -1:
            return True
        elif url_status == 401 or url_status == 403:
            return False
        elif url_status in (503, 504):
            # Usually transient
            return False
        elif url_status == 999:
            # Returned by some websites (e.g. LinkedIn) that think you're crawling them.
            return False
        elif url_status >= 400:
            return True
        return False

There's 3 errors here that are errors that have been hardcoded to not be errors. I noticed this because we have URLs that don't work. That always get a 504 but htmlproofer doesn't pick up. Given recent changes where you can now ignore all of a given code using raise_error_excludes: and *, I think that these 503, 504, and 999's should be removed so they can be configured by the user.

If you agree with the change @manuzhang, I'm happy to do it. If not then I think that the raise_error_excludes docs should be changed so they dont contain:

      raise_error_excludes:
        504: ['https://www.mkdocs.org/']

as 504 is always ignored at the moment and that bit of configuration while valid is also misleading.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions