Skip to content

Respect HTTP error codes in CLI inputs#2101

Merged
mre merged 14 commits into
masterfrom
fix-1883
Apr 3, 2026
Merged

Respect HTTP error codes in CLI inputs#2101
mre merged 14 commits into
masterfrom
fix-1883

Conversation

@mre

@mre mre commented Mar 25, 2026

Copy link
Copy Markdown
Member

Emit a warning hard error when a CLI input URL returns 4xx/5xx but keep processing other inputs.
Fixes #1883.

Before

$ lychee http://example.com/this-link-does-not-exist

1/1 ━━━━━━━━━━━━━━━━━━━━ Finished extracting links                                                    
[WARN] lychee detected 1 redirect. You might want to consider replacing redirecting URLs with the resolved URLs. Run lychee in verbose mode (-v/--verbose) to see details about the redirections.
🔍 1 Total (in 1s 10ms) ✅ 0 OK 🚫 0 Errors 🔀 1 Redirects

The error got silently ignored.

After

$ lychee http://example.com/this-link-does-not-exist

0/0 ━━━━━━━━━━━━━━━━━━━━ Extracting links                                                             
[WARN] Input URL http://example.com/this-link-does-not-exist returned status code 404 Not Found
1/1 ━━━━━━━━━━━━━━━━━━━━ Finished extracting links                                                    
[WARN] lychee detected 1 redirect. You might want to consider replacing redirecting URLs with the resolved URLs. Run lychee in verbose mode (-v/--verbose) to see details about the redirections.
🔍 1 Total (in 783ms) ✅ 0 OK 🚫 0 Errors 🔀 1 Redirects

Much After

Update! Converted to hard error (see discussion below):

$ lychee http://example.com/this-link-does-not-exist
Error: Cannot read input content from URL: status code 404 Not Found

Note that modifying --accept has no impact on the outcome since this is an input-level error.

@mre mre force-pushed the fix-1883 branch 2 times, most recently from 8bf9db2 to 2061195 Compare March 25, 2026 09:58

@katrinafyi katrinafyi left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm happy with using the fixed range of HTTP errors.

In a vacuum, I would lean towards making it an error with an option to bypass the error. I don't think it's too much disruption, and most people would probably appreciate the obvious error. But either way is much more helpful than silent with no message.

Comment thread lychee-lib/src/types/resolver.rs Outdated

@thomas-zahner thomas-zahner left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall 🚀

Comment thread lychee-lib/src/types/resolver.rs Outdated
Comment thread lychee-bin/tests/cli.rs Outdated
@mre

mre commented Mar 26, 2026

Copy link
Copy Markdown
Member Author

@katrinafyi

In a vacuum, I would lean towards making it an error with an option to bypass the error. I don't think it's too much disruption, and most people would probably appreciate the obvious error. But either way is much more helpful than silent with no message.

I was debating this myself! Initially, I tried to be conservative, but after thinking about it some more, let's be bold and the right thing. A hard error will definitely prevent users from accidentally checking an empty/error page and assuming everything is fine. Let me update this PR to return an error instead of a warning. What do you think about deferring the introduction the bypass-flag until someone explicitly asks for it? Too pragmatic?

Comment thread lychee-lib/src/types/error.rs Outdated

@katrinafyi katrinafyi left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about deferring the introduction the bypass-flag until someone explicitly asks for it? Too pragmatic?

I think I'm just very cautious when it comes to changing defaults and errors, after past experience 😅 But in this case, it is probably okay to go ahead without the flag for now.

Comment thread lychee-lib/src/types/error.rs Outdated
@mre

mre commented Mar 30, 2026

Copy link
Copy Markdown
Member Author

I think I'm just very cautious when it comes to changing defaults and errors, after past experience 😅

Being cautious is generally the wise approach, yes, but in this case, my goal is to avoid increasing the number of CLI flags we have to maintain. There's a chance we will have to add another flag, but my hope is that people just treat it as an actual bugfix and accept it as the expected behavior when providing invalid inputs. We'll have to see. :^)

Comment thread lychee-bin/src/main.rs Outdated
@mre mre force-pushed the fix-1883 branch 2 times, most recently from 1100b41 to a34f7b9 Compare April 3, 2026 19:29
@mre

mre commented Apr 3, 2026

Copy link
Copy Markdown
Member Author

The actual change is pretty simple, but we went through a few iterations to find the right approach. I'm still not 100% satisfied, since the new match arm returns a sentinel value. I've added a TODO to refactor that:

// This `details()` method never gets called for incorrect CLI
// inputs, so whatever we put here, it won't be shown to the user.
//
// This returns an empty string as a sentinel value because it's handled as a
// fatal application error rather than a link-level error.
//
// TODO: In the future, we should return an Option<String> or separate
// application errors from library errors.
ErrorKind::ReadInputUrlStatusCode(_) => String::new(),

Comment thread lychee-bin/tests/cli.rs
Comment on lines -3948 to +3953
.arg("https://example.com/cargo_exclude_test_str")
.arg("input.txt")

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Had to convert this to an input file since the original URL was invalid and that now causes an input error.

@mre mre merged commit 8bbeeb5 into master Apr 3, 2026
8 checks passed
@mre mre deleted the fix-1883 branch April 3, 2026 20:03
@mre mre mentioned this pull request Apr 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

HTTP error codes in command-line inputs are ignored

3 participants