-
Notifications
You must be signed in to change notification settings - Fork 2k
enhancement(http sink): Retrying the HTTP sink in case of 404s and request timeouts #21457
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Hi @noble-varghese, thank you for the contribution. I think retrying for these codes is reasonable, I will take a closer look tomorrow. Note: Need to update the docs as well. See https://vector.dev/docs/reference/configuration/sinks/http/#retry-policy. |
|
@pront Thanks for pointing it out ! I have updated the docs page and added unit tests on the http utils as well :) |
pront
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks!
pront
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hey @noble-varghese, I noticed the E2E test http_to_http failed. This change actually is a breaking one. Since it is hard to evaluate the risk, it is better to err on the side of caution. After some discussion with the team, I created #21469 to layout a solution for this.
|
To unblock this PR, perhaps we can add 408 to the defaults? Still a breaking change, but seems desirable and low risk. cc @jszwedko |
|
I tried to push directly to the fork but I don't have write permissions. |
I will push those changes. Thanks ! |
|
@pront Just curious to know, how do you run these tests on the local. It was taking me about 30-40mins for it to run. Is there a faster way ? |
Co-authored-by: Jesse Szwedko <jesse@szwedko.me>
You could try |
Thanks Jesse, very neat. Also, I always run the following before pushing to the remote branch:
|
Adding the functionality to retry the log sending on getting a request timeout (408) from the HTTP sink. Since the default behavior is to drop the events during such events, this would affect the log propagation to the sinks.
solves: #10870