Skip to content
This repository was archived by the owner on Sep 21, 2023. It is now read-only.
This repository was archived by the owner on Sep 21, 2023. It is now read-only.

Shipper doesn't support infinite retry / robust failure handling #262

@faec

Description

@faec

#151 originally called for the Elasticsearch output to default to infinite retry, similar to Beats, but our Elasticsearch ingestion library go-elasticsearch doesn't support this feature. We need to add this feature to the library and/or develop a more robust error handling mechanism to report consistent failures. Infinite retry is brittle: if we attempt it because we don't realize a particular error type is deterministically fatal, it can block the entire pipeline permanently, which can lead to data loss in many common configurations. One possible approach is to implement infinite retry for an allow-list of explicit errors that we know are always retryable, while keeping bounded retry for other error types and instead adding better error reporting so permanent failures can be recognized and diagnosed instead of blocking the rest of the pipeline.

Metadata

Metadata

Labels

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions