You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Sep 21, 2023. It is now read-only.
#151 originally called for the Elasticsearch output to default to infinite retry, similar to Beats, but our Elasticsearch ingestion library go-elasticsearch doesn't support this feature. We need to add this feature to the library and/or develop a more robust error handling mechanism to report consistent failures. Infinite retry is brittle: if we attempt it because we don't realize a particular error type is deterministically fatal, it can block the entire pipeline permanently, which can lead to data loss in many common configurations. One possible approach is to implement infinite retry for an allow-list of explicit errors that we know are always retryable, while keeping bounded retry for other error types and instead adding better error reporting so permanent failures can be recognized and diagnosed instead of blocking the rest of the pipeline.
#151 originally called for the Elasticsearch output to default to infinite retry, similar to Beats, but our Elasticsearch ingestion library
go-elasticsearchdoesn't support this feature. We need to add this feature to the library and/or develop a more robust error handling mechanism to report consistent failures. Infinite retry is brittle: if we attempt it because we don't realize a particular error type is deterministically fatal, it can block the entire pipeline permanently, which can lead to data loss in many common configurations. One possible approach is to implement infinite retry for an allow-list of explicit errors that we know are always retryable, while keeping bounded retry for other error types and instead adding better error reporting so permanent failures can be recognized and diagnosed instead of blocking the rest of the pipeline.