It would be nice to have a flag for ingest pipelines that run during index/bulk requests to allow for some configuration of how errors are handled during ingestion.
Currently, any exception that is caused during an ingest pipeline, the pipeline halts and the document being ingested is not indexed. Some environments may prefer to not block on such failures and continue to index the failed document anyways. For these scenarios it would be nice to have an continue_on_failure flag turned on to instruct Elasticsearch to index the document as it was before it was processed by the ingest pipeline. Such documents can be tagged so that they can be found in the index for re-processing.
_simulate should also support this flag to help with recreating pipeline behavior
depends on #14888
It would be nice to have a flag for ingest pipelines that run during index/bulk requests to allow for some configuration of how errors are handled during ingestion.
Currently, any exception that is caused during an ingest pipeline, the pipeline halts and the document being ingested is not indexed. Some environments may prefer to not block on such failures and continue to index the failed document anyways. For these scenarios it would be nice to have an
continue_on_failureflag turned on to instruct Elasticsearch to index the document as it was before it was processed by the ingest pipeline. Such documents can be tagged so that they can be found in the index for re-processing._simulateshould also support this flag to help with recreating pipeline behaviordepends on #14888