Skip to content

[Ingest] Add more descriptive pipeline factory errors in _simulate and PUT pipeline/id #16010

@talevy

Description

@talevy

When someone issues the following request to _simulate, The pipeline factory will error out while
attempting to construct the supplied pipeline. The reason for this is that the date processor is not configured correctly, locale is invalid. The exception is thrown in the processor's create here: https://github.com/elastic/elasticsearch/blob/feature/ingest/core/src/main/java/org/elasticsearch/ingest/processor/DateProcessor.java#L130.

{
            "pipeline": {
              "description": "_description",
              "processors": [
                {
                  "date" : {
                  "processor_tag" : "date_processor",
                  "match_field" : "date_source_field",
                  "target_field" : "date_target_field",
                  "match_formats" : ["dd/MM/yyyy"],
                  "locale": "UNDEFINED"
                  }
                },
                {
                  "set" : {
                    "field" : "field3",
                    "value" : "third_val"
                  }
                }
              ]
            },
            "docs": [
              {
                "_index": "index",
                "_type": "type",
                "_id": "id",
                "_source": {
                  "foo": "bar"
                }
              }
            ]
          }

The response from this is an internal server error. This makes it difficult to know which processor was incorrectly configured within the pipeline definition. What would be nicer is a response object that is similar to this:

{
 "pipeline_configuration_error": [
   {
     "errors": [
       {
         "processor_tag": "date_processor",
         "error": {
           "root_cause": [
             {
               "type": "illegal_argument_exception",
               "reason": "Invalid language tag specified: UNDEFINED"
             }
           ],
           "type": "illegal_argument_exception",
           "reason": "Invalid language tag specified: UNDEFINED"
         }
       }
     ]
   }
 ]
}

It would be great if we can return this processor-level configuration error to all processors that are improperly configured, instead of just the first one. This would require extra logic in the pipline factory to record exceptions and collect them into one large exception instead of throwing the first one it receives.

@martijnvg @javanna what do you think?

@BigFunger, let me know if this is what you meant? The actual response object is still up in the air.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions