Elasticsearch version: v5.0.0-alpha5

Plugins installed: []

JVM version: 1.8.0_92-b14

OS version: Mac 10.11.15

Description of the problem including expected versus actual behavior:

The convert pipeline processor throws an error if the target field to convert does not exist in the document. Ideally adding an option of ignore_missing or similar would exist and would default to true so that way for the majority of users it just works. The ignore_failure option would not work in this scenario because we would still want to fail in the situation where there was a parse error or other unknown error that happened.

This can be illustrated with the very common/poster-child example of apache log data where we would have the equivalent Logstash config in an ingest node pipeline:

PUT _ingest/pipeline/apachelogs
{
  "description": "Pipeline to parse Apache logs",
  "processors": [
    {
      "grok": {
        "field": "message",
        "patterns": [
          "%{COMBINEDAPACHELOG}"
        ]
      }
    },
    {
      "date": {
        "field": "timestamp",
        "target_field": "timestamp",
        "formats": [
          "dd/MMM/YYYY:HH:mm:ss Z"
        ]
      }
    },
    {
      "convert": {
        "field": "response",
        "type": "integer"
      }
    },
    {
      "convert": {
        "field": "bytes",
        "type": "integer"
      }
    }
  ]
}

In this situation bytes is optional and not present in the case of HEAD/OPTIONS/DELETE requests. If the bytes field is not present ideally the pipeline should still work for an end user. See related elastic/beats#2229 for original discovery and background information.

CC @talevy as talked briefly about it earlier this morning