Skip to content

[Ingest] Change document field of _simulate with verbose=true to diff view #14698

@talevy

Description

@talevy

This is in regards to the _simulate endpoint.

It would be nice to have a view of what was modified in a document between processors in a pipeline instead of seeing a whole new copy of the full document.

Currently, the output for a pipeline looks like this

{
      "error": false,
      "processor_results": [
        {
          "processor_id": "processor[mutate]-0",
          "error": false,
          "modified": true,
          "doc": {
            "_index": "index",
            "_type": "type",
            "_source": {
              "bar": "hello",
              "foo": 5
            },
            "_id": "id2"
          }
        },
        {
          "processor_id": "processor[mutate]-1",
          "error": false,
          "modified": true,
          "doc": {
            "_index": "index",
            "_type": "type",
            "_source": {
              "bar": "HELLO",
              "foo": 5
            },
            "_id": "id2"
          }
        }
      ]
    }

From this, you can see that the doc field shows all of the document's fields. It would be great to only show what has been modified from after the previous processor's mutations.

Something like this would be nice:

{
      "error": false,
      "processor_results": [
        {
          "processor_id": "processor[mutate]-0",
          "error": false,
          "modified": true,
          "doc_diff": {
            "updated": {
              "_source.foo": 5
            },
            "added": {},
            "removed": {}
          }
        },
        ...
      ]
    }

depends on: #14572

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions