Skip to content

[Ingest] Pipeline template usage  #14990

@martijnvg

Description

@martijnvg

This issue's purpose is to collect ideas on mustache template usage in pipelines. The mustache initial template support will be added via the meta processor (#14991), but it leaves adoption of templates by other processors open.

Open questions:

  • What other processors and settings should be templatable? If some options are just strings and some are templates then how should ingest know this at pipeline parse time? Do we need a template json object that distinguish between a template or regular string? Or any string that contains {{ and }} is considered a template. The latter makes things less verbose IMO.
  • How should we refer to the original document in the pipeline? Right now in the meta processor the source of the document being processed is directly accessible in the moustache template, but maybe we want to make the the source available under the _source prefix? (e.g. _source.field1 or _source.field1.field2 etc.) This allows to for example put metadata attributes of document being processed under a different prefix. Also do we want like the update api use the ctx prefix in front of _source?
  • It isn't possible the access array/list elements in the current mustache implementation. I think we want to add support for that?

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions