Skip to content

[Transform] Support top_metrics size param #74420

@sophiec20

Description

@sophiec20

Request

Include support for top_metrics.size param.

Note: This is more complicated than it seems, because it means a single input doc will produce multiple output docs in the transform destination index.

Background

Transforms top_metrics support was added in 7.14+.

To recap, adding top_metrics allows for:

  • Performance optimisations - it is more efficient to include entity attributes as top_metrics instead of including them as group_by fields. For example, if grouping a transform by customer_id, then customer first name, last name and email address can be more efficiently included as top_metrics.
  • Multi-value aggregation support - Code refactored to allow support for aggs that return multiple values, which benefits future code extensibility

The top_metrics aggregation includes a size parameter - however the implementation of top_metrics in transforms will only return the first element as described in the limitations #71850.

Currently, the size param can be specified in the transform configuration. This is passed to the elasticsearch aggregation framework, and only the first element will ever be written to the transform destination index. It would have been better from an end-user perspective for values other than size:1 to fail fast, however because validation is being performed by the aggs framework then this would have added disproportionate code complexity.

Status: Not scheduled, requires use case.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions