Skip to content

Add an ILM action to rollup an index #48003

@csoulios

Description

@csoulios

Rollup should no longer be a continuous running job. Instead, it should be an action that can be triggered on ILM-managed indices. The action should iterate over all grouping tuples (dimensions) and calculate aggregate metrics (min/max/avg/value_count) generating one document per grouping tuple (dimension).

In the context of this issue we should implement the following:

  • Implement a "grouping tuple" field type to generate the tuple at index time for the document.
  • Implement a "rollup metric" field type to store aggregation results (min/max/count/avg etc). This metric should provide the correct information to requesting aggregator (e.g. avg agg on rollup_metric will internally fetch sum + count)
  • Store document count somewhere on the document (docvalue field?)
  • Create an ILM rollup action

Metadata

Metadata

Assignees

No one assigned

    Labels

    :Data Management/ILM+SLMDO NOT USE. Use ":StorageEngine/ILM" or ":Distributed Coordination/SLM" instead.:StorageEngine/RollupTurn fine-grained time-based data into coarser-grained data>featureTeam:AnalyticsMeta label for analytical engine team (ESQL/Aggs/Geo)Team:Data Management (obsolete)DO NOT USE. This team no longer exists.

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions