Skip to content

New mapping parameters to annotate dimensions and metrics in timeseries data #74014

@csoulios

Description

@csoulios

As Elasticsearch embraces time series data, we must make sure that fundamental timeseries concepts become first class citizens. We propose that we implement two new mapping parameters to be used for annotating dimension and metric fields in the index mapping.

Mapping dimensions

To mark a field as a dimension we will create a mapping paramenter named time_series_dimension that can take boolean values.

Mapping metrics

To mark a field as a metric we must create a mapping paramenter named time_series_metric. Its value will be a string that can take one of the following values: gauge, counter, histogram and summary.

For each metric type there should be a set of supported downsampling aggregations. However, there are cases that users want to override the downsampling aggregations that are supported by default. To allow them to override the default aggregations, we can allow the time_series_metric to be an object containing a type and a aggregations field.

An example to illustrate the index mapping can be found below:

PUT my-index-000001
{
  "mappings": {
    "properties": {
      "hostname": {
        "type": "keyword",
        "time_series_dimension": true
      },
      "free_memory": {
        "type": "double",
        "time_series_metric": "gauge"
      },
      "free_memory": {
        "type": "double",
        "time_series_metric": {
            "type" : "counter",
            "aggregations": ["min". "max", "value_count", "sum"]
        }
      }
    }
  }
}

This issue deprecates the support for metric_type key in the field mapping meta (#72536).

Also, we should expose those parameters through the field capabilities API so that Kibana can access this information. However, this feature will be described in a separate issue.

Metadata

Metadata

Assignees

No one assigned

    Labels

    :Search Foundations/MappingIndex mappings, including merging and defining field types:StorageEngine/RollupTurn fine-grained time-based data into coarser-grained data>enhancementTeam:AnalyticsMeta label for analytical engine team (ESQL/Aggs/Geo)Team:Search FoundationsMeta label for the Search Foundations team in Elasticsearch

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions