Skip to content

Make timestamp field mapping part of data stream definition #58583

Closed
#59076
Closed
Make timestamp field mapping part of data stream definition#58583
#59076
Labels
:Search Foundations/MappingIndex mappings, including merging and defining field types:StorageEngine/Data streamsData streams and their lifecyclesTeam:Data Management (obsolete)DO NOT USE. This team no longer exists.Team:Search FoundationsMeta label for the Search Foundations team in Elasticsearch
@martijnvg

Description

@martijnvg
martijnvg
opened on Jun 26, 2020

Currently when creating a composable index template with a data stream definition, the timestamp field mapping must be defined too:

PUT /_index_template/logs_data_stream
{
  "index_patterns": [ "logs*" ],
  "data_stream": {
    "timestamp_field": "@timestamp"
  },
  "template": {
    "mappings": {
      "properties": {
        "@timestamp": {
          "type": "date"
        }
      }
    }
  }
}

The template is much cleaner to read if the mapping itself for timestamp field mapping
is inlined with the data stream definition:

PUT /_index_template/logs_data_stream
{
  "index_patterns": [ "logs*" ],
  "data_stream": {
    "timestamp_field": "@timestamp",
    "timestamp_mapping": {
      "type": "date"
    }
  }
}

Also since most of time, just "type": "date" will be provided, it makes sense to make this the default:

PUT /_index_template/logs_data_stream
{
  "index_patterns": [ "logs*" ],
  "data_stream": {
    "timestamp_field": "@timestamp"
  }
}

In the last two example the backing indices created will have the following mapping:

{
  "_doc": {
    "properties": {
      "@timestamp": {
        "type": "date"
      }
    }
  }
}

The simulate index template api should also include the inserted / generated timestamp field mapping
in the response.

Metadata

Metadata

Assignees

No one assigned

    Labels

    :Search Foundations/MappingIndex mappings, including merging and defining field types:StorageEngine/Data streamsData streams and their lifecyclesTeam:Data Management (obsolete)DO NOT USE. This team no longer exists.Team:Search FoundationsMeta label for the Search Foundations team in Elasticsearch

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions