-
Notifications
You must be signed in to change notification settings - Fork 25.8k
Create a default mapping template for metrics #72536
Copy link
Copy link
Closed
Labels
:Search Foundations/MappingIndex mappings, including merging and defining field typesIndex mappings, including merging and defining field types:StorageEngine/Data streamsData streams and their lifecyclesData streams and their lifecycles>enhancementTeam:Data Management (obsolete)DO NOT USE. This team no longer exists.DO NOT USE. This team no longer exists.Team:Search FoundationsMeta label for the Search Foundations team in ElasticsearchMeta label for the Search Foundations team in Elasticsearch
Metadata
Metadata
Assignees
Labels
:Search Foundations/MappingIndex mappings, including merging and defining field typesIndex mappings, including merging and defining field types:StorageEngine/Data streamsData streams and their lifecyclesData streams and their lifecycles>enhancementTeam:Data Management (obsolete)DO NOT USE. This team no longer exists.DO NOT USE. This team no longer exists.Team:Search FoundationsMeta label for the Search Foundations team in ElasticsearchMeta label for the Search Foundations team in Elasticsearch
Type
Fields
Give feedbackNo fields configured for issues without a type.
With dynamic templates in bulk requests (#69948) merged, we are considering the option of creating a default template to allow for dynamic metrics ingestion. This would be leveraged by many Elastic Agent integrations, and is specially interesting for the ones reporting dynamic metrics (where we only get to know the metric / field names at runtime, and hence, we cannot define a mapping before ingestion).
The overall idea is to map all possible combinations of <metric_type> and <unit>, so each combination gets a unique predictable name that can be referenced at ingest time. For example:
The obvious concern is: We would be creating a dynamic mapping entry per every combination of <metric_type> and , which will explode into many entries:
byte,percent,d,h,m,s,ms,micros,nanos, with more units to come as we onboard other metrics.long,double,integer,byte,float,scaled_float.So the main question is: Would this be considered a good practice? Can it cause any issues because of a too large mapping template? Perhaps we should do this in a different way?