Fleet installs packages and with it Elasticsearch index and component templates. This issue is to discuss potential improvements to the templates settings. Mapping improvements should be discussed separately. Below is a current example for metrics-system.cpu. The following entries I would like to discuss:

• mapping.total_fields.limit: This is set to 10000 likely because it was set that way in Beats. Because of the new data stream naming scheme I assume we don't need this anymore and can remove it and just use the default?
• refresh_interval: This is set to 5s. I would assume this could be removed and just use the default. This could lead to better ingest performance.
• number_of_shards: This is set to 1. But AFAIK by now Elasticsearch also uses 1 as the default, so this could be removed?
• query.default_field: We have stored in here all the keywords if I remember correctly. It is used to define what fields should be queried on when no field is used in the search bar. Do we still need this or could query just on all fields as now each data stream has less fields? @andrewkroh might also remember more here.
• number_of_routing_shards: This is set to 30 and likely was copied from Beats as we required this in the early days to split up shards. Is this still needed? If yes, what should it be set to?

Any other entries we should discuss further?

Component template example from metrics-system.cpu:

{
  "index": {
    "lifecycle": {
      "name": "metrics"
    },
    "codec": "best_compression",
    "mapping": {
      "total_fields": {
        "limit": "10000"
      }
    },
    "refresh_interval": "5s",
    "number_of_shards": "1",
    "query": {
      "default_field": [
        "cloud.account.id",
        "cloud.availability_zone",
        "cloud.instance.id",
        "cloud.instance.name",
        "cloud.machine.type",
        "cloud.provider",
        "cloud.region",
        "cloud.project.id",
        "cloud.image.id",
        "container.id",
        "container.image.name",
        "container.name",
        "host.architecture",
        "host.domain",
        "host.hostname",
        "host.id",
        "host.mac",
        "host.name",
        "host.os.family",
        "host.os.kernel",
        "host.os.name",
        "host.os.platform",
        "host.os.version",
        "host.os.build",
        "host.os.codename",
        "host.os.full",
        "host.type"
      ]
    },
    "number_of_routing_shards": "30"
  }
}