Today, there's an API to get per index field or all fields:
GET <index>/_field_caps?fields=*
which returns:
{
"indices" : [
"my_index",
"my_index2",
"my_index3",
...
],
"fields" : {
"my_field1" : {
"keyword" : {
"type" : "keyword",
"searchable" : true,
"aggregatable" : true
}
},
"my_field2" : {
"keyword" : {
"type" : "keyword",
"searchable" : true,
"aggregatable" : true
}
},
...
}
}
But the output from the above doesn't have break down by index, so it's not easy to troubleshoot which index is having mapping explosion problem.
I can potentially loop over my indices list and running GET <index>/field_caps?fields=* API to extract the field length per index, but again, this is not ideal.
It would be most useful if Elasticsearch can have an API out-of-the-box to count the number of fields breakdown by the indices, expected output should look like:
index_name field_count
my_index 10
my_index2 20
my_index3 30
This can be part of the GET indices/stats or cat indices API.
Btw, kibana might be able to take advantage of the API instead of doing its own aggregation/counting to show index pattern field count:
Not sure if the following is the best script to extract the field count, but with some jq over the GET _mapping output, I am able to get the desired format:
curl -XGET https://localhost:9200/_mapping > mapping.json
jq -r '[to_entries[]| .key as $index| [ .value.mappings|to_entries[]|{(.key): ([.value|..|.type?|select(.!=null)]|length)} ]|map(to_entries)| flatten| sort_by(.value)|from_entries as $types | {index_name: $index, index_field_count: ([$types|to_entries[].value]|add)}]|.[]|.|map(.)|@tsv ' mapping.json | column -t | sort
Today, there's an API to get per index field or all fields:
which returns:
But the output from the above doesn't have break down by index, so it's not easy to troubleshoot which index is having mapping explosion problem.
I can potentially loop over my indices list and running
GET <index>/field_caps?fields=*API to extract the field length per index, but again, this is not ideal.It would be most useful if Elasticsearch can have an API out-of-the-box to count the number of fields breakdown by the indices, expected output should look like:
This can be part of the
GET indices/statsorcat indicesAPI.Btw, kibana might be able to take advantage of the API instead of doing its own aggregation/counting to show index pattern field count:
Not sure if the following is the best script to extract the field count, but with some
jqover theGET _mappingoutput, I am able to get the desired format: