In Filebeat we are close to going over 1024 fields in the default_field setting in Elasticsearch index template. This issue could affect other Beats too in the future (most likely Metricbeat). This will cause certain queries to the index to fail with an exception like:
"caused_by": {
"type": "illegal_argument_exception",
"reason": "field expansion matches too many fields, limit: 1024, got: 1293"
}
In Beats when the index template is generated it automatically adds all text and keyword fields to the default_field list.
|
addToDefaultFields(&field) |
We need a plan to deal with the growing number of fields in default_field. This issue is causing problems for me because I'm adding fields from CEF to the fields.yml.
In Filebeat we are close to going over 1024 fields in the
default_fieldsetting in Elasticsearch index template. This issue could affect other Beats too in the future (most likely Metricbeat). This will cause certain queries to the index to fail with an exception like:In Beats when the index template is generated it automatically adds all
textandkeywordfields to thedefault_fieldlist.beats/libbeat/template/processor.go
Line 106 in cbd7749
We need a plan to deal with the growing number of fields in
default_field. This issue is causing problems for me because I'm adding fields from CEF to the fields.yml.