As we continue to add more SavedObject types, we are starting to push up against the default field limit of 1000 mapped fields in Elasticsearch.
One common culprit that causes this is the usage of dynamic mappings in SavedObject schemas that cause the mappings to grow over time and eventually hit the 1000 limit. This failure scenario mappings are hard to test for since it depends on the shape of the data being ingested which can vary based on user input.
In most cases, dynamic mappings are not needed since we do not need to aggregate or search on these fields. Instead, we should either be using index: false or dynamic: false in order to support documents with dynamic key-value pairs without needing to index these keys in Elasticsearch.
An audit needs to be performed, but we may be able to remove support for this altogether in order to eliminate this problem.
Related #69869
As we continue to add more SavedObject types, we are starting to push up against the default field limit of 1000 mapped fields in Elasticsearch.
One common culprit that causes this is the usage of dynamic mappings in SavedObject schemas that cause the mappings to grow over time and eventually hit the 1000 limit. This failure scenario mappings are hard to test for since it depends on the shape of the data being ingested which can vary based on user input.
In most cases, dynamic mappings are not needed since we do not need to aggregate or search on these fields. Instead, we should either be using
index: falseordynamic: falsein order to support documents with dynamic key-value pairs without needing to index these keys in Elasticsearch.An audit needs to be performed, but we may be able to remove support for this altogether in order to eliminate this problem.
Related #69869