-
Notifications
You must be signed in to change notification settings - Fork 25.8k
Find a way to Deduplicate Index Settings #78892
Copy link
Copy link
Open
Labels
:Distributed/Cluster CoordinationCluster formation and cluster state publication, including cluster membership and fault detection.Cluster formation and cluster state publication, including cluster membership and fault detection.>enhancementTeam:DistributedMeta label for distributed team.Meta label for distributed team.
Metadata
Metadata
Assignees
Labels
:Distributed/Cluster CoordinationCluster formation and cluster state publication, including cluster membership and fault detection.Cluster formation and cluster state publication, including cluster membership and fault detection.>enhancementTeam:DistributedMeta label for distributed team.Meta label for distributed team.
Type
Fields
Give feedbackNo fields configured for issues without a type.
Relates #77466
Unlike the mapping metadata which we store in compressed+serialized for on this master node, we do store the settings for each index as deserialized
Settingsobject. This can be a significant source of heap usage if there's a large number of indices with non-trivial settings. An example of this would be the audit-beats template which contains a fairly long list of field names inindex.query.default_field. In this example, handling 10k audit-beat indices takes almost 500MB of master heap just for storing the duplicate lists of field names inSettingsinstances.I will look for an easy win here, it shouldn't be too hard to deduplicate these in some form when building index metadata.