Skip to content

Explosion of mapping fields exceeds default 1k limit on upgrade #69869

@tylersmalley

Description

@tylersmalley

We recently had a failure on CI due to our total fields exceeded 1k during an upgrade.

Error: Limit of total fields [1000] in index [.kibana_2] has been exceeded

In this situation, the security solution is defining a mapping for the Kibana index from a few months ago. During the functional tests, the migrations are ran on this index due to it being out of date. Migrations will continue to persist previous mappings into the future index to ensure we can copy data from plugins which are unavailable. There was recently a rename for the maps-telemetry object type which plays a part in this (#69816).

@aaronjcaldwell, we might want to consider reverting #69816 for the reason mentioned above until we have a solution.

Additionally, the maps-telemetry type defines two properties with dynamic: true. We should prevent this, as it can greatly effect the number of fields which we have. An alternative option if the plugin is outside of OSS, would be to use the flattened type which might be more fitting.

I am marking this a blocker for 7.9 as it seems likely users who are upgrading would encounter this as well.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Team:CorePlatform Core services: plugins, logging, config, saved objects, http, ES client, i18n, etc t//blockerv7.9.0

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions