Skip to content

[ML] Auto-created data frame analytics destination index can omit required settings #89795

@droberts195

Description

@droberts195

When a data frame analytics job is started it will automatically create its destination index if it doesn't already exist.

When this happens the mappings are copied from the source indices. In addition two settings are copied from the source indices: index.number_of_shards and index.number_of_replicas.

However, for certain mappings in can be necessary that more settings are copied from the source indices. For example, index.analysis, index.similarity and index.mapping might be needed to make the destination index search results look like those of the source indices. In the case of custom analyzers defined in the index settings the destination index definition might not even be valid without the index.analysis setting that defines the custom analyzers.

So, we need to copy index.analysis, index.similarity and index.mapping from the source settings to destination settings. Some thought is still required as this could be complex in the case of multiple source indices with different settings. It might be possible to do a simple merge like we do for mappings.

Metadata

Metadata

Assignees

Labels

:mlMachine learning>bugTeam:MLMeta label for the ML team

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions