Skip to content

[7.x] Use V2 index templates during index creation (#54669)#54750

Merged
dakrone merged 1 commit intoelastic:7.xfrom
dakrone:backport/7.x/pr-54669
Apr 3, 2020
Merged

[7.x] Use V2 index templates during index creation (#54669)#54750
dakrone merged 1 commit intoelastic:7.xfrom
dakrone:backport/7.x/pr-54669

Conversation

@dakrone
Copy link
Copy Markdown
Member

@dakrone dakrone commented Apr 3, 2020

  • Use V2 index templates during index creation

This commit changes our index creation code to use (and favor!) V2 index templates during index
creation. The creation precedence goes like so, in order of precedence:

  • Existing source IndexMetadata - for example, when recovering from a peer or a shrink/split/clone
    where index templates should not be applied
  • A matching V2 index template, if one is found
    • When a V2 template is found, all component templates (in the composed_of field) are applied
      in the order that they appear, with the index template having the 2nd highest precedence (the
      create index request always has the top priority when it comes to index settings)
  • All matching V1 templates (the old style)

This also adds index template validation when PUT-ing a new v2 index template (because this was
required) and ensures that all index and component templates specify no top-level mapping type (it
is automatically added when the template is added to the cluster state).

This does not yet implement fine-grained component template merging of mappings, where we favor
merging only a single field's configuration, that will be done in subsequent work.

This also keeps the existing hidden index behavior present for v1 templates, where a hidden index
will match v2 index templates unless they are global (*) templates.

Relates to #53101

* Use V2 index templates during index creation

This commit changes our index creation code to use (and favor!) V2 index templates during index
creation. The creation precedence goes like so, in order of precedence:

- Existing source `IndexMetadata` - for example, when recovering from a peer or a shrink/split/clone
  where index templates should not be applied
- A matching V2 index template, if one is found
  - When a V2 template is found, all component templates (in the `composed_of` field) are applied
    in the order that they appear, with the index template having the 2nd highest precedence (the
    create index request always has the top priority when it comes to index settings)
- All matching V1 templates (the old style)

This also adds index template validation when `PUT`-ing a new v2 index template (because this was
required) and ensures that all index and component templates specify *no* top-level mapping type (it
is automatically added when the template is added to the cluster state).

This does not yet implement fine-grained component template merging of mappings, where we favor
merging only a single field's configuration, that will be done in subsequent work.

This also keeps the existing hidden index behavior present for v1 templates, where a hidden index
will match v2 index templates unless they are global (`*`) templates.

Relates to elastic#53101
@dakrone dakrone added :Data Management/Indices APIs DO NOT USE. Use ":Distributed/Indices APIs" or ":StorageEngine/Templates" instead. backport v7.8.0 labels Apr 3, 2020
@elasticmachine
Copy link
Copy Markdown
Collaborator

Pinging @elastic/es-core-features (:Core/Features/Indices APIs)

@dakrone dakrone merged commit 814c248 into elastic:7.x Apr 3, 2020
@dakrone dakrone deleted the backport/7.x/pr-54669 branch April 3, 2020 20:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport :Data Management/Indices APIs DO NOT USE. Use ":Distributed/Indices APIs" or ":StorageEngine/Templates" instead. v7.8.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants