Skip to content

Templates match indices with date math expressions#71433

Merged
danhermann merged 3 commits intoelastic:masterfrom
danhermann:69727_fix_final_pipeline
Apr 8, 2021
Merged

Templates match indices with date math expressions#71433
danhermann merged 3 commits intoelastic:masterfrom
danhermann:69727_fix_final_pipeline

Conversation

@danhermann
Copy link
Copy Markdown
Contributor

@danhermann danhermann commented Apr 7, 2021

The root cause of the bug identified in #69727 (reported independently by several users) is that the findV?Templates methods in MetadataIndexTemplateService do not resolve index names with date math expressions before attempting to match them. The reproduction scenario in #69727 demonstrates one flavor of this problem. Before 7.9, an ingest pipeline with a date_index_name processor would (accidentally) work around that problem, but the simple statement below would fail to execute the final pipeline in any ES version (assuming a template that defines a final pipeline for an index pattern of test-*):

# url-encoded form of <test-{now/d}>
POST /%3Ctest-%7Bnow%2Fd%7D%3E/_doc 
{
  "field": "value"
}

A change to the way ingest pipelines work in 7.9 caused the date_index_name scenario to fail as well. This change resolves both problems.

Fixes #69727.

@danhermann danhermann added >bug :Distributed/Ingest Node Execution or management of Ingest Pipelines v8.0.0 v7.13.0 v7.12.1 labels Apr 7, 2021
@elasticmachine elasticmachine added the Team:Data Management (obsolete) DO NOT USE. This team no longer exists. label Apr 7, 2021
@elasticmachine
Copy link
Copy Markdown
Collaborator

Pinging @elastic/es-core-features (Team:Core/Features)

@danhermann
Copy link
Copy Markdown
Contributor Author

@elasticmachine update branch

Copy link
Copy Markdown
Member

@dakrone dakrone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@danhermann danhermann merged commit 072607a into elastic:master Apr 8, 2021
@danhermann danhermann deleted the 69727_fix_final_pipeline branch April 8, 2021 12:15
@danhermann
Copy link
Copy Markdown
Contributor Author

Thanks, @dakrone!

@AthenaEryma
Copy link
Copy Markdown
Contributor

@danhermann Looks like we're good to remove backport pending from this PR, yes?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

>bug :Distributed/Ingest Node Execution or management of Ingest Pipelines Team:Data Management (obsolete) DO NOT USE. This team no longer exists. v7.12.1 v7.13.0 v8.0.0-alpha1

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Final Pipeline is not applied when routing using a pipeline

5 participants