Skip to content

[Ingest] Add index template for the '.ingest' index#15631

Closed
martijnvg wants to merge 0 commit intoelastic:feature/ingestfrom
martijnvg:ingest_index_template
Closed

[Ingest] Add index template for the '.ingest' index#15631
martijnvg wants to merge 0 commit intoelastic:feature/ingestfrom
martijnvg:ingest_index_template

Conversation

@martijnvg
Copy link
Copy Markdown
Member

Also added logic that ensures that the index template is installed.

An index template for the '.ingest' index is required because:

  • We don't want arbitrary fields in pipeline documents, because that can turn into upgrade problems if we add more properties to the pipeline dsl.
  • We know what are the usages are of the '.ingest' index, so we can optimize for that and prevent that this index is used for different purposes.

@martijnvg martijnvg added review :Distributed/Ingest Node Execution or management of Ingest Pipelines labels Dec 23, 2015
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so, if we fail to install the template, we just flag and continue working as usual? So ingest template is optimistic, and not a requirement.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This method will only be invoked if ingest is running and the ingest template gets removed. (for example, a user deletes all his index templates). In this case we try to put the index template back if that fails we try again on the next cluster state update. (on each modification to the cluster the clusterChanged gets invoked)

In the case that we're not started and adding the index template fails we don't start the PipelineStore and on the next cluster state update we try to add the template and start the PipelineStore again.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

@talevy
Copy link
Copy Markdown
Contributor

talevy commented Dec 23, 2015

LGTM

@martijnvg martijnvg force-pushed the ingest_index_template branch from 814f7fd to d302748 Compare December 24, 2015 14:08
@martijnvg martijnvg closed this Dec 24, 2015
@martijnvg martijnvg force-pushed the ingest_index_template branch from d302748 to 1936d61 Compare December 24, 2015 14:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Distributed/Ingest Node Execution or management of Ingest Pipelines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants