Skip to content

[Ingest] add node.ingest setting that controls whether ingest is active#15610

Merged
martijnvg merged 1 commit intoelastic:feature/ingestfrom
martijnvg:ingest_node_ingest_setting
Dec 22, 2015
Merged

[Ingest] add node.ingest setting that controls whether ingest is active#15610
martijnvg merged 1 commit intoelastic:feature/ingestfrom
martijnvg:ingest_node_ingest_setting

Conversation

@martijnvg
Copy link
Copy Markdown
Member

Defaults to false.

If node.ingest isn't active then ingest related API calls fail and if the pipeline_id parameter is set then index and bulk requests fail.

@martijnvg martijnvg added review :Distributed/Ingest Node Execution or management of Ingest Pipelines labels Dec 22, 2015
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this makes me wonder: if a node has node.ingest set to false, for sure no processors should run, hence simulate should be off and IngestDisabledActionFilter should throw exception when a pipeline_id is used as it does now. But how about crud actions for pipelines? One has to go to specific nodes to store them, that have node.ingest set to true? this may not be needed, as those are just index, delete and get operations that any node supports...it's like making client nodes reject index requests, they can forward them to the proper nodes, no problem with that.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also disabling put delete and get actions on the rest layer doesn't disable them on the transport layer, I think we should remove that part.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh sorry I see that you don't register the transport action when ingest is off, sorry I had missed that.

@javanna
Copy link
Copy Markdown
Contributor

javanna commented Dec 22, 2015

I reviewed this and my comments were only about disabling put, delete and get template when node.ingest is set to false. I had doubts on this given that it's going to affect which nodes clients can send those requests to, but clients will need to be aware of the ingest plugin anyway if we disable data ingestion. I am ok with getting this in as-is. I would implement a redirect internally rather sooner than later though instead of pushing this to clients.

@javanna
Copy link
Copy Markdown
Contributor

javanna commented Dec 22, 2015

that was a LGTM then :)

@martijnvg
Copy link
Copy Markdown
Member Author

The idea is that for now clients have to be aware to what nodes they send ingest requests too and enabling indexing with a pipeline enabled. So I felt it is okay to disable both index/bulk requests with a pipeline enabled and the ingest APIs if node.ingest=false. As a follow up issue we can add a redirect feature, that redirects index/bulk requests with a pipeline enabled and ingest API requests if node.ingest=false.

…or not. Defaults to `false`.

If `node.ingest` isn't active then ingest related API calls fail and if the `pipeline_id` parameter is set then index and bulk requests fail.
@martijnvg martijnvg force-pushed the ingest_node_ingest_setting branch from 17cfaa7 to dbbb296 Compare December 22, 2015 21:39
@martijnvg martijnvg merged commit dbbb296 into elastic:feature/ingest Dec 22, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Distributed/Ingest Node Execution or management of Ingest Pipelines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants