Add an internal refresh api that makes pipeline changes visible on all ingest nodes#15203
Conversation
|
I've updated this PR to move some initialisation logic out of the store into a new class called PipelineStoreBootstrapper. The pipeline store became a bit convoluted with initialisation logic. |
There was a problem hiding this comment.
s/representaion/representation/
|
would it be beneficial to have refresh action/request tests in |
|
I think it makes sense to add a test for this api, even though this is an internal api that is designed to be used only via put and delete pipeline apis. The reason that I did not add an integration test for this yet, is that it is already tests by the put and delete pipeline rest and ClientIT tests. |
|
edit: never-mind, that is handled from within the pipelinestore |
There was a problem hiding this comment.
if the api is really internal, I think we can simplify this. Do we need to use a client here? Can we instead use the transport service directly? In that case we wouldn't need the RefreshAction, and the RefreshRequestBuilder. Otherwise the api ends up being exposed anyways, no matter if we say it's internal, but it doesn't have a corresponding REST handler, which makes things inconsistent.
There was a problem hiding this comment.
agreed and this what I initially tried (like the shards exists internal api, See IndicesStore.java), but then I ran into cycle dependency issue with transport service.
There was a problem hiding this comment.
ok...but client depends on the transport service anyway no? I think I don't get it
There was a problem hiding this comment.
yes it does. all the ingest transport stuff gets instantiate with the the
action module and not with the node modules of the plugin. (See
IngestPlugin#onModule(ActionModule) ) So that is why the current approach
works.
On 4 December 2015 at 12:47, Luca Cavanna notifications@github.com wrote:
In
plugins/ingest/src/main/java/org/elasticsearch/plugin/ingest/PipelineStore.java
#15203 (comment)
:DeleteRequest deleteRequest = new DeleteRequest(request); deleteRequest.index(PipelineStore.INDEX); deleteRequest.type(PipelineStore.TYPE); deleteRequest.id(request.id()); deleteRequest.refresh(true);
client().delete(deleteRequest, listener);client().delete(deleteRequest, new ActionListener<DeleteResponse>() {@Overridepublic void onResponse(DeleteResponse deleteResponse) {client().execute(RefreshAction.INSTANCE, new RefreshRequest(), new ActionListener<RefreshResponse>() {ok...but client depends on the transport service anyway no? I think I
don't get it—
Reply to this email directly or view it on GitHub
https://github.com/elastic/elasticsearch/pull/15203/files#r46674077.
Met vriendelijke groet,
Martijn van Groningen
f914155 to
9af7646
Compare
|
@javanna @talevy I've updated the PR. Changed:
|
There was a problem hiding this comment.
sorry but we have to find other solutions for this than using these injection providers. We can't sustain this kind of software engineering here.
There was a problem hiding this comment.
This one is easy. Can make this an inner class of PipelineStore and then there isn't a cyclic dependency.
There was a problem hiding this comment.
why is shit guice exposed at all? Can't we reduce the number of @Inject here please it's such a pain to unwire all of this. Can we simply remove the @Inject and all the wiring and to in PipelineStoreBootstrapper
ReloadPipelinesAction action = new ReloadPipelinesAction(settings, pipelineStore, clusterService, transportService);we can go even further and also don't wire PipelineStore and just call new in the bootstrapper as well.
There was a problem hiding this comment.
so the reason pipeline store instance is managed by Guice is that the transport action classes uses it as a dependency. But that shouldn't be a problem, because if the store is exposed as a getter and the transport action classes depend on PipelineStoreBootstrapper then the constructor of the transport action classes can use this getter.
instead of only relying on async background task that periodically loads pipelines into memory. This async loading mechanism remains for loading pipelines during startup.
node.ingestsetting totrue. This is used to figure out to what nodes to send the refresh request too. This setting isn't yet configurable. This will be done in a follow up issue.