Skip to content

First cleanup of the WIP commit#13941

Merged
martijnvg merged 1 commit intoelastic:feature/ingestfrom
martijnvg:ingest/cleanup/part1
Oct 6, 2015
Merged

First cleanup of the WIP commit#13941
martijnvg merged 1 commit intoelastic:feature/ingestfrom
martijnvg:ingest/cleanup/part1

Conversation

@martijnvg
Copy link
Copy Markdown
Member

  1. only update pipeline if the content has been changed
  2. split the actual fetching of pipeline docs from the pipeline store to make unit testing easier
  3. replaced hardcoded processor lookups with simple map registry

@martijnvg martijnvg added :Distributed/Ingest Node Execution or management of Ingest Pipelines review labels Oct 5, 2015
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a bit of a problem here. A builder as it is presented here, is a stateful construct, meaning, you use a builder to build a "configured" instance of a processor. But the passed in registry should not be stateful... effectively, the builders in the passed in registry should act as "factories".

either separate the two notions (builder & factory) or have the registry hold "prototype" builders and then change the void fromMap(Map) to Processor build(Map). Personally I like the separation of the constructs - builds clear API with clear roles and responsibility for each class. But core moved more towards the "prototype" direction, so perhaps a prototype here will be more aligned with core.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we move to prototypes than that means that we a processor can only built with a map of maps. I though that also the idea was to have a real builder that has setters for its properties. So maybe we should then take the builder & factory direction?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well.. as I said, IMO the separation is cleaner. But if you keep the prototype way, you'll eventually need to keep the current build() method and introduce a new Processor build(Map) method next to it... it's a bit messy tbh... the same applies to the Pipeline.Builder.

Another option is leave things as they are (so leave the current void fromMap(Map) method) and then introduce a factory to a builder... however you look at it, we need a factory somewhere... a construct that is registered by "type" that can create the appropriate builder/processor for that type... something along the lines of:

Processor.Builder builder = registry.get("type").create();

Another option is to mandate all builders to have an empty ctor and use reflection to create those. So instead of injecting a set of builders, you'd work with a set of builder classes and then the code will just create new instances using reflection on default ctor.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@uboness makes sense, I added the factory for the processor builders, so the registry passed to the processor builder doesn't hold state any more.

@uboness
Copy link
Copy Markdown
Contributor

uboness commented Oct 6, 2015

left some comments, OTT LGTM

split the actual fetching of pipeline docs from the pipeline store to make unit testing easier
intoduced factory for builders
replaced hardcoded processor lookups with simple factory based registry
@martijnvg martijnvg force-pushed the ingest/cleanup/part1 branch from 1d20c58 to 2071db6 Compare October 6, 2015 12:57
@martijnvg martijnvg merged commit 2071db6 into elastic:feature/ingest Oct 6, 2015
@martijnvg martijnvg removed the review label Oct 6, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Distributed/Ingest Node Execution or management of Ingest Pipelines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants