Introduce instant enrich policies.#73407
Closed
martijnvg wants to merge 6 commits intoelastic:mainfrom
Closed
Conversation
Instant enrich policies don't need to be executed and the enrich processor performs the lookups directly against the enrich policy's source index. Instant enrich policies are a trade off to allow more flexibility. Changes made to the source index are almost immediately taken into account by the enrich processor using an instant enrich policy. There is no latency between changes made to the source index and when the enrich policy is executed to create the new enrich index. An enrich policy with the `instant` property set to `false` is more efficient when it comes to performing the lookup for the enrichment, because the lookup is performed against an optimized enrich index. Relates to elastic#48988
No need to provide Supplier<EnrichPolicy>, because an enrich policy can't be updated after it has been created.
Contributor
|
@martijnvg has there been any progress on this? or maybe you can point me to another ongoing effort to make the enrich policy more dynamic, i.e. an enrich policy sourcing its data directly from the source index. Thanks in advance for sheding some light on this |
Member
Author
|
@consulthys As far as I know, this is the only effort of making enrich policies more dynamic. |
10 tasks
Member
|
Closing for now until we can get back to this, we will probably end up re-using this code in the future though. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Instant enrich policies don't need to be executed and
the enrich processor performs the lookups directly against
the enrich policy's source index.
Instant enrich policies are a trade off to allow more flexibility.
Changes made to the source index are almost immediately taken into
account by the enrich processor using an instant enrich policy.
There is no latency between changes made to the source index and
when the enrich policy is executed to create the new enrich index.
An enrich policy with the
instantproperty set tofalseismore efficient when it comes to performing the lookup for the enrichment,
because the lookup is performed against an optimized enrich index.
Relates to #48988
Example usage: