Add support to skip remote peer forwarding based on configuration#5127
Merged
kkondaka merged 3 commits intoopensearch-project:mainfrom Oct 30, 2024
Merged
Conversation
dlvenable
requested changes
Oct 29, 2024
| final PeerForwarderProvider peerForwarderProvider, | ||
| final String pipelineName, | ||
| final String pluginId, | ||
| final List<String> noPeerForwardingKeys, |
Member
There was a problem hiding this comment.
Let's take this in as a Collection<Collection<String>> to indicate that the order does not matter. In fact we use it as a set.
Also, note my changes above for why it is a nested collection.
| @JsonAlias("serverPort") | ||
| final String serverPort, | ||
| @JsonProperty("no_peer_forwarding_keys") | ||
| final List<String> noPeerForwardingKeys, |
Member
There was a problem hiding this comment.
This logic is setup to exclude a key. I think that is not exactly the solution. It should exclude identification_keys.
So a couple of changes here:
- Rename this to
exclude_identification_keys - Change to a list of list of string.
exclude_identification_keys:
- ['traceId']
- ['someId1', 'someId2']
The peer forwarder should still forward for a processor with identificationKeys of traceId, someOtherValue.
| private static boolean isPeerForwardingDisabled(Processor processor, List<String> noPeerForwardingKeys) { | ||
| if (processor instanceof RequiresPeerForwarding && noPeerForwardingKeys != null && noPeerForwardingKeys.size() > 0) { | ||
| Set<String> noPeerForwardKeys = new HashSet<>(noPeerForwardingKeys); | ||
| for (final String identificationKey: ((RequiresPeerForwarding) processor).getIdentificationKeys()) { |
Member
There was a problem hiding this comment.
With the changes I suggested above, this should become something like the following:
Collection<String> identificationKeys = ((RequiresPeerForwarding) processor).getIdentificationKeys());
for(Set<String> excludedIdentificationKeys : allExcludedIdentificationKeys) {
if(excludedIdentificationKeys.equals(identificationKeys)) {
return false;
}
}
04dbc0a to
bc2393a
Compare
dlvenable
requested changes
Oct 29, 2024
| Set<String> notMatchingIdentificationKeys = Set.of("key1", "key3"); | ||
| when(requiresPeerForwarding.getIdentificationKeys()).thenReturn(identificationKeys); | ||
| final List<Processor> processors = createObjectUnderTestDecoratedProcessorsWithExcludeIdentificationKeys(Collections.singletonList((Processor) requiresPeerForwarding), List.of(notMatchingIdentificationKeys)); | ||
| for (final Processor processor: processors) { |
Member
There was a problem hiding this comment.
Please assert the number of processors before this loop. Otherwise, this could result in no verification.
assertThat(processors, hasSize(2));
| Set<String> identificationKeys = Set.of("key1", "key2"); | ||
| when(requiresPeerForwarding.getIdentificationKeys()).thenReturn(identificationKeys); | ||
| final List<Processor> processors = createObjectUnderTestDecoratedProcessorsWithExcludeIdentificationKeys(Collections.singletonList((Processor) requiresPeerForwarding), List.of(identificationKeys)); | ||
| for (final Processor processor: processors) { |
Member
There was a problem hiding this comment.
Please assert the number of processors before this loop. Otherwise, this could result in no verification.
assertThat(processors, hasSize(2));
| @JsonAlias("privateKeyPassword") | ||
| final String privateKeyPassword, | ||
| @JsonProperty("exclude_identification_keys") | ||
| final List<List<String>> excludeIdentificationKeys, |
Member
There was a problem hiding this comment.
Suggested change
| final List<List<String>> excludeIdentificationKeys, | |
| final List<Set<String>> excludeIdentificationKeys, |
Signed-off-by: Krishna Kondaka <krishkdk@dev-dsk-krishkdk-2c-b72a134e.us-west-2.amazon.com> Signed-off-by: Krishna Kondaka <krishkdk@amazon.com>
Signed-off-by: Krishna Kondaka <krishkdk@amazon.com>
241da36 to
286e822
Compare
dlvenable
previously approved these changes
Oct 29, 2024
Signed-off-by: Krishna Kondaka <krishkdk@amazon.com>
dlvenable
approved these changes
Oct 30, 2024
graytaylor0
approved these changes
Oct 30, 2024
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add support to not do remote peer forwarding optionally for certain identification keys.
If this config option is used, any processors using the same identification keys as specified in the config, do not do remote peer forwarding. All those events are locally processed.
Issues Resolved
Resolves #[Issue number to be closed when this PR is merged]
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.