feat: Add configurable stream read constraints for JSON input codec#5541
Merged
dlvenable merged 3 commits intoopensearch-project:mainfrom Apr 9, 2025
Merged
Conversation
- Introduced a configurable option for setting the maximum event length in the JSON input codec. - Updated JsonDecoder to accept max_event_length parameter and apply it to StreamReadConstraints. - Added validation to ensure the maximum event length is within acceptable limits. Fixes opensearch-project#5466 Signed-off-by: Pallempati Saketh <pallempati.saketh@fmr.com>
graytaylor0
previously approved these changes
Mar 25, 2025
...cessor/src/main/java/org/opensearch/dataprepper/plugins/codec/json/JsonInputCodecConfig.java
Outdated
Show resolved
Hide resolved
data-prepper-api/src/main/java/org/opensearch/dataprepper/model/codec/JsonDecoder.java
Show resolved
Hide resolved
Signed-off-by: Pallempati Saketh <pallempati.saketh@fmr.com>
Contributor
Author
|
Hi @graytaylor0 , |
graytaylor0
previously approved these changes
Mar 28, 2025
san81
reviewed
Mar 28, 2025
...cessor/src/main/java/org/opensearch/dataprepper/plugins/codec/json/JsonInputCodecConfig.java
Outdated
Show resolved
Hide resolved
chenqi0805
previously approved these changes
Mar 31, 2025
Collaborator
chenqi0805
left a comment
There was a problem hiding this comment.
Thanks for making the change!
Contributor
Author
@chenqi0805 Thanks for approval. |
Signed-off-by: Pallempati Saketh <pallempati.saketh@fmr.com>
san81
approved these changes
Apr 1, 2025
dlvenable
approved these changes
Apr 9, 2025
Member
dlvenable
left a comment
There was a problem hiding this comment.
Thank you @saketh-pallempati for this improvement!
Contributor
Author
|
Thanks to all ✌️ |
amdhing
pushed a commit
to amdhing/data-prepper
that referenced
this pull request
Apr 16, 2025
…pensearch-project#5541) feat: Add configurable stream read constraints for JSON input codec - Introduced a configurable option for setting the maximum event length in the JSON input codec. - Updated JsonDecoder to accept max_event_length parameter and apply it to StreamReadConstraints. - Added validation to ensure the maximum event length is within acceptable limits. Fixes opensearch-project#5466 * Set maxEventLength default to null for safer usage. * Set min value of maxEventLength to Jackson's default Signed-off-by: Pallempati Saketh <pallempati.saketh@fmr.com>
Davidding4718
pushed a commit
to Davidding4718/data-prepper
that referenced
this pull request
Apr 25, 2025
…pensearch-project#5541) feat: Add configurable stream read constraints for JSON input codec - Introduced a configurable option for setting the maximum event length in the JSON input codec. - Updated JsonDecoder to accept max_event_length parameter and apply it to StreamReadConstraints. - Added validation to ensure the maximum event length is within acceptable limits. Fixes opensearch-project#5466 * Set maxEventLength default to null for safer usage. * Set min value of maxEventLength to Jackson's default Signed-off-by: Pallempati Saketh <pallempati.saketh@fmr.com>
Davidding4718
pushed a commit
to Davidding4718/data-prepper
that referenced
this pull request
Apr 25, 2025
…pensearch-project#5541) feat: Add configurable stream read constraints for JSON input codec - Introduced a configurable option for setting the maximum event length in the JSON input codec. - Updated JsonDecoder to accept max_event_length parameter and apply it to StreamReadConstraints. - Added validation to ensure the maximum event length is within acceptable limits. Fixes opensearch-project#5466 * Set maxEventLength default to null for safer usage. * Set min value of maxEventLength to Jackson's default Signed-off-by: Pallempati Saketh <pallempati.saketh@fmr.com>
Mamol27
pushed a commit
to Mamol27/data-prepper
that referenced
this pull request
May 6, 2025
…pensearch-project#5541) feat: Add configurable stream read constraints for JSON input codec - Introduced a configurable option for setting the maximum event length in the JSON input codec. - Updated JsonDecoder to accept max_event_length parameter and apply it to StreamReadConstraints. - Added validation to ensure the maximum event length is within acceptable limits. Fixes opensearch-project#5466 * Set maxEventLength default to null for safer usage. * Set min value of maxEventLength to Jackson's default Signed-off-by: Pallempati Saketh <pallempati.saketh@fmr.com> Signed-off-by: mamol27 <mamol27@yandex.ru>
Mamol27
pushed a commit
to Mamol27/data-prepper
that referenced
this pull request
May 6, 2025
…pensearch-project#5541) feat: Add configurable stream read constraints for JSON input codec - Introduced a configurable option for setting the maximum event length in the JSON input codec. - Updated JsonDecoder to accept max_event_length parameter and apply it to StreamReadConstraints. - Added validation to ensure the maximum event length is within acceptable limits. Fixes opensearch-project#5466 * Set maxEventLength default to null for safer usage. * Set min value of maxEventLength to Jackson's default Signed-off-by: Pallempati Saketh <pallempati.saketh@fmr.com> Signed-off-by: mamol27 <mamol27@yandex.ru>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
JsonDecoderto acceptmax_event_lengthparameter and apply it toStreamReadConstraints.Description
This change introduces a new configurable option that allows users to set the maximum event length for the JSON input codec. By setting this parameter, users can control the maximum size of JSON strings that the codec will process, ensuring that excessively large events are not processed and potentially causing issues. The
JsonDecoderhas been updated to accept themax_event_lengthparameter and apply it to theStreamReadConstraints. Additionally, validation has been added to ensure that the maximum event length is within acceptable limits.Issues Resolved
fixes #5466
Check List
The documentation issue link : opensearch-project/documentation-website#9370
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.