Skip to content

Fixed kinesis retrieval config argument passing to KCL scheduler#5272

Merged
dlvenable merged 1 commit intoopensearch-project:mainfrom
RashmiRam:kinesis-retrieval-config-issue
Jan 21, 2025
Merged

Fixed kinesis retrieval config argument passing to KCL scheduler#5272
dlvenable merged 1 commit intoopensearch-project:mainfrom
RashmiRam:kinesis-retrieval-config-issue

Conversation

@RashmiRam
Copy link
Copy Markdown
Contributor

Description

Polling retrieval config is passed to creation of KCL scheduler instead of new default retrieval config if consumer strategy is declared as polling

Issues Resolved

Resolves #5269

Check List

  • New functionality includes testing.
  • New functionality has a documentation issue. Please link to it in this PR.
  • New functionality has javadoc added
  • Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: RashmiRam <ras.xena@gmail.com>
Copy link
Copy Markdown
Member

@dlvenable dlvenable left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @RashmiRam for this contribution!

@sb2k16 , Can you also look?

@sb2k16
Copy link
Copy Markdown
Member

sb2k16 commented Jan 10, 2025

Thank you @RashmiRam for fixing this!

@RashmiRam
Copy link
Copy Markdown
Contributor Author

@dlvenable @sb2k16 All the failing checks are related to AWS region not being set in Kinesis-source-integration tests. Can you help in verifying if the secrets are properly set?

@dlvenable dlvenable merged commit 9c61e03 into opensearch-project:main Jan 21, 2025
@dlvenable
Copy link
Copy Markdown
Member

Thank you @RashmiRam for the contribution!

chenqi0805 pushed a commit to chenqi0805/data-prepper that referenced this pull request Apr 2, 2025
…nsearch-project#5272)

Signed-off-by: RashmiRam <ras.xena@gmail.com>
Signed-off-by: George Chen <qchea@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Kinesis source doesn't pass the given polling retrieval config to underlying KCL

3 participants