Skip to content

Add bucket_owner option to S3 DLQ#5519

Merged
graytaylor0 merged 1 commit intoopensearch-project:mainfrom
graytaylor0:DlqBucketOwnerShip
Mar 18, 2025
Merged

Add bucket_owner option to S3 DLQ#5519
graytaylor0 merged 1 commit intoopensearch-project:mainfrom
graytaylor0:DlqBucketOwnerShip

Conversation

@graytaylor0
Copy link
Copy Markdown
Member

@graytaylor0 graytaylor0 commented Mar 12, 2025

Description

This adds a bucket_owner configuration option to the S3 DLQ and passes this bucket owner to expectedBucketOwner in the PutObject request.

dlq:
  s3:
    bucket_owner: "123456789012"

Issues Resolved

Resolves #5498

Check List

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Taylor Gray <tylgry@amazon.com>
private void doWrite(final List<DlqObject> dlqObjects, final String pipelineName, final String pluginId) throws IOException {
final PutObjectRequest putObjectRequest = PutObjectRequest.builder()
.bucket(bucket)
.expectedBucketOwner(bucketOwner)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it backward compatible if bucketOwner is none?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes you can pass null here and it won't apply a bucket owner

@graytaylor0 graytaylor0 merged commit 5db4331 into opensearch-project:main Mar 18, 2025
45 of 47 checks passed
chenqi0805 pushed a commit to chenqi0805/data-prepper that referenced this pull request Apr 2, 2025
Signed-off-by: Taylor Gray <tylgry@amazon.com>
chenqi0805 pushed a commit to chenqi0805/data-prepper that referenced this pull request Apr 2, 2025
Signed-off-by: Taylor Gray <tylgry@amazon.com>
Signed-off-by: George Chen <qchea@amazon.com>
amdhing pushed a commit to amdhing/data-prepper that referenced this pull request Apr 16, 2025
Signed-off-by: Taylor Gray <tylgry@amazon.com>
Davidding4718 pushed a commit to Davidding4718/data-prepper that referenced this pull request Apr 25, 2025
Signed-off-by: Taylor Gray <tylgry@amazon.com>
Davidding4718 pushed a commit to Davidding4718/data-prepper that referenced this pull request Apr 25, 2025
Signed-off-by: Taylor Gray <tylgry@amazon.com>
Mamol27 pushed a commit to Mamol27/data-prepper that referenced this pull request May 6, 2025
Signed-off-by: Taylor Gray <tylgry@amazon.com>
Signed-off-by: mamol27 <mamol27@yandex.ru>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

S3 DLQ should pass expected bucket owner to PutObject request

3 participants