Skip to content

(aws-s3-deployment): Fails to deploy when a large number of object keys are present #28579

@watersry

Description

@watersry

Describe the bug

When using the BucketDeployment for a large number of individual source files, the custom resource correctly persists them to S3, but ultimately fails the deployment due the large response send back to CloudFormation, exceeding the 4k limit for response objects.

Expected Behavior

The Cfn deployment should succeed if the BucketDeployment's underlying custom resource succeeds.

Current Behavior

Cfn fails to deploy the stack with Response object is too long despite the BucketDeployment lambda succeeding and successfully deploying all resources as expected.

Reproduction Steps

Any bucket deployment where the number of sources is sufficient to exceed the 4k response count. In our case, 50 config files succeeded to persist to s3 but failed due to the response to Cfn.

    new BucketDeployment(this, `Deployment`, {
      sources: configs.map((conf) => Source.jsonData("someKey", conf)), 
      destinationBucket: bucket,
      memoryLimit: 1024,
      prune: false,
    });

Possible Solution

The underlying custom resource provides the ability to choose an output path in the response, this could be used to truncate the response to Cfn and avoid this issue when the number of keys is sufficiently large or if a user intentionally configures it to do so.

Additional Information/Context

No response

CDK CLI Version

2.103.1

Framework Version

No response

Node.js Version

18

OS

any

Language

TypeScript

Language Version

No response

Other information

No response

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions