Skip to content

s3|ecr: auto-delete-[objects|images] breaks on cloudformation rollback (suspected) #27199

@kaizencc

Description

@kaizencc

Based on the way the custom resource is implemented, it is likely that unexpected behavior happens on Cloudformation rollback, i.e. the custom resource will prematurely delete the objects.

Consider the following scenario:

UPDATE target resource (replacement, creates a new resource)
UPDATE custom resource (old -> new, objects in old bucket are deleted)
(...stuff happens...)
ERROR, triggers a rollback
UPDATE custom resource (new -> old)
DELETE target resource (deletes the new resource, remembers the existing one)

We will have deleted objects in the bucket that has been rolled back to in this scenario.


The correct way to handle this is what was done in synthetics:

async function onUpdate(event: AWSLambda.CloudFormationCustomResourceEvent) {
const updateEvent = event as AWSLambda.CloudFormationCustomResourceUpdateEvent;
const newCanaryName = updateEvent.ResourceProperties?.CanaryName;
// If the name of the canary has changed, CloudFormation will delete the canary
// and create a new one with the new name. Returning a PhysicalResourceId that
// differs from the event's PhysicalResourceId will trigger a `Delete` event
// for this custom resource. Here, if `newCanaryName` differs from `event.PhysicalResourceId`
// then this will trigger a `Delete` event.
return { PhysicalResourceId: newCanaryName };
}

As opposed to

async function onUpdate(event: AWSLambda.CloudFormationCustomResourceEvent) {
const updateEvent = event as AWSLambda.CloudFormationCustomResourceUpdateEvent;
const oldBucketName = updateEvent.OldResourceProperties?.BucketName;
const newBucketName = updateEvent.ResourceProperties?.BucketName;
const bucketNameHasChanged = newBucketName != null && oldBucketName != null && newBucketName !== oldBucketName;
/* If the name of the bucket has changed, CloudFormation will try to delete the bucket
and create a new one with the new name. So we have to delete the contents of the
bucket so that this operation does not fail. */
if (bucketNameHasChanged) {
return onDelete(oldBucketName);
}
}

Metadata

Metadata

Assignees

Labels

@aws-cdk/aws-ecrRelated to Amazon Elastic Container Registry@aws-cdk/aws-s3Related to Amazon S3@aws-cdk/custom-resourcesRelated to AWS CDK Custom ResourcesbugThis issue is a bug.effort/smallSmall work item – less than a day of effortp1

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions