Just thinking: the other way around might be easier.

Why don't we download the current template from the Stack, and then add the resources we're trying to import? Seems safer than dropping resources from the target template.

So the workflow looks a bit like:

• Take the new template, identify candidate resources to import by looking at cdk diff
• Obtain the import map by a combination of heuristics and user input. Include the resource definitions of the resources in the import map

Then:

• Download the old template
• Add the resources from the previous step to the template
• (Potentially: check the integrity of the template)
• Do an IMPORT update with the import map

I wonder what we're going to do about dependencies on non-imported resources here... but I also don't have a good intuition for where and how those are going to crop up. I guess the best path forward is roll this feature out and seeing where it's going to creak 😅

Yes, I am happy to revert the logic here, although it should produce exactly the same template, as:

• we already download the current template and perform the diff
• for any modification (besides CDKMetadata that we copy over) we fail so non-addition difference will never get to the import action
• for any newly defined resource, we decide whether it is going to be imported (we keep it) or not (we remove it) ... which would be the same other way around: if it's going to be imported we add it, otherwise we skip it

But now thinking about it, the above applies only to the Resources section of the template. I will double-check what if Parameters or Outputs change - this can add some more complexity. Imagine a resource properties of a resource-to-be-imported reference a parameter that did not exist in the previous template.

Exactly. It just feels to me that taking a known-good template and doing a very concrete set of operations to it that we thought about beforehand is going to be safer than taking an unknown, new template, and hoping we remember to take out all the unintended modifications.

On the other hand, some Parameters and Mappings might be necessary to support the new resource?

rephrased to reflect this

This is worth calling out in the above section as well.

I assume this happens before doing the actual import?

I wonder if running a drift detection afterwards might be good enough for a start?

The CCAPI will require us to do a JSON diff, and I'm not sure it's complete enough. Have you experimented with this?

I have experimented with this and it works quite nicely. The greatest benefit is that CCAPI uses the same resource type nomenclature as CloudFormation so we can simply pass the type to the API call. There are two issues I am still looking at though:

Firstly, I need to understand how the "import identifiers" of a resource relate to "primary" and "secondary" indentifiers that CCAPI uses. While for CFN import, explicit identifier key has to be provided (e.g. BucketName for S3 Bucket, or VpcId for VPC), with CCAPI call you always provide Identifier key that has a value based on the type - it can be either primary or secondary identifier. My plan is to get all resource schemas from CFN resource catalog, extract primary/secondary identifiers info, then run get-template-summary for a template containing all resource types and do some analysis on that. It will be somewhat time consuming though, not sure when I can get to that.

Secondly, CCAPI returns all resource properties, but some of them are not mandatory in CFN templates and if not specified, will be set to default values. So in order to do the comparison correctly, I would need to know the default values of all optional properties of all resources, didn't find a way to do that yet (in best-effort approach, we could only compare properties explicitly set in CFN).

I agree with your suggestion: If it is OK to get this merged without this extra validation, let's do it and either run drift detection automatically or display a post import message to tell user to run it. And leave the CCAPI extra validation for the next iteration.

For now, I have updated the RFC to omit the CCAPI validations and I have added a "Possible future expansions" section with some thoughts on how to leverage CCAPI to provide a better user experience in future - perhaps beyond simple validations.

Does that mean everything is going to be implemented in 1 big PR?

Is there a logical order in which we can deliver things incrementally?

What's the aftercare/future extensions story?

Not sure about the lifecycle of the code post-merge, I guess that is for you and your team to decide - I can't guarantee I will be able to support this afterwards, although I will certainly watch the space for some time as I will be interested how is my so-far-biggest contribution doing :)

It might make sense to split the feature into multiple stages (and in fact, I believe the first stage might be pretty much ready now), I would take your guidance on how to split it.

I'm thinking we slice the elephant here. We'll say something like:

• First we implement interactive import with current credentials
• Then noninteractive import
• Then support using the lookup role
• Then support CCAPI validations

I see you put some in the extensions piece, that's a good place too.

modified as suggested

Does this require additional permissions for the cdk-deploy role in the bootstrap stack? Or are we going to use the cdk-lookup role for this?

That's a very good point. Not sure but I will have a look

So here I am actually using an existing SdkProvider that is initialized in the main file of CDK CLI (same as the cdk deploy command is using). If I understand the code right, it will only ever use the AWS CLI credentials - it doesn't seem to be assuming any role from the bootstrap stack or so:

https://github.com/aws/aws-cdk/blob/master/packages/aws-cdk/bin/cdk.ts#L212-L219

So I don't think the bootstrap stack needs to be updated, am I correct?

It probably will eventually, but for now I suppose the trivial solution will do.

Can you add a note in the RFC (execution plan probably) that we're going to initially support "current" credentials, and later on support using bootstrapped roles?

(For the part where we determine the identifiers to be used, so also when --create-resource-mapping is used. For the part where we do the actual import, we should always be using the CDK deploy role if available)

done

This is good to identify as a use case. I like that you thought about it.

But in practice, we're going to have (in AWS) 20+ copies of every app deployed... so if you really want to refactor something across 20 copies, we're going to need more automation to take care of this.

In any case -- not for this RFC, we will deal with that when we get to it! Perhaps people will be clever and build something on top of the structured input file.

added in the "possible future expansions" section

I think mass refactoring support (across multiple stacks) would be a great extension we could write down here.

It's going to require some additional tooling I think.

done