As we progress in our design for an end-to-end developer experience for cloud applications, and start to venture into more complex use cases, there's a common pattern that keeps emerging, where more than a single artifact is needed in order to deploy cloud applications. Moreover, these artifacts interact with each other.

Some examples:

1. Runtime code: a zip file with a Lambda handler, a Docker image for a container-based app, an AMI for an EC2 fleet, etc.
2. Nested stacks: nested stacks are CloudFormation resources that point to other templates and deploy them in a single transaction.
3. Apps with multiple, related stacks: real-world cloud apps normally consist of more than a single stack, and there are cross references between these stacks.
4. Phased deployments: we know that there are use cases where deployments of a stack must happen in phases (for example, data migration might require deploying a stack that performs the migration and only after it's done, deploy a stack that removes the old resources). Other examples are limitations in CloudFormation such as Bucket Notifications #201.

In all of these cases, we still want the developer and CI/CD workflows to operate at the app level. What does it mean? It means that if a construct defines a Lambda function that uses some runtime code, when a developer invokes cdk deploy, the runtime code should be uploaded to an S3 bucket and the CloudFormation template should reference the correct bucket/key (and also make sure Lambda has permissions to read from the bucket). This should also seamlessly work within a CI/CD pipeline.

Since the CDK's unit of reuse is a construct library, any solution must not look at the problem from the app's perspective, but from a construct perspective. It should be possible for a construct library to encapsulate runtime code, nested stacks, etc. Then, when this library is consumed, the workflow above should continue to work exactly in the same way.

Design approach

At the moment, synthesis is actually performed at the App level and is tailored to produce a single artifact (CloudFormation template). The proposed design will allow any construct in the tree to participate in the synthesis process and emit arbitrary artifacts.

But it is not sufficient to just emit multiple artifacts. We need to model their interaction somehow (dependencies, data-flow, how do these artifact interact with cloud resources, etc).

Generalizing this, we effective need to have some way to describe the model for our app. If a CloudFormation template define the model for a single stack, we need a way to describe an entire cloud application.

Naturally, we should prefer a desired state configuration approach where the app model doesn't describe steps but rather the desired state, and then tools (like the CDK toolkit or CI/CD systems) can help achieve this desired state.

Let's say that the toolkit only knows how to work with app model files, which describe the desired state of an app in a format similar to CloudFormation templates:

{
  "Resources": {
    "MyLambdaCodePackage": {
      "Type": "AWS::App::Asset",
      "Properties": {
        "File": "./my-handler.zip"
      }
    },
    "MyTemplate": {
      "Type": "AWS::App::Asset",
      "Properties": {
        "File": "./my-template.json"
      }
    },
    "MyStack": {
      "Type": "AWS::App::CloudFormationStack",
      "Proeprties": {
        "TemplateURL": { "Fn::GetAtt": [ "MyTemplate", "URL" ] },
        "Parameters": {
          "MyLambdaCodeS3Bucket": { "Fn::GetAtt": [ "MyLambdaCodePackage", "Bucket" ] },
          "MyLambdaCodeS3Key": { "Fn::GetAtt": [ "MyLambdaCodePackage", "Key" ] }
        }
      }
    }
  }
}

This is not a CloudFormation template! It's an App Model file. It uses the same structure to define the desired state of an entire application. This file, together with all the artifacts synthesized from the app form a self-contained cloud app package ("cloud executable"?).

When tools read this file, they can produce a deployment plan for this app:

1. Upload the files ./my-handler.zip and ./my-template.json to an S3 bucket.
2. Deploy a CloudFormation stack. When executing the CreateStack API, use the S3 URL to specify the template URL and pass in parameters that resolve to the location of the S3 bucket and key of the Lambda runtime archive.

The power of this approach is that it is highly extensible. Anyone can implement App Resources which will participate in this process. The desired state approach deems that each resource needs to be able to be CREATED, UPDATED or DELETED, and also DIFFed against the desired state.

Implementation

Synthesis

Each construct in the tree may implement a method synthesize(workdir) which will be called during synthesis. Constructs can emit files (or symlinks) into a working directory at this time.

App Model Resources

Similarly to CloudFormation Resources, we can defined constructs that represent app model resources (AppStack, AppAsset). Similarly to how CloudFormation resources are implemented, these constructs will implement a method toAppModel() which will return an app model JSON chunk to be merged into the complete app model.

The App construct is now a simple construct, in it's synthesis() method, it will collect all app resources from the tree, merge them together and emit an app-model.json (or index.json) into the working directory.

The toolkit will expect to always find an index.json file in the working directory. It will read the file and form a deployment plan (calculate dependency graph based on references). Then, it will execute the deployment plan.

The toolkit can either deploy the entire app or only a subset of the resources, in which case it can also deploy any dependencies of this set.

Each app resource will have a "provider" implemented in the toolkit via an extensible framework. Providers will implement the following operations:

1. Diff(desired-state) the desired state against the current deployed state
2. Create/Update(desired-state) the resource to reach the desired state (idempotent)
3. Delete the resource
4. GetAtt(name) return the runtime value from the deployed resource (i.e. the actual bucket name)

In the normal flow, the toolkit will simply invoke the create/update operation in topological order (and concurrently if possible). It will supply the contents of the Properties object which represents the desired state. If a property includes a reference to another resource (via Fn::GetAtt), it will replace the token with the value from the other resource.

TODO

List of use cases we should tackle for this new design:

• Transactionality: what happens if a deployment task fails? Should/can we roll it back? I guess it depends on the boundaries of a transaction. Perhaps we should make that part of the solution and for certain tasks (like S3), just "no-op". For tasks that support it, we can trigger an actual rollback if the entire transaction fails.
• Environments: how do environments play here? A stack (and also an S3 object) should be bound to an environment.
• Cross stack refs: how do we implement cross stack/cross env references in this model?
• Docker images
• Runtime values (allowing runtime code to use resolved attributes of infrastructure resources)
• Environmental context
• Construct metadata - where does it go now?
• Vending runtime client libraries for constructs: it should be possible to supply reusable runtime code for a construct. This could be as simple as an API client generated from Swagger definitions or more complex as a Lambda handler base class which requires you to implement a bunch of methods and does magic for you. At any rate, this is something that needs to be somehow cross-language.
• Packaging (see @kiiadi comment below)