Skip to content

Commit 05c2b67

Browse files
mergify[bot]mergify[bot]
authored
Merge branch 'master' into apigw-sfn-method
2 parents 37e694d + 01b538e commit 05c2b67

11 files changed

Lines changed: 402 additions & 17 deletions

File tree

‎.github/PULL_REQUEST_TEMPLATE.md‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,10 +3,15 @@
33

44
### All Submissions:
55

6-
* [ ] Have you followed the guidelines in our [Contributing guide?](../CONTRIBUTING.md)
6+
* [ ] Have you followed the guidelines in our [Contributing guide?](https://github.com/aws/aws-cdk/blob/master/CONTRIBUTING.md)
77

88
### Adding new Unconventional Dependencies:
99

10-
* [ ] This PR adds new unconventional dependencies following the process described [here](../CONTRIBUTING.md/#adding-new-unconventional-dependencies)
10+
* [ ] This PR adds new unconventional dependencies following the process described [here](https://github.com/aws/aws-cdk/blob/master/CONTRIBUTING.md/#adding-new-unconventional-dependencies)
11+
12+
### New Features
13+
14+
* [ ] Have you added the new feature to an [integration test](https://github.com/aws/aws-cdk/blob/master/INTEGRATION_TESTS.md)?
15+
* [ ] Did you use `cdk-integ` to deploy the infrastructure and generate the snapshot (i.e. `cdk-integ` without `--dry-run`)?
1116

1217
*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*

‎.github/workflows/yarn-upgrade.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ jobs:
2727
run: echo "::set-output name=dir::$(yarn cache dir)"
2828

2929
- name: Restore Yarn cache
30-
uses: actions/cache@v2.1.7
30+
uses: actions/cache@v3
3131
with:
3232
path: ${{ steps.yarn-cache.outputs.dir }}
3333
key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}

‎CONTRIBUTING.md‎

Lines changed: 21 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -234,9 +234,17 @@ Integration tests perform a few functions in the CDK code base -
234234
3. (Optionally) Acts as a way to validate that constructs set up the CloudFormation resources as expected. A successful
235235
CloudFormation deployment does not mean that the resources are set up correctly.
236236

237-
If you are working on a new feature that is using previously unused CloudFormation resource types, or involves
238-
configuring resource types across services, you need to write integration tests that use these resource types or
239-
features.
237+
**When are integration tests required?**
238+
239+
The following list contains common scenarios where we _know_ that integration tests are required.
240+
This is not an exhaustive list and we will, by default, require integration tests for all
241+
new features unless there is a good reason why one is not needed.
242+
243+
1. Adding a new feature that is using previously unused CloudFormation resource types
244+
2. Adding a new feature that is using previously unused (or untested) CloudFormation properties
245+
3. Involves configuring resource types across services (i.e. integrations)
246+
4. Adding a new supported version (e.g. a new [AuroraMysqlEngineVersion](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_rds.AuroraMysqlEngineVersion.html))
247+
5. Adding any functionality via a [Custom Resource](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.custom_resources-readme.html)
240248

241249
To the extent possible, include a section (like below) in the integration test file that specifies how the successfully
242250
deployed stack can be verified for correctness. Correctness here implies that the resources have been set up correctly.
@@ -254,6 +262,16 @@ Examples:
254262
* [integ.destinations.ts](https://github.com/aws/aws-cdk/blob/master/packages/%40aws-cdk/aws-lambda-destinations/test/integ.destinations.ts#L7)
255263
* [integ.token-authorizer.lit.ts](https://github.com/aws/aws-cdk/blob/master/packages/%40aws-cdk/aws-apigateway/test/authorizers/integ.token-authorizer.lit.ts#L7-L12)
256264

265+
**What do do if you cannot run integration tests**
266+
267+
If you are working on a PR that requires an update to an integration test and you are unable
268+
to run the `cdk-integ` tool to perform a real deployment, please call this out on the pull request
269+
so a maintainer can run the tests for you. Please **do not** run the `cdk-integ` tool with `--dry-run`
270+
or manually update the snapshot.
271+
272+
See the [integration test guide](./INTEGRATION_TESTS.md) for a more complete guide on running
273+
CDK integration tests.
274+
257275
#### yarn watch (Optional)
258276

259277
We've added a watch feature to the CDK that builds your code as you type it. Start this by running `yarn watch` for

‎INTEGRATION_TESTS.md‎

Lines changed: 225 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,225 @@
1+
# Integration Tests
2+
3+
This document describes the purpose of integration tests as well as acting as a guide
4+
on what type of changes require integrations tests and how you should write integration tests.
5+
6+
- [What are CDK Integration Tests](#what-are-cdk-integration-tests)
7+
- [When are integration tests required](#when-are-integration-tests-required)
8+
- [How to write Integration Tests](#how-to-write-integration-tests)
9+
- [Creating a test](#creating-a-test)
10+
- [New L2 Constructs](#new-l2-constructs)
11+
- [Existing L2 Constructs](#existing-l2-constructs)
12+
- [Assertions](#assertions)
13+
14+
## What are CDK Integration Tests
15+
16+
All Construct libraries in the CDK code base have integration tests that serve to -
17+
18+
1. Acts as a regression detector. It does this by running `cdk synth` on the integration test and comparing it against
19+
the `*.expected.json` file. This highlights how a change affects the synthesized stacks.
20+
2. Allows for a way to verify if the stacks are still valid CloudFormation templates, as part of an intrusive change.
21+
This is done by running `yarn integ` which will run `cdk deploy` across all of the integration tests in that package.
22+
If you are developing a new integration test or for some other reason want to work on a single integration test
23+
over and over again without running through all the integration tests you can do so using
24+
`yarn integ integ.test-name.js` .Remember to set up AWS credentials before doing this.
25+
3. (Optionally) Acts as a way to validate that constructs set up the CloudFormation resources as expected.
26+
A successful CloudFormation deployment does not mean that the resources are set up correctly.
27+
28+
29+
## When are Integration Tests Required
30+
31+
The following list contains common scenarios where we _know_ that integration tests are required.
32+
This is not an exhaustive list and we will, by default, require integration tests for all
33+
new features unless there is a good reason why one is not needed.
34+
35+
**1. Adding a new feature that is using previously unused CloudFormation resource types**
36+
For example, adding a new L2 construct for an L1 resource. There should be a new integration test
37+
to test that the new L2 successfully creates the resources in AWS.
38+
39+
**2. Adding a new feature that is using previously unused (or untested) CloudFormation properties**
40+
For example, there is an existing L2 construct for a CloudFormation resource and you are adding
41+
support for a new property. This could be either a new property that has been added to CloudFormation
42+
or an existing property that the CDK did not have coverage for. You should either update and existing
43+
integration test to cover this new property or create a new test.
44+
45+
Sometimes the CloudFormation documentation is incorrect or unclear on the correct way to configure
46+
a property. This can lead to introducing new features that don't actually work. Creating
47+
an integration test for the new feature can ensure that it works and avoid unnecessary bugs.
48+
49+
**3. Involves configuring resource types across services (i.e. integrations)**
50+
For example, you are adding functionality that allows for service x to integrate with service y.
51+
A good example of this is the [aws-stepfunctions-tasks](./packages/@aws-cdk/aws-stepfunctions-tasks) or
52+
[aws-apigatewayv2-integrations](./packages/@aws-cdk/aws-apigatewayv2-integrations) modules. Both of these
53+
have L2 constructs that provide functionality to integrate services.
54+
55+
Sometimes these integrations involve configuring/formatting json/vtl or some other type of data.
56+
For these types of features it is important to create an integration test that not only validates
57+
that the infrastructure deploys successfully, but that the intended functionality works. This could
58+
mean deploying the integration test and then manually making an HTTP request or invoking a Lambda function.
59+
60+
**4. Adding a new supported version (e.g. a new [AuroraMysqlEngineVersion](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_rds.AuroraMysqlEngineVersion.html))**
61+
Sometimes new versions introduce new CloudFormation properties or new required configuration.
62+
For example Aurora MySQL version 8 introduced a new parameter and was not compatible with the
63+
existing parameter (see [#19145](https://github.com/aws/aws-cdk/pull/19145)).
64+
65+
**5. Adding any functionality via a [Custom Resource](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.custom_resources-readme.html)**
66+
Custom resources involve non-standard functionality and are at a higher risk of introducing bugs.
67+
68+
## How to write Integration Tests
69+
70+
This section will detail how to write integration tests, how they are executed and how to ensure
71+
you have good test coverage.
72+
73+
### Creating a Test
74+
75+
An integration tests is any file located in the `test/` directory that has a name that starts with `integ.`
76+
(e.g. `integ.*.ts`).
77+
78+
To create a new integration test, first create a new file, for example `integ.my-new-construct.ts`.
79+
The contents of this file should be a CDK app. For example, a very simple integration test for a
80+
Lambda Function would look like this:
81+
82+
_integ.lambda.ts_
83+
```ts
84+
import * as iam from '@aws-cdk/aws-iam';
85+
import * as cdk from '@aws-cdk/core';
86+
import * as lambda from '../lib';
87+
88+
const app = new cdk.App();
89+
90+
const stack = new cdk.Stack(app, 'aws-cdk-lambda-1');
91+
92+
const fn = new lambda.Function(stack, 'MyLambda', {
93+
code: new lambda.InlineCode('foo'),
94+
handler: 'index.handler',
95+
runtime: lambda.Runtime.NODEJS_10_X,
96+
});
97+
98+
app.synth();
99+
```
100+
101+
To run the test you would run:
102+
103+
*Note - filename must be `*.js`*
104+
```
105+
npm run cdk-integ integ.lambda.js
106+
```
107+
108+
This will:
109+
1. Synthesize the CDK app
110+
2. `cdk deploy` to your AWS account
111+
3. `cdk destroy` to delete the stack
112+
4. Save a snapshot of the synthed CloudFormation template to `integ.lambda.expected.json`
113+
114+
Now when you run `npm test` it will synth the integ app and compare the result with the snapshot.
115+
If the snapshot has changed the same process must be followed to update the snapshot.
116+
117+
### New L2 Constructs
118+
119+
When creating a new L2 construct (or new construct library) it is important to ensure you have a good
120+
coverage base from which future contributions can build on.
121+
122+
Some general rules to follow are:
123+
124+
- **1 test with all default values**
125+
One test for each L2 that only populates the required properties. For a Lambda Function this would look like:
126+
127+
```ts
128+
new lambda.Function(this, 'Handler', {
129+
code,
130+
handler,
131+
runtime,
132+
});
133+
```
134+
135+
- **1 test with all values provided**
136+
One test for each L2 that populates non-default properties. Some of this will come down to judgement, but this should
137+
be based on major functionality. For example, when testing a Lambda Function there are 37 (*at the time of this writing) different
138+
input parameters. Some of these can be tested together and don't represent large pieces of functionality,
139+
while others do.
140+
141+
For example, the test for a Lambda Function might look like this. For most of these properties we are probably fine
142+
testing them together and just testing one of their values. For example we don't gain much by testing a bunch of
143+
different `memorySize` settings, as long as we test that we can `set` the memorySize then we should be good.
144+
145+
```ts
146+
new lambda.Function(this, 'Handler', {
147+
code,
148+
handler,
149+
runtime,
150+
architecture,
151+
description,
152+
environment,
153+
environmentEncryption,
154+
functionName,
155+
initialPolicy,
156+
insightsVersion,
157+
layers,
158+
maxEventAge,
159+
memorySize,
160+
reservedConcurrentExecutions,
161+
retryAttempts,
162+
role,
163+
timeout,
164+
tracing,
165+
});
166+
```
167+
168+
Other parameters might represent larger pieces of functionality and might create other resources for us or configure
169+
integrations with other services. For these it might make sense to split them out into separate tests so it is easier
170+
to reason about them.
171+
172+
A couple of examples would be
173+
(you could also mix in different configurations of the above parameters with each of these):
174+
175+
_testing filesystems_
176+
```ts
177+
new lambda.Function(this, 'Handler', {
178+
filesystem,
179+
});
180+
```
181+
182+
_testing event sources_
183+
```ts
184+
new lambda.Function(this, 'Handler', {
185+
events,
186+
});
187+
```
188+
189+
_testing VPCs_
190+
```ts
191+
new lambda.Function(this, 'Handler', {
192+
securityGroups,
193+
vpc,
194+
vpcSubnets,
195+
});
196+
```
197+
198+
### Existing L2 Constructs
199+
200+
Updating an existing L2 Construct could consist of:
201+
202+
1. **Adding coverage for a new (or previously uncovered) CloudFormation property.**
203+
In this case you would want to either add this new property to an existing integration test or create a new
204+
integration test. A new integration test is preferred for larger update (e.g. adding VPC connectivity, etc).
205+
206+
2. **Updating functionality for an existing property.**
207+
In this case you should first check if you are already covered by an existing integration test. If not, then you would follow the
208+
same process as adding new coverage.
209+
210+
3. **Changing functionality that affects asset bundling**
211+
Some constructs deal with asset bundling (i.e. `aws-lambda-nodejs`, `aws-lambda-python`, etc). There are some updates that may not
212+
touch any CloudFormation property, but instead change the way that code is bundled. While these types of changes may not require
213+
a change to an integration test, you need to make sure that the integration tests and assertions are rerun.
214+
215+
An example of this would be making a change to the way `aws-lambda-nodejs` bundles Lambda code. A couple of things could go wrong that would
216+
only be caught by rerunning the integration tests.
217+
218+
1. The bundling commands are only running when performing a real synth (not part of unit tests). Running the integration test confirms
219+
that the actual bundling was not broken.
220+
2. When deploying Lambda Functions, CloudFormation will only update the Function configuration with the new code,
221+
but it will not validate that the Lambda function can be invoked. Because of this, it is important to rerun the integration test
222+
to deploy the Lambda Function _and_ then rerun the assertions to ensure that the function can still be invoked.
223+
224+
### Assertions
225+
...Coming soon...

‎packages/@aws-cdk/aws-iot/README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,7 @@ Import it into your code:
3636

3737
```ts nofixture
3838
import * as iot from '@aws-cdk/aws-iot';
39+
import * as actions from '@aws-cdk/aws-iot-actions-alpha';
3940
```
4041

4142
## `TopicRule`

‎packages/@aws-cdk/custom-resources/README.md‎

Lines changed: 14 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -354,7 +354,7 @@ This sample demonstrates the following concepts:
354354

355355
### Customizing Provider Function name
356356

357-
In multi-account environments or when the custom resource may be re-utilized across several
357+
In multi-account environments or when the custom resource may be re-utilized across several
358358
stacks it may be useful to manually set a name for the Provider Function Lambda and therefore
359359
have a predefined service token ARN.
360360

@@ -401,9 +401,19 @@ the `installLatestAwsSdk` prop to `false`.
401401
You must provide the `policy` property defining the IAM Policy that will be applied to the API calls.
402402
The library provides two factory methods to quickly configure this:
403403

404-
* **`AwsCustomResourcePolicy.fromSdkCalls`** - Use this to auto-generate IAM Policy statements based on the configured SDK calls.
405-
Note that you will have to either provide specific ARN's, or explicitly use `AwsCustomResourcePolicy.ANY_RESOURCE` to allow access to any resource.
406-
* **`AwsCustomResourcePolicy.fromStatements`** - Use this to specify your own custom statements.
404+
* **`AwsCustomResourcePolicy.fromSdkCalls`** - Use this to auto-generate IAM
405+
Policy statements based on the configured SDK calls. Keep two things in mind
406+
when using this policy:
407+
* This policy variant assumes the IAM policy name has the same name as the API
408+
call. This is true in 99% of cases, but there are exceptions (for example,
409+
S3's `PutBucketLifecycleConfiguration` requires
410+
`s3:PutLifecycleConfiguration` permissions, Lambda's `Invoke` requires
411+
`lambda:InvokeFunction` permissions). Use `fromStatements` if you want to
412+
do a call that requires different IAM action names.
413+
* You will have to either provide specific ARNs, or explicitly use
414+
`AwsCustomResourcePolicy.ANY_RESOURCE` to allow access to any resource.
415+
* **`AwsCustomResourcePolicy.fromStatements`** - Use this to specify your own
416+
custom statements.
407417

408418
The custom resource also implements `iam.IGrantable`, making it possible to use the `grantXxx()` methods.
409419

‎packages/@aws-cdk/custom-resources/lib/aws-custom-resource/aws-custom-resource.ts‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -199,6 +199,13 @@ export class AwsCustomResourcePolicy {
199199
*
200200
* Each SDK call with be translated to an IAM Policy Statement in the form of: `call.service:call.action` (e.g `s3:PutObject`).
201201
*
202+
* This policy generator assumes the IAM policy name has the same name as the API
203+
* call. This is true in 99% of cases, but there are exceptions (for example,
204+
* S3's `PutBucketLifecycleConfiguration` requires
205+
* `s3:PutLifecycleConfiguration` permissions, Lambda's `Invoke` requires
206+
* `lambda:InvokeFunction` permissions). Use `fromStatements` if you want to
207+
* do a call that requires different IAM action names.
208+
*
202209
* @param options options for the policy generation
203210
*/
204211
public static fromSdkCalls(options: SdkCallsPolicyOptions) {

‎packages/@aws-cdk/lambda-layer-awscli/layer/requirements.txt‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
awscli==1.22.73
1+
awscli==1.22.77

‎packages/aws-cdk/test/integ/helpers/test-helpers.ts‎

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -15,14 +15,18 @@ export function integTest(
1515
timeoutMillis?: number,
1616
) {
1717

18-
// Integ tests can run concurrently, and are responsible for blocking themselves if they cannot.
19-
// Because `test.concurrent` executes the test code immediately (to obtain a promise), we allow
20-
// setting the `JEST_TEST_CONCURRENT` environment variable to 'false' in order to use `test`
21-
// instead of `test.concurrent` (this is necessary when specifying a test pattern to verify).
22-
const testKind = process.env.JEST_TEST_CONCURRENT === 'false' ? test : test.concurrent;
18+
// Integ tests can run concurrently, and are responsible for blocking
19+
// themselves if they cannot. Because `test.concurrent` executes the test
20+
// code immediately, regardles of any `--testNamePattern`, this cannot be the
21+
// default: test filtering simply does not work with `test.concurrent`.
22+
// Instead, we make it opt-in only for the pipeline where we don't do any
23+
// selection, but execute all tests unconditionally.
24+
const testKind = process.env.JEST_TEST_CONCURRENT === 'true' ? test.concurrent : test;
2325
const runner = shouldSkip(name) ? testKind.skip : testKind;
2426

2527
runner(name, async () => {
28+
// eslint-disable-next-line no-console
29+
console.log(`running test ${name} using ${runner.name}`);
2630
const output = new MemoryStream();
2731

2832
output.write('================================================================\n');

0 commit comments

Comments
 (0)