-
Notifications
You must be signed in to change notification settings - Fork 4.5k
core: CrossRegionExportReader: Response object is too long #25114
Description
Describe the bug
When using crossRegionReferences: true the cdk deploy fails in the reader region with Response object is too long.
Similar to #23958, but on the reader side, likely with a different root cause.
Expected Behavior
I expect crossRegionReferences: true to work, and it be possible to reference variables across stacks in different regions.
Current Behavior
When referencing a variable in a stack in a different region, it fails in the reader region with this error:
testing-cdk-cross-region-references-as-example-for-github-issue-rootstack-us-west-2 | 3/14 | 12:28:01 PM | CREATE_COMPLETE | AWS::Lambda::Function | Custom::CrossRegionExportReaderCustomResourceProvider/Handler (CustomCrossRegionExportReaderCustomResourceProviderHandler46647B68)
[12:28:03] Stack testing-cdk-cross-region-references-as-example-for-github-issue-rootstack-us-west-2 has an ongoing operation in progress and is not stable (CREATE_IN_PROGRESS)
testing-cdk-cross-region-references-as-example-for-github-issue-rootstack-us-west-2 | 3/14 | 12:28:06 PM | CREATE_IN_PROGRESS | Custom::CrossRegionExportReader | ExportsReader/Resource/Default (ExportsReader8B249524)
[12:28:08] Stack testing-cdk-cross-region-references-as-example-for-github-issue-rootstack-us-west-2 has an ongoing operation in progress and is not stable (CREATE_IN_PROGRESS)
[12:28:13] Stack testing-cdk-cross-region-references-as-example-for-github-issue-rootstack-us-west-2 has an ongoing operation in progress and is not stable (CREATE_IN_PROGRESS)
testing-cdk-cross-region-references-as-example-for-github-issue-rootstack-us-west-2 | 3/14 | 12:28:15 PM | CREATE_FAILED | Custom::CrossRegionExportReader | ExportsReader/Resource/Default (ExportsReader8B249524) Response object is too long.
new CustomResource (/path/to/code/node_modules/aws-cdk-lib/core/lib/custom-resource.ts:135:21)
\_ new ExportReader (/path/to/code/node_modules/aws-cdk-lib/core/lib/custom-resource-provider/cross-region-export-providers/export-reader-provider.ts:62:27)
\_ Function.getOrCreate (/path/to/code/node_modules/aws-cdk-lib/core/lib/custom-resource-provider/cross-region-export-providers/export-reader-provider.ts:29:9)
\_ ExportWriter.addToExportReader (/path/to/code/node_modules/aws-cdk-lib/core/lib/custom-resource-provider/cross-region-export-providers/export-writer-provider.ts:122:39)
\_ ExportWriter.exportValue (/path/to/code/node_modules/aws-cdk-lib/core/lib/custom-resource-provider/cross-region-export-providers/export-writer-provider.ts:114:17)
\_ createCrossRegionImportValue (/path/to/code/node_modules/aws-cdk-lib/core/lib/private/refs.ts:240:33)
\_ resolveValue (/path/to/code/node_modules/aws-cdk-lib/core/lib/private/refs.ts:121:12)
\_ resolveValue (/path/to/code/node_modules/aws-cdk-lib/core/lib/private/refs.ts:105:12)
\_ resolveReferences (/path/to/code/node_modules/aws-cdk-lib/core/lib/private/refs.ts:34:24)
\_ prepareApp (/path/to/code/node_modules/aws-cdk-lib/core/lib/private/prepare-app.ts:30:20)
\_ synthesize (/path/to/code/node_modules/aws-cdk-lib/core/lib/private/synthesis.ts:45:13)
\_ App.synth (/path/to/code/node_modules/aws-cdk-lib/core/lib/stage.ts:217:33)
\_ process.<anonymous> (/path/to/code/node_modules/aws-cdk-lib/core/lib/app.ts:195:45)
\_ Object.onceWrapper (node:events:628:26)
\_ process.emit (node:events:513:28)
\_ process.emit (node:domain:489:12)
\_ process.emit.sharedData.processEmitHook.installedValue [as emit] (/path/to/code/node_modules/@cspotcode/source-map-support/source-map-support.js:745:40)
testing-cdk-cross-region-references-as-example-for-github-issue-rootstack-us-west-2 | 3/14 | 12:28:16 PM | ROLLBACK_IN_PROGRESS | AWS::CloudFormation::Stack | testing-cdk-cross-region-references-as-example-for-github-issue-rootstack-us-west-2 The following resource(s) failed to create: [ExportsReader8B249524]. Rollback requested by user.
This is using Nested Stacks, and long stack names, with multiple cross-region references.
Reproduction Steps
Node.js: 18.15.0
CDK bin and lib: 2.73.0
TypeScript: 5.0.4
Command: npx cdk deploy --all --require-approval=never --verbose --debug --app 'npx ts-node app.ts'
app.ts:
#!/usr/bin/env npx ts-node
import * as awscdk from 'aws-cdk-lib'
const ACCOUNT_ID = '111222333444'
const STACK_PREFIX = 'testing-cdk-cross-region-references-as-example-for-github-issue'
// 8 or less works, 9 or more fails with "Response object is too long"
const NUMBER_OF_KEYS_TO_CREATE = 9
function createRootStack (app: awscdk.App, region: string): awscdk.Stack {
const stackName = `${STACK_PREFIX}-rootstack-${region}`
const props = {
env: {
account: ACCOUNT_ID,
region: region
},
crossRegionReferences: true,
synthesizer: new awscdk.LegacyStackSynthesizer()
}
const stack = new awscdk.Stack(app, stackName, props)
return stack
}
function createPrimaryKeyStack (rootStack: awscdk.Stack, keyName: string): string {
const nestedStack = new awscdk.NestedStack(rootStack, `${STACK_PREFIX}-nestedstack-${keyName}`)
const keyProps = getKeyProps(keyName)
const kmsCfnKey = new awscdk.aws_kms.CfnKey(nestedStack, `${STACK_PREFIX}-key-${keyName}`, keyProps)
const aliasName = `alias/${STACK_PREFIX}-${keyName}`
const aliasProps = getAliasProps(aliasName, kmsCfnKey.attrKeyId)
new awscdk.aws_kms.CfnAlias(nestedStack, `${STACK_PREFIX}-alias-${keyName}`, aliasProps)
return kmsCfnKey.attrArn
}
function createReplicaKeyStack (rootStack: awscdk.Stack, keyName: string, primaryKeyArn: string) {
const nestedStack = new awscdk.NestedStack(rootStack, `${STACK_PREFIX}-nestedstack-${keyName}`)
const keyProps = getReplicaKeyProps(keyName, primaryKeyArn)
const kmsCfnKey = new awscdk.aws_kms.CfnReplicaKey(nestedStack, `${STACK_PREFIX}-key-${keyName}`, keyProps)
const aliasName = `alias/${STACK_PREFIX}-${keyName}`
const aliasProps = getAliasProps(aliasName, kmsCfnKey.attrKeyId)
new awscdk.aws_kms.CfnAlias(nestedStack, `${STACK_PREFIX}-alias-${keyName}`, aliasProps)
}
function getKeyPolicy (): awscdk.aws_iam.PolicyDocument {
const thisPrincipal = new awscdk.aws_iam.AccountPrincipal(ACCOUNT_ID)
const policyStatement = new awscdk.aws_iam.PolicyStatement({
sid: 'Allow all key access',
effect: awscdk.aws_iam.Effect.ALLOW,
actions: [
'kms:*'
],
resources: ['*'],
principals: [thisPrincipal]
})
const policyDocument = new awscdk.aws_iam.PolicyDocument({ statements: [policyStatement] })
return policyDocument
}
function getKeyProps (keyName: string): awscdk.aws_kms.CfnKeyProps {
const policyDocument = getKeyPolicy()
const keyProps: awscdk.aws_kms.CfnKeyProps = {
description: `Testing cross-region references in CDK - ${keyName}`,
keyPolicy: policyDocument,
multiRegion: true,
enableKeyRotation: false,
enabled: true,
keyUsage: 'ENCRYPT_DECRYPT',
pendingWindowInDays: 7
}
return keyProps
}
function getReplicaKeyProps (keyName: string, primaryKeyArn: string): awscdk.aws_kms.CfnReplicaKeyProps {
const policyDocument = getKeyPolicy()
const keyProps: awscdk.aws_kms.CfnReplicaKeyProps = {
description: `Testing cross-region references in CDK - ${keyName}`,
primaryKeyArn: primaryKeyArn,
keyPolicy: policyDocument,
enabled: true,
pendingWindowInDays: 7
}
return keyProps
}
function getAliasProps (aliasName:string, targetKeyId: string): awscdk.aws_kms.CfnAliasProps {
const aliasProps: awscdk.aws_kms.CfnAliasProps = {
aliasName: aliasName,
targetKeyId: targetKeyId
}
return aliasProps
}
function main (args: Array<string> | undefined = undefined) {
const app = new awscdk.App()
const rootStackEast = createRootStack(app, 'us-east-1')
const rootStackWest = createRootStack(app, 'us-west-2')
for (let i = 1; i <= NUMBER_OF_KEYS_TO_CREATE; i++) {
const primaryKeyArn = createPrimaryKeyStack(rootStackEast, i.toString())
createReplicaKeyStack(rootStackWest, i.toString(), primaryKeyArn)
}
}
if (require.main === module) {
main()
}At the top of app.ts the constant ACCOUNT_ID will need to be changed. The constant NUMBER_OF_KEYS_TO_CREATE can be lowered below 9 to not trigger the error.
I don't see any errors in the Lambda logs. Parameter Store values are written. And in the reader region the Parameter Store tags are set and removed, according to the Lambda logs.
Possible Solution
My guess is that it is triggered long stack names, including nested stacks creating even longer reference names.
Additional Information/Context
No response
CDK CLI Version
2.73.0 (build 43e681e)
Framework Version
No response
Node.js Version
18.15.0
OS
macOS
Language
Typescript
Language Version
TypeScript (5.0.4)
Other information
No response