-
Notifications
You must be signed in to change notification settings - Fork 4.5k
Description
What is the problem?
CDK asset publishing doesn't correctly assume the target asset publishing role in certain cases and fails to deploy S3 artifacts.
The issue seems to have surfaced with the release of cdk 1.135.0 but was possibly introduced in earlier versions.
Initial investigation suggests this line isn't passing the stack's asset publishing role and instead making API calls as the base role.
When v1.135 is used to deploy against an account bootstrapped with stack < v9, this code branch will trigger the above account() call which causes this.host.aborted to become true and this line to never be reached.
Reproduction Steps
- Have role in
AccountA - Bootstrap
AccountBwith stack v8 ( cdk <= 1.134.0 ), and--trustrole inAccountA - Create CDK application with some S3 asset (lambda functions or similar), point environment to
AccountB - Synth application with
@aws-cdk/core:newStyleStackSynthesis=true - npx aws-cdk@1.135.0 deploy --debug -vvvvv
What did you expect to happen?
CDK deploy assumes the asset publishing role in AccountB and correctly uploads artifacts
What actually happened?
fail: Need to perform AWS calls for account <AccountB>, but the current credentials are for <AccountA>
CDK CLI Version
1.135.0
Framework Version
No response
Node.js Version
v14.16.0
OS
MacOS
Language
Typescript
Language Version
No response
Other information
Complete deploy log:
CDK toolkit version: 1.135.0 (build 3825f59)
Command line arguments: {
_: [ 'deploy' ],
debug: true,
v: 5,
verbose: 5,
'outputs-file': 'cdk-outputs.json',
O: 'cdk-outputs.json',
outputsFile: 'cdk-outputs.json',
app: 'cdk.out/',
a: 'cdk.out/',
'require-approval': 'any-change',
requireApproval: 'any-change',
lookups: true,
'ignore-errors': false,
ignoreErrors: false,
json: false,
j: false,
ec2creds: undefined,
i: undefined,
'version-reporting': undefined,
versionReporting: undefined,
'path-metadata': true,
pathMetadata: true,
'asset-metadata': true,
assetMetadata: true,
'role-arn': undefined,
r: undefined,
roleArn: undefined,
staging: true,
'no-color': false,
noColor: false,
all: false,
'build-exclude': [],
E: [],
buildExclude: [],
ci: false,
execute: true,
force: false,
f: false,
parameters: [ {} ],
'previous-parameters': true,
previousParameters: true,
'$0': '../.npm/_npx/69094/bin/cdk',
STACKS: [ 'cdk-issue-debug' ],
'S-t-a-c-k-s': [ 'cdk-issue-debug' ]
}
cdk.json: {
"app": "npx ts-node -P tsconfig.json --prefer-ts-exts src/main.ts",
"context": {
"@aws-cdk/core:newStyleStackSynthesis": "true",
"aws-cdk:enableDiffNoFail": true,
"@aws-cdk/aws-apigateway:usagePlanKeyOrderInsensitiveId": true,
"@aws-cdk/core:enableStackNameDuplicates": true,
"@aws-cdk/core:stackRelativeExports": true,
"@aws-cdk/aws-ecr-assets:dockerIgnoreSupport": true,
"@aws-cdk/aws-secretsmanager:parseOwnedSecretName": true,
"@aws-cdk/aws-kms:defaultKeyPolicies": true,
"@aws-cdk/aws-s3:grantWriteWithoutAcl": true,
"@aws-cdk/aws-ecs-patterns:removeDefaultDesiredCount": true,
"@aws-cdk/aws-rds:lowercaseDbIdentifier": true,
"@aws-cdk/aws-efs:defaultEncryptionAtRest": true,
"@aws-cdk/aws-lambda:recognizeVersionProps": true,
"@aws-cdk/aws-cloudfront:defaultSecurityPolicyTLSv1.2_2021": true
},
"output": "cdk.out",
"build": "npx projen bundle",
"watch": {
"include": [
"src/**/*.ts",
"test/**/*.ts"
],
"exclude": [
"README.md",
"cdk*.json",
"**/*.d.ts",
"**/*.js",
"tsconfig.json",
"package*.json",
"yarn.lock",
"node_modules"
]
},
"//": "~~ Generated by projen. To modify, edit .projenrc.js and run \"npx projen\"."
}
merged settings: {
versionReporting: true,
pathMetadata: true,
output: 'cdk.out',
context: {
'@aws-cdk/core:newStyleStackSynthesis': 'true',
'aws-cdk:enableDiffNoFail': true,
'@aws-cdk/aws-apigateway:usagePlanKeyOrderInsensitiveId': true,
'@aws-cdk/core:enableStackNameDuplicates': true,
'@aws-cdk/core:stackRelativeExports': true,
'@aws-cdk/aws-ecr-assets:dockerIgnoreSupport': true,
'@aws-cdk/aws-secretsmanager:parseOwnedSecretName': true,
'@aws-cdk/aws-kms:defaultKeyPolicies': true,
'@aws-cdk/aws-s3:grantWriteWithoutAcl': true,
'@aws-cdk/aws-ecs-patterns:removeDefaultDesiredCount': true,
'@aws-cdk/aws-rds:lowercaseDbIdentifier': true,
'@aws-cdk/aws-efs:defaultEncryptionAtRest': true,
'@aws-cdk/aws-lambda:recognizeVersionProps': true,
'@aws-cdk/aws-cloudfront:defaultSecurityPolicyTLSv1.2_2021': true
},
app: 'cdk.out/',
build: 'npx projen bundle',
watch: {
include: [ 'src/**/*.ts', 'test/**/*.ts' ],
exclude: [
'README.md',
'cdk*.json',
'**/*.d.ts',
'**/*.js',
'tsconfig.json',
'package*.json',
'yarn.lock',
'node_modules'
]
},
'//': '~~ Generated by projen. To modify, edit .projenrc.js and run "npx projen".',
debug: true,
assetMetadata: true,
requireApproval: 'any-change',
toolkitBucket: {},
staging: true,
outputsFile: 'cdk-outputs.json',
bundlingStacks: [ '*' ],
lookups: true
}
Determining if we're on an EC2 instance.
Does not look like an EC2 instance.
Toolkit stack: CDKToolkit
Setting "CDK_DEFAULT_REGION" environment variable to ap-southeast-2
Resolving default credentials
Looking up default account ID from STS
[AWS sts 200 0.181s 0 retries] getCallerIdentity({})
Default account ID: AccountA
Setting "CDK_DEFAULT_ACCOUNT" environment variable to AccountA
context: {
'@aws-cdk/core:newStyleStackSynthesis': 'true',
'aws-cdk:enableDiffNoFail': true,
'@aws-cdk/aws-apigateway:usagePlanKeyOrderInsensitiveId': true,
'@aws-cdk/core:enableStackNameDuplicates': true,
'@aws-cdk/core:stackRelativeExports': true,
'@aws-cdk/aws-ecr-assets:dockerIgnoreSupport': true,
'@aws-cdk/aws-secretsmanager:parseOwnedSecretName': true,
'@aws-cdk/aws-kms:defaultKeyPolicies': true,
'@aws-cdk/aws-s3:grantWriteWithoutAcl': true,
'@aws-cdk/aws-ecs-patterns:removeDefaultDesiredCount': true,
'@aws-cdk/aws-rds:lowercaseDbIdentifier': true,
'@aws-cdk/aws-efs:defaultEncryptionAtRest': true,
'@aws-cdk/aws-lambda:recognizeVersionProps': true,
'@aws-cdk/aws-cloudfront:defaultSecurityPolicyTLSv1.2_2021': true,
'aws:cdk:enable-path-metadata': true,
'aws:cdk:enable-asset-metadata': true,
'aws:cdk:version-reporting': true,
'aws:cdk:bundling-stacks': [ '*' ]
}
--app points to a cloud assembly, so we bypass synth
Reading existing template for stack cdk-issue-debug.
Retrieved account ID AccountA from disk cache
Assuming role 'arn:aws:iam::AccountB:role/cdk-hnb659fds-deploy-role-AccountB-ap-southeast-2'.
[AWS cloudformation 200 0.291s 0 retries] describeStacks({ StackName: 'cdk-issue-debug' })
[AWS cloudformation 200 0.175s 0 retries] getTemplate({ StackName: 'cdk-issue-debug', TemplateStage: 'Original' })
cdk-issue-debug: deploying...
Retrieved account ID AccountA from disk cache
Assuming role 'arn:aws:iam::AccountB:role/cdk-hnb659fds-deploy-role-AccountB-ap-southeast-2'.
Waiting for stack CDKToolkit to finish creating or updating...
[AWS cloudformation 200 0.212s 0 retries] describeStacks({ StackName: 'CDKToolkit' })
[AWS ssm 200 0.236s 0 retries] getParameter({ Name: '/cdk-bootstrap/hnb659fds/version' })
[0%] start: Publishing ea955e1dd13eac7a289683a4c42a8f95466ebb1e7002c2b524369eab874ebd6c:AccountB-ap-southeast-2
Retrieved account ID AccountA from disk cache
Assuming role 'arn:aws:iam::AccountB:role/cdk-hnb659fds-file-publishing-role-AccountB-ap-southeast-2'.
[0%] check: Check s3://cdk-hnb659fds-assets-AccountB-ap-southeast-2/ea955e1dd13eac7a289683a4c42a8f95466ebb1e7002c2b524369eab874ebd6c.zip
[AWS s3 200 0.183s 0 retries] getBucketLocation({ Bucket: 'cdk-hnb659fds-assets-AccountB-ap-southeast-2' })
[AWS s3 200 0.169s 0 retries] listObjectsV2({
Bucket: 'cdk-hnb659fds-assets-AccountB-ap-southeast-2',
Prefix: 'ea955e1dd13eac7a289683a4c42a8f95466ebb1e7002c2b524369eab874ebd6c.zip',
MaxKeys: 1
})
[AWS s3 403 0.147s 0 retries] getBucketEncryption({ Bucket: 'cdk-hnb659fds-assets-AccountB-ap-southeast-2' })
Call failed: getBucketEncryption({"Bucket":"cdk-hnb659fds-assets-AccountB-ap-southeast-2"}) => Access Denied (code=AccessDenied)
[50%] fail: Need to perform AWS calls for account AccountB, but the current credentials are for AccountA
[50%] start: Publishing 4ed6eb06863732b08308385e96b4da247bb35b3e8b6aca611ee88573da8a870a:AccountB-ap-southeast-2
Retrieved account ID AccountA from disk cache
[50%] check: Check s3://cdk-hnb659fds-assets-AccountB-ap-southeast-2/4ed6eb06863732b08308385e96b4da247bb35b3e8b6aca611ee88573da8a870a.json
[AWS s3 200 0.165s 0 retries] listObjectsV2({
Bucket: 'cdk-hnb659fds-assets-AccountB-ap-southeast-2',
Prefix: '4ed6eb06863732b08308385e96b4da247bb35b3e8b6aca611ee88573da8a870a.json',
MaxKeys: 1
})
[100%] fail: Need to perform AWS calls for account AccountB, but the current credentials are for AccountA
❌ cdk-issue-debug failed: Error: Failed to publish one or more assets. See the error messages above for more information.
at Object.publishAssets (/Users/msessa/.npm/_npx/69094/lib/node_modules/aws-cdk/lib/util/asset-publishing.ts:25:11)
at processTicksAndRejections (internal/process/task_queues.js:93:5)
at CloudFormationDeployments.publishStackAssets (/Users/msessa/.npm/_npx/69094/lib/node_modules/aws-cdk/lib/api/cloudformation-deployments.ts:305:7)
at CloudFormationDeployments.deployStack (/Users/msessa/.npm/_npx/69094/lib/node_modules/aws-cdk/lib/api/cloudformation-deployments.ts:202:5)
at CdkToolkit.deploy (/Users/msessa/.npm/_npx/69094/lib/node_modules/aws-cdk/lib/cdk-toolkit.ts:194:24)
at initCommandLine (/Users/msessa/.npm/_npx/69094/lib/node_modules/aws-cdk/bin/cdk.ts:267:9)
Failed to publish one or more assets. See the error messages above for more information.
Error: Failed to publish one or more assets. See the error messages above for more information.
at Object.publishAssets (/Users/msessa/.npm/_npx/69094/lib/node_modules/aws-cdk/lib/util/asset-publishing.ts:25:11)
at processTicksAndRejections (internal/process/task_queues.js:93:5)
at CloudFormationDeployments.publishStackAssets (/Users/msessa/.npm/_npx/69094/lib/node_modules/aws-cdk/lib/api/cloudformation-deployments.ts:305:7)
at CloudFormationDeployments.deployStack (/Users/msessa/.npm/_npx/69094/lib/node_modules/aws-cdk/lib/api/cloudformation-deployments.ts:202:5)
at CdkToolkit.deploy (/Users/msessa/.npm/_npx/69094/lib/node_modules/aws-cdk/lib/cdk-toolkit.ts:194:24)
at initCommandLine (/Users/msessa/.npm/_npx/69094/lib/node_modules/aws-cdk/bin/cdk.ts:267:9)