aws-cdk-eks: stack fails and corrupts after upgrade to v2.35 due to EKS logging config change #21515
Description
Describe the bug
Upgrading to v2.35 (from v2.33) and deploying (just the upgrade- no other code difference) to an existing EKS cluster pushes what appears to be a change in logging configuration to the stack.
Not only does the stack fail to deploy, but the nested stack fails to rollback, staying in UPDATE_ROLLBACK_COMPLETE_CLEANUP_IN_PROGRESS state and failing the parent stack rollback. The only way to remediate is to delete the OnEventHandler and IsCompleteHandler resources, thereby corrupting the entire thing.
Expected Behavior
a stack that doesn't blow up
Current Behavior
cdk diff on EKS stack after upgrading to v2.35:
Resources
[~] Custom::AWSCDK-EKS-Cluster eks/cluster/gggg/Resource/Resource eksclustergggg4CAC641D
└─ [~] Config
└─ [~] .logging:
└─ [~] .clusterLogging:
└─ @@ -8,5 +8,9 @@
[ ] "controllerManager",
[ ] "scheduler"
[ ] ]
[+] },
[+] {
[+] "enabled": false,
[+] "types": []
[ ] }
[ ] ]
This results in a deploy failure in the ClusterResourceProviderNestedStack in the OnEventHandler custom resource:
{
"errorType": "InvalidParameterException",
"errorMessage": "No changes needed for the logging config provided",
"code": "InvalidParameterException",
"message": "No changes needed for the logging config provided",
"time": "2022-08-08T23:14:41.167Z",
"requestId": "29aa9de7-7bd6-4f3e-9db3-54777d84a714",
"statusCode": 400,
"retryable": false,
"retryDelay": 28.276972511621913,
"stack": [
"InvalidParameterException: No changes needed for the logging config provided",
" at Object.extractError (/var/runtime/node_modules/aws-sdk/lib/protocol/json.js:52:27)",
" at Request.extractError (/var/runtime/node_modules/aws-sdk/lib/protocol/rest_json.js:49:8)",
" at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:106:20)",
" at Request.emit (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:78:10)",
" at Request.emit (/var/runtime/node_modules/aws-sdk/lib/request.js:686:14)",
" at Request.transition (/var/runtime/node_modules/aws-sdk/lib/request.js:22:10)",
" at AcceptorStateMachine.runTo (/var/runtime/node_modules/aws-sdk/lib/state_machine.js:14:12)",
" at /var/runtime/node_modules/aws-sdk/lib/state_machine.js:26:10",
" at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:38:9)",
" at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:688:12)"
]
}
Full diff for reference:
Resources
[~] Custom::AWSCDK-EKS-Cluster eks/cluster/gggg/Resource/Resource eksclustergggg4CAC641D
└─ [~] Config
└─ [~] .logging:
└─ [~] .clusterLogging:
└─ @@ -8,5 +8,9 @@
[ ] "controllerManager",
[ ] "scheduler"
[ ] ]
[+] },
[+] {
[+] "enabled": false,
[+] "types": []
[ ] }
[ ] ]
[~] AWS::CloudFormation::Stack @aws-cdk--aws-eks.ClusterResourceProvider.NestedStack/@aws-cdk--aws-eks.ClusterResourceProvider.NestedStackResource awscdkawseksClusterResourceProviderNestedStackawscdkawseksClusterResourceProviderNestedStackResource9827C454
├─ [~] NestedTemplate
│ └─ [~] .Resources:
│ ├─ [~] .CDKMetadata:
│ │ └─ [~] .Properties:
│ │ └─ [~] .Analytics:
│ │ ├─ [-] v2:deflate64:H4sIAAAAAAAA/1VPy04DIRT9lu6ZqykbXbYmujHNpE3cEgrXEYeBhgsqIfy7MK0LV+dxn2cLnMP9Rn7ToPQ8WHOGckCKqE9RqplZuZy1FFZmDMJ5jeIS/E8WckIX4dCMsetdl6+9ibVVohAXkggjwa4DIw77pGaMe0l4Wwrl6d2tM28YyHjHnpNTsZNW+OOVGblAOXqL3V5x9Nao3OWVVYZqC+WEKgUT80vw6dKr/4xa2fpLyzUZNzGVKPpFBCSfgkKCFuTL6Jagn7m5lY05fnh3x+EBHjefZMwQkotmQThe8RfRyLD6QgEAAA==
│ │ └─ [+] v2:deflate64:H4sIAAAAAAAA/1VPy04DIRT9lu6Zq+nERJetiW5MM2kTt4TCdaTDQMOFVkL4d2FaF67O4z7PGvoneFyJK3VSTZ3RR8g7pIDqEIScmBHzUQluRELPrVPIz979JC5GtAF21Ria3jT50ZpYXcUz9VwQYSDYNGDUwzbKCcNWEN6XQn79ssvMJ3rSzrK3aGVopBb+eGFazJD3zmCzFxyc0TI1eWOFoVxDPqCMXof07l08t+o/oxS2/FJzjdqOTEYKbuYeyUUvkaAGuWhVE7Qzd7ewIYVvZx96eIaX1Ym07ny0Qc8I+xv+AqDKDaBCAQAA
│ ├─ [~] .IsCompleteHandler7073F4DA:
│ │ ├─ [~] .Metadata:
│ │ │ └─ [~] .aws:asset:path:
│ │ │ ├─ [-] asset.04a62144012b2cd32474867b1533a36eb328db26f983a17ff81189234aff4705
│ │ │ └─ [+] asset.4ac7c7f6f1e1700aa9ace7e1f38c92b9245129096921bcdc5ff2b1748d57c29d
│ │ └─ [~] .Properties:
│ │ └─ [~] .Code:
│ │ └─ [~] .S3Key:
│ │ ├─ [-] 04a62144012b2cd32474867b1533a36eb328db26f983a17ff81189234aff4705.zip
│ │ └─ [+] 4ac7c7f6f1e1700aa9ace7e1f38c92b9245129096921bcdc5ff2b1748d57c29d.zip
│ └─ [~] .OnEventHandler42BEBAE0:
│ ├─ [~] .Metadata:
│ │ └─ [~] .aws:asset:path:
│ │ ├─ [-] asset.04a62144012b2cd32474867b1533a36eb328db26f983a17ff81189234aff4705
│ │ └─ [+] asset.4ac7c7f6f1e1700aa9ace7e1f38c92b9245129096921bcdc5ff2b1748d57c29d
│ └─ [~] .Properties:
│ └─ [~] .Code:
│ └─ [~] .S3Key:
│ ├─ [-] 04a62144012b2cd32474867b1533a36eb328db26f983a17ff81189234aff4705.zip
│ └─ [+] 4ac7c7f6f1e1700aa9ace7e1f38c92b9245129096921bcdc5ff2b1748d57c29d.zip
└─ [~] TemplateURL
└─ [~] .Fn::Join:
└─ @@ -5,6 +5,6 @@
[ ] {
[ ] "Ref": "AWS::URLSuffix"
[ ] },
[-] "/cdk-hnb659fds-assets-ACCOUNT-us-east-1/0786f62ccf83e5ac76dc61ff94d274b13763f161e7a55aaea426b86951b77e81.json"
[+] "/cdk-hnb659fds-assets-ACCOUNT-us-east-1/a4c76fa1256e9f6a80e010b5f0639632c344bded01dd736325bda4eb9d5c045a.json"
[ ] ]
[ ] ]
[~] AWS::CloudFormation::Stack @aws-cdk--aws-eks.KubectlProvider.NestedStack/@aws-cdk--aws-eks.KubectlProvider.NestedStackResource awscdkawseksKubectlProviderNestedStackawscdkawseksKubectlProviderNestedStackResourceA7AEBA6B
├─ [~] NestedTemplate
│ └─ [~] .Resources:
│ ├─ [~] .AwsCliLayerF44AAF94:
│ │ ├─ [~] .Metadata:
│ │ │ └─ [~] .aws:asset:path:
│ │ │ ├─ [-] asset.ba6af152ce7366ad6f9019e5c6719704a92411a2eb6f3bc1cb39dea393ece19e.zip
│ │ │ └─ [+] asset.fc2a3ca6356f84129511297b00cd5cef3c7b67abdf955eb5efa75c711da7399f.zip
│ │ └─ [~] .Properties:
│ │ └─ [~] .Content:
│ │ └─ [~] .S3Key:
│ │ ├─ [-] ba6af152ce7366ad6f9019e5c6719704a92411a2eb6f3bc1cb39dea393ece19e.zip
│ │ └─ [+] fc2a3ca6356f84129511297b00cd5cef3c7b67abdf955eb5efa75c711da7399f.zip
│ └─ [~] .CDKMetadata:
│ └─ [~] .Properties:
│ └─ [~] .Analytics:
│ ├─ [-] v2:deflate64:H4sIAAAAAAAA/02Pu47CMBBFv4XeGRBpoASkbaCIQKK1HGdgZ+PYkscGRZH/nThhEdV9THHPrKEsYbVQTy500xaGahjGIAdsGY6xRh1M5d2DGvTCqK5uFAw/0epAzorDzX77k+rRX9HzmJMg1cFwdgbzadLKGdJ9jrNLgkupmDEw7LKMGfZRtxj2ijGJqbwEdSd7f69Lk0dkhtSGYPfkg6FpWLQz7j/2VCahIwfXSY/sotfI8HknY73bJKo+/Dq7LGED28UfExU+2kAdwnnWFy09WdcqAQAA
│ └─ [+] v2:deflate64:H4sIAAAAAAAA/02PTYvCMBCGf4v3dFwsgh5V8LJ7KBW8hjSddcemCWQSpZT8d5vWXfb0fszhfWYD5RY+VurJhW67wlAD4xTkiB3DZ2xQB1N596AWvTCqb1oF4zlaHchZcfq2//2XGtBf0fOUkyDVw1g7g/k0a+UM6SHHxSXBpVTMGBgOWaYMx6g7DEfFmMRcXoK6kb2916XJIzJDakNwePLJ0DwsugX3F3suk9CRg+ulR3bRa2T4eydjvdskqiH8OLsuYQf71Z2JCh9toB6hXvQFupW1TSoBAAA=
└─ [~] TemplateURL
└─ [~] .Fn::Join:
└─ @@ -5,6 +5,6 @@
[ ] {
[ ] "Ref": "AWS::URLSuffix"
[ ] },
[-] "/cdk-hnb659fds-assets-ACCOUNT-us-east-1/d52fdb7e93bf3a10795401b23fc961cc2a4e16bd05842beef2b11c3f7b8bc369.json"
[+] "/cdk-hnb659fds-assets-ACCOUNT-us-east-1/34d4db1389512a45b874c10e5b3846c106f23f5e42b06dec51e3421cb0b4a692.json"
[ ] ]
[ ] ]
Reproduction Steps
- existing EKS stack on v2.33
cdk diff-> no changes- upgrade to v2.35
cdk diff-> see changescdk deploystack on v2.35- 💥
Possible Solution
No response
Additional Information/Context
No response
CDK CLI Version
2.35.0 (build 5c23578)
Framework Version
2.35.0
Node.js Version
v16.15.1
OS
mac 12.4
Language
Python
Language Version
3.8.9
Other information
No response