Skip to content

eks: authentication mode failed to update #31032

@gokendra1

Description

@gokendra1

Describe the bug

CloudFormation stack (created using the AWS CDK) that manage an EKS cluster that is no longer in synch due to a failed deployment.

Steps that caused this issue:

  • Initially the stack is correctly deployed and the EKS cluster has configuration: accessConfig: {};
  • A new deploy of the stack is performed with an updated EKS cluster configuration with:accessConfig: {"authenticationMode":"API_AND_CONFIG_MAP"}, and some other resources updates;
  • After the EKS cluster property was updated, the stack deploy failed because of the other resources (not in scope of this request);
  • The stack tried to rollback, but failed because "Switching authentication modes on an existing cluster is a one-way operation" and "Once the access entry method is enabled, it cannot be disabled." (https://docs.aws.amazon.com/eks/latest/userguide/grant-k8s-access.html#set-cam);
  • continued update rollback skipping the EKS cluster resource, so the template still has EKS cluster configuration: accessConfig: {};
  • Now trying to deploy again the stack (without any other resources updates) with the new EKS cluster configuration fails with "Unsupported authentication mode update from API_AND_CONFIG_MAP to API_AND_CONFIG_MAP", because the actual resources already has the new configuration.

Additional points :

  • Checked with EKS team: You can change between API_AND_CONFIG_MAP and CONFIG_MAP. Then you can change to API from API_AND_CONFIG_MAP. These operations can't be reversed in the opposite direction. Meaning that once you convert to API, you cannot go back to CONFIG_MAP or API_AND_CONFIG_MAP. Additionally, you can't change from API_AND_CONFIG_MAP to CONFIG_MAP.
  • Checking with CDK team: Since CDK uses the custom resource , EKS cluster can't be remove and import it back to the cluster

Resources:
#https://github.com/aws/aws-cdk/blob/main/packages/%40aws-cdk/custom-resource-handlers/lib/aws-eks/cluster-resource-handler/cluster.ts

Expected Behavior

to handle errors gracefully in the custom resource. If a setting or configuration is already set or identical in the target resource, the custom resource should send a success signal to the CloudFormation (CFN) service.

Current Behavior

Now trying to deploy again the stack (without any other resources updates) with the new EKS cluster configuration fails with "Unsupported authentication mode update from API_AND_CONFIG_MAP to API_AND_CONFIG_MAP", because the actual resources already has the new configuration.

Reproduction Steps

Mentioned in the description

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

2.148.0

Framework Version

No response

Node.js Version

NA

OS

NA

Language

TypeScript, .NET

Language Version

No response

Other information

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    @aws-cdk/aws-eksRelated to Amazon Elastic Kubernetes ServicebugThis issue is a bug.effort/mediumMedium work item – several days of effortp2

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions