Skip to content

feat(eks-v2-alpha): eks auto mode support#33373

Merged
mergify[bot] merged 38 commits intoaws:mainfrom
pahud:eks-auto-mode
Feb 25, 2025
Merged

feat(eks-v2-alpha): eks auto mode support#33373
mergify[bot] merged 38 commits intoaws:mainfrom
pahud:eks-auto-mode

Conversation

@pahud
Copy link
Copy Markdown
Contributor

@pahud pahud commented Feb 11, 2025 •
edited
Loading

Issue # (if applicable)

Address #32364 in aws-eks-v2-alpha.

For EKS Auto Mode, all required configs, including computeConfig, kubernetesNetworkConfig, and blockStorage are managed through the defaultCapacityType enum. When set to DefaultCapacityType.AUTOMODE (which is the default), these configurations are automatically enabled. The Cluster construct in aws-eks-v2-alpha enables EKS Auto Mode by default, managing compute resources through node pools instead of creating default capacity or nodegroups. Users can still opt-in to traditional nodegroup management by setting defaultCapacityType to NODEGROUP or EC2.

User Experience:

// Default usage - Auto Mode enabled by default
new eks.Cluster(this, 'hello-eks', {
  vpc,
  version: eks.KubernetesVersion.V1_32,
  kubectlProviderOptions: {
    kubectlLayer: new KubectlV32Layer(this, 'kubectl'),
  },
  // Auto Mode is enabled by default, no need to specify anything
});

// Explicit Auto Mode configuration
new eks.Cluster(this, 'hello-eks', {
  vpc,
  version: eks.KubernetesVersion.V1_32,
  kubectlProviderOptions: {
    kubectlLayer: new KubectlV32Layer(this, 'kubectl'),
  },
  defaultCapacityType: eks.DefaultCapacityType.AUTOMODE,  // Optional, this is default
  compute: {
    nodePools: ['system', 'general-purpose'],  // Optional, these are default values
    nodeRole: customRole,  // Optional, custom IAM role for nodes
  }
});

Update Summary

  • EKS Auto Mode is the default mode for Cluster construct in V2. When enabled:
    • Automatically manages compute resources through node pools
    • Enables elastic load balancing in Kubernetes networking
    • Enables block storage configuration
    • Will not create defaultCapacity as a NODEGROUP(major difference from aws-eks module)
  • Node pools are case-sensitive and must be "system" and/or "general-purpose"
  • Auto Mode can coexist with manually added node groups for hybrid deployments
  • Required IAM policies are automatically attached
  • Restore the outputConfigCommand support previously in aws-eks module
  • integration test
  • unit tests

Description of how you validated changes

On deploy the autoMode enabled cluster using the code above.

% kubectl create deployment nginx --image=nginx
% kubectl get events --sort-by='.lastTimestamp'
20m         Normal    Nominated                 pod/nginx-5869d7778c-52pzg        Pod should schedule on: nodeclaim/general-purpose-87brc
20m         Normal    Launched                  nodeclaim/general-purpose-87brc   Status condition transitioned, Type: Launched, Status: Unknown -> True, Reason: Launched
20m         Normal    DisruptionBlocked         nodeclaim/general-purpose-87brc   Nodeclaim does not have an associated node
19m         Normal    NodeHasSufficientPID      node/i-0322e9d8dd1b95a51          Node i-0322e9d8dd1b95a51 status is now: NodeHasSufficientPID
19m         Normal    NodeAllocatableEnforced   node/i-0322e9d8dd1b95a51          Updated Node Allocatable limit across pods
19m         Normal    NodeReady                 node/i-0322e9d8dd1b95a51          Node i-0322e9d8dd1b95a51 status is now: NodeReady
19m         Normal    Ready                     node/i-0322e9d8dd1b95a51          Status condition transitioned, Type: Ready, Status: False -> True, Reason: KubeletReady, Message: kubelet is posting ready status
19m         Normal    Synced                    node/i-0322e9d8dd1b95a51          Node synced successfully
19m         Normal    NodeHasNoDiskPressure     node/i-0322e9d8dd1b95a51          Node i-0322e9d8dd1b95a51 status is now: NodeHasNoDiskPressure
19m         Normal    NodeHasSufficientMemory   node/i-0322e9d8dd1b95a51          Node i-0322e9d8dd1b95a51 status is now: NodeHasSufficientMemory
19m         Warning   InvalidDiskCapacity       node/i-0322e9d8dd1b95a51          invalid capacity 0 on image filesystem
19m         Normal    Starting                  node/i-0322e9d8dd1b95a51          Starting kubelet.
19m         Normal    Registered                nodeclaim/general-purpose-87brc   Status condition transitioned, Type: Registered, Status: Unknown -> True, Reason: Registered
19m         Normal    Ready                     nodeclaim/general-purpose-87brc   Status condition transitioned, Type: Ready, Status: Unknown -> True, Reason: Ready
19m         Normal    Initialized               nodeclaim/general-purpose-87brc   Status condition transitioned, Type: Initialized, Status: Unknown -> True, Reason: Initialized
19m         Normal    RegisteredNode            node/i-0322e9d8dd1b95a51          Node i-0322e9d8dd1b95a51 event: Registered Node i-0322e9d8dd1b95a51 in Controller
19m         Normal    DisruptionBlocked         node/i-0322e9d8dd1b95a51          Node is nominated for a pending pod
19m         Normal    Scheduled                 pod/nginx-5869d7778c-52pzg        Successfully assigned default/nginx-5869d7778c-52pzg to i-0322e9d8dd1b95a51
19m         Warning   FailedCreatePodSandBox    pod/nginx-5869d7778c-52pzg        Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "9bd199c61bd9e93437b10a85af3ddc6965888e01bda96706e153b9e9852f67af": plugin type="aws-cni" name="aws-cni" failed (add): add cmd: Error received from AddNetwork gRPC call: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:50051: connect: connection refused"
19m         Normal    Pulling                   pod/nginx-5869d7778c-52pzg        Pulling image "nginx"
19m         Normal    Pulled                    pod/nginx-5869d7778c-52pzg        Successfully pulled image "nginx" in 2.307s (2.307s including waiting). Image size: 72188133 bytes.
19m         Normal    Created                   pod/nginx-5869d7778c-52pzg        Created container: nginx
19m         Normal    Started                   pod/nginx-5869d7778c-52pzg        Started container nginx

verify the nodes and pods

% kubectl get no
NAME                  STATUS   ROLES    AGE   VERSION
i-0322e9d8dd1b95a51   Ready    <none>   21m   v1.32.0-eks-2e66e76
% kubectl get po
NAME                     READY   STATUS    RESTARTS   AGE
nginx-5869d7778c-52pzg   1/1     Running   0          90m

Checklist

References

eksctl YAML experience

# cluster.yaml
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
  name: my-auto-cluster
  region: us-west-2

autoModeConfig:
  # defaults to false
  enabled: true
  # optional, defaults to [general-purpose, system]
  # suggested to leave unspecified
  nodePools: []string
  # optional, eksctl creates a new role if this is not supplied
  # and nodePools are present
  nodeRoleARN: string

Terraform experience:

provider "aws" {
  region = "us-east-1"
}

module "eks" {
  source          = "terraform-aws-modules/eks/aws"
  cluster_name    = "eks-auto-mode-cluster"
  cluster_version = "1.27"

  vpc_id     = "<your-vpc-id>"
  subnet_ids = ["<subnet-id-1>", "<subnet-id-2>"]

  cluster_compute_config = {
    enabled    = true
    node_pools = ["general-purpose"] # Default pool for Auto Mode
  }

  bootstrap_self_managed_addons = true
}

Pulumi experience

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

// Create EKS cluster with Auto Mode enabled
const cluster = new aws.eks.Cluster("example", {
    name: "example",
    version: "1.31",
    bootstrapSelfManagedAddons: false,  // Required: Must be false for Auto Mode
    computeConfig: {
        enabled: true,  // Enable Auto Mode compute
        nodePools: ["general-purpose"],
    },
    kubernetesNetworkConfig: {
        elasticLoadBalancing: {
            enabled: true,  // Required for Auto Mode
        },
    },
    storageConfig: {
        blockStorage: {
            enabled: true,  // Required for Auto Mode
        },
    },
});

Links

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

Loading
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@aws-cdk-automation aws-cdk-automation aws-cdk-automation left review comments

+2 more reviewers

@Issacwww Issacwww Issacwww approved these changes

@xazhao xazhao xazhao approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@xazhao xazhao

Labels

contribution/core This is a PR that came from AWS. p2

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@pahud @aws-cdk-automation @Issacwww @xazhao