Skip to content

aws-ecs: Cluster.addAsgCapacityProvider blocks EC2 metadata in instances created by the ASG #28270

@Brads3290

Description

@Brads3290

Describe the bug

Calling Cluster.addAsgCapacityProvider modifies the AutoScalingGroup to add a userdata script to the launch template which blocks access to 169.254.169.254:

"MyEcsClusterAsgLaunchTemplate854B9AAD": {
   "Type": "AWS::EC2::LaunchTemplate",
   "Properties": {
    "LaunchTemplateData": {

     ...

     "UserData": {
      "Fn::Base64": {
       "Fn::Join": [
        "",
        [
         "#!/bin/bash\necho ECS_CLUSTER=",
         {
          "Ref": "MyEcsClusterACF79753"
         },
         " >> /etc/ecs/ecs.config\nsudo iptables --insert FORWARD 1 --in-interface docker+ --destination 169.254.169.254/32 --jump DROP\nsudo service iptables save\necho ECS_AWSVPC_BLOCK_IMDS=true >> /etc/ecs/ecs.config"
        ]
       ]
      }
     }
    },
    
    ...
}

In particular: sudo iptables --insert FORWARD 1 --in-interface docker+ --destination 169.254.169.254/32 --jump DROP

Without the call to Cluster.addAsgCapacityProvider (and possibly other methods do this too), this isn't added.

This behaviour isn't documented anywhere that I can find, and causes the error No RegionEndpoint or ServiceURL configured from the SDK when using the default settings and expecting everything to be picked up from the environment (which is how it seems to work when creating the cluster from the console).

Expected Behavior

Don't block access to EC2 metadata, or at least clearly document it and offer alternatives if someone wants to make use of the TaskRole inside the container

Current Behavior

The EC2 metadata endpoint is blocked using sudo iptables --insert FORWARD 1 --in-interface docker+ --destination 169.254.169.254/32 --jump DROP

Reproduction Steps

var app = new App();
var stack = new Stack(app, "ReproStack", new StackProps() {
    Env = new Amazon.CDK.Environment {
        Account = System.Environment.GetEnvironmentVariable("CDK_DEFAULT_ACCOUNT"),
        Region = System.Environment.GetEnvironmentVariable("CDK_DEFAULT_REGION"),
    },
});

var defaultVpc = Vpc.FromLookup(stack, "defaultVpc", new VpcLookupOptions() {
    IsDefault = true,
});

var asg = new AutoScalingGroup(stack, "AutoScalingGroup", new AutoScalingGroupProps() {
    Vpc = defaultVpc,
    InstanceType = InstanceType.Of(InstanceClass.T3A, InstanceSize.SMALL),
    MachineImage = EcsOptimizedImage.AmazonLinux2(),
});

var cluster = new Cluster(stack, "Cluster", new ClusterProps() {
    Vpc = defaultVpc,
});

var provider = new AsgCapacityProvider(stack, "AsgCapacityProvider", new AsgCapacityProviderProps() {
    AutoScalingGroup = asg,
});

// With this line, the UserData is included in the ASG's launch template; without it, it is not
cluster.AddAsgCapacityProvider(provider);

app.Synth();

Possible Solution

Don't block access to EC2 metadata, or at least clearly document it and offer alternatives if someone wants to make use of the TaskRole inside the container

Additional Information/Context

No response

CDK CLI Version

2.114.0 (build 12879fa)

Framework Version

No response

Node.js Version

v18.16.1

OS

MacOS 13.6.2

Language

.NET

Language Version

.NET 8

Other information

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    @aws-cdk/aws-ecsRelated to Amazon Elastic ContainerbugThis issue is a bug.effort/mediumMedium work item – several days of effortp2

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions