The new instanceSelector feature looks awesome and has the potential to simplify our eks config yaml as well as make it more intuitive and increase supportability.
What feature/behavior/change do you want?
I was testing the new instanceSelector feature available in eksctl 0.44.0-rc.0 in EKS using Kubernetes version 1.19 on cluster in us-east-1 using platform version eks.2. When requesting instanceTypes that support a certain vCPUs/memory/gpu filters, I expect it to only return instanceTypes that are available for all of the availability zones requested in my nodegroup. While this has worked in most cases, I found at least one that does not work.
Consider the below eksctl yaml snippet for a managedNodeGroup and running eksctl create nodegroup...:
managedNodeGroups:
- name: my-node-group
availabilityZones:
- us-east-1a
- us-east-1b
- us-east-1d
instanceSelector:
vCPUs: 4
memory: "16"
spot: trueThe nodegroup fails to spin up due to this CloudFormation error: Your requested instance type (d3en.xlarge) is not supported in your requested Availability Zone (us-east-1a). Please retry your request by not specifying an Availability Zone or choosing us-east-1b, us-east-1c, us-east-1d.
Why do you want this feature?
I want this feature so that I can have confidence that using instanceSelector will work reliably for my environment. I don't want to worry about new conditions that may prevent cluster-autoscaler from spinning up more nodes. This could result in CloudFormation or ASG errors that affect my cluster's ability to meet increased demand. Conditions such as new instanceTypes being released for a region that are not yet released in all availability zones.
Deliverable
AWS Support suggested that I raise an issue directly on the AWS Instance Selector github page to request appropriate functionality since my understanding is that tool powers this feature for eksctl. I will do that (issue creation pending) but wanted to create an issue here for two reasons: raise awareness for any others running into the same issue and in case a fix for that tool requires refactoring on the eksctl side.
The new instanceSelector feature looks awesome and has the potential to simplify our eks config yaml as well as make it more intuitive and increase supportability.
What feature/behavior/change do you want?
I was testing the new
instanceSelectorfeature available ineksctl 0.44.0-rc.0in EKS using Kubernetes version1.19on cluster inus-east-1using platform versioneks.2. When requestinginstanceTypesthat support a certain vCPUs/memory/gpu filters, I expect it to only returninstanceTypesthat are available for all of the availability zones requested in my nodegroup. While this has worked in most cases, I found at least one that does not work.Consider the below eksctl yaml snippet for a
managedNodeGroupand runningeksctl create nodegroup...:managedNodeGroups: - name: my-node-group availabilityZones: - us-east-1a - us-east-1b - us-east-1d instanceSelector: vCPUs: 4 memory: "16" spot: trueThe nodegroup fails to spin up due to this CloudFormation error:
Your requested instance type (d3en.xlarge) is not supported in your requested Availability Zone (us-east-1a). Please retry your request by not specifying an Availability Zone or choosing us-east-1b, us-east-1c, us-east-1d.Why do you want this feature?
I want this feature so that I can have confidence that using instanceSelector will work reliably for my environment. I don't want to worry about new conditions that may prevent cluster-autoscaler from spinning up more nodes. This could result in CloudFormation or ASG errors that affect my cluster's ability to meet increased demand. Conditions such as new
instanceTypesbeing released for a region that are not yet released in all availability zones.Deliverable
AWS Support suggested that I raise an issue directly on the AWS Instance Selector github page to request appropriate functionality since my understanding is that tool powers this feature for
eksctl.I will do that (issue creation pending) butwanted to create an issue here for two reasons: raise awareness for any others running into the same issue and in case a fix for that tool requires refactoring on theeksctlside.