fix(eks): can't deploy with Bottlerocket amiType #17775
fix(eks): can't deploy with Bottlerocket amiType #17775mergify[bot] merged 26 commits intoaws:masterfrom pahud:pahud/eks-unable-to-use-bottlerocket-17641
Conversation
|
Hey @neilkuan I'll need your initial review. Let's discuss this offline. |
| throw new Error(`The specified AMI does not match the instance types architecture, either specify ${expectedAmiType} or dont specify any`); | ||
| /** | ||
| * if the user explicitly configured instance types, we can't caculate the expected ami type as we support | ||
| * Amazon Linux 2 and Bottlerocket now. However we can check: |
There was a problem hiding this comment.
Could we do the opposite? Take the AMI type as a starting point (if it's provided) and figure out which instance types are valid for it?
There was a problem hiding this comment.
@otaviomacedo Yes this is exactly what I am working on.
I introduced possibleAmiTypes in this PR and calculate from the provided instance types.
Let's say if you provide all x86 instance types the possibleAmiTypes would only be AL2_X86_64 and BOTTLEROCKET_X86_64. In this case if amiType is provided we can check if it's included in the possibleAmiTypes.
|
for testing I know about if I not define so if I define
|
There was a problem hiding this comment.
Some NITS. Other than that, looks good.
Just need to be cautious that we need to calculate the recommended amiType corresponding to the instanceTypes when the user doesn't specify amiType or ami in launchTemplate. Also need to care the backward compatibility when calculating the recommended amiType for user.
Maybe add something like this in the if (instanceTypes && instanceTypes.length > 0) block:
if (!props.amiType && !props.launchTemplateSpec) {
// calculate the recommended `amiType` corresponding to the `instanceTypes`
}or when constructing the CfnNodegroup, use:
const resource = new CfnNodegroup(this, 'Resource', {
...
amiType: props.launchTemplateSpec ? props.amiType : (props.amiType ?? possibleAmiTypes[0]),
...
});Co-authored-by: kirintw <kirin@xendit.co>
Thanks for the suggested changes. Let's dive a little bit deeper here. We have some optional properties in this construct:
When
That being said, it doesn't make difference where props.launchTemplateSpec is defined or not. The validation check is literally the same. Now let's check the following cases: Case 1: Previously users would have Case 2: We just use |
There was a problem hiding this comment.
- Can you also add testing for got this error message
instanceTypes of different CPU architectures is not allowed?!
ex:
this.cluster.addNodegroupCapacity('BottlerocketNG1', {
instanceTypes: [new ec2.InstanceType('t4g.large'), new ec2.InstanceType('t2.large')],
});- Can you add the
c7gin to instance-types.ts for graviton3?!
other LGTM.
The test is already there.
c7g is still in preview so I am not adding it in this PR. https://aws.amazon.com/tw/ec2/instance-types/c7g/
|
Yeah, I see how changing type MyArchitecture = InstanceArchitecture | 'GPU';
function getPossibleAmiTypes(instanceTypes: InstanceType[]): NodegroupAmiType[] {
function typeToArch(instanceType: InstanceType): MyArchitecture {
return isGpuInstanceType(instanceType) ? 'GPU' : instanceType.architecture;
}
const archAmiMap = new Map<MyArchitecture, NodegroupAmiType[]>([
[InstanceArchitecture.ARM_64, arm64AmiTypes],
[InstanceArchitecture.X86_64, x8664AmiTypes],
['GPU', gpuAmiTypes],
]);
const architectures: Set<MyArchitecture> = new Set(instanceTypes.flatMap(typeToArch));
if (architectures.size === 0) {
throw new Error(`Cannot determine any ami type comptaible with instance types: ${instanceTypes.map(i => i.toString).join(',')}`);
}
if (architectures.size > 1) {
throw new Error('instanceTypes of different architectures is not allowed');
}
return archAmiMap.get(Array.from(architectures)[0])!;
} |
@otaviomacedo This is interesting! Let me try it. |
|
@otaviomacedo all fixed. Please take a look. |
|
@kirintwn all resolved :-) |
aws-cdk/packages/@aws-cdk/aws-eks/lib/managed-nodegroup.ts Lines 481 to 489 in cd5f79c can you confirm this? thanks! |
@kirintwn nice catch! all fixed! Maybe another look? copy @otaviomacedo |
|
Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork). |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
|
Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork). |
As `Nodegroup` now supports different AMI types including Bottlerocket for both x86_64 and ARM_64, we cannot determine correct amiType simply from the `instanceTypes` property(aws#17641 ). However, when `instanceTypes` are provided, we still can check: 1. if instance types of different CPU architectures are mxied and throw an error 2. if the provided `amyType` compatible with the instanceTypes If user opt in Bottlerocket or any other AMI types other than AL2, users have to specify the `amiType` explicitly. If it's unspecified, we will use AL2 implicitly to avoid breaking changes, which is the default behavior previously. The only case `amiType` has to be undefined is that when custom AMI is defined in the launch template. As we can't check this case, users have to explicitly leave it undefined. We add a notice in the property doc string for this. Related to aws#12441 Fixes: aws#17641 ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*

As
Nodegroupnow supports different AMI types including Bottlerocket for both x86_64 and ARM_64, we cannot determine correct amiType simply from theinstanceTypesproperty(#17641 ). However, wheninstanceTypesare provided, we still can check:amyTypecompatible with the instanceTypesIf user opt in Bottlerocket or any other AMI types other than AL2, users have to specify the
amiTypeexplicitly. If it's unspecified, we will use AL2 implicitly to avoid breaking changes, which is the default behavior previously.The only case
amiTypehas to be undefined is that when custom AMI is defined in the launch template. As we can't check this case, users have to explicitly leave it undefined. We add a notice in the property doc string for this.Related to #12441
Fixes: #17641
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license