✨ Add ExistingLaunchTemplate (BYO) support for EKS managed node groups - #5906
✨ Add ExistingLaunchTemplate (BYO) support for EKS managed node groups#5906AmitSahastra wants to merge 7 commits into
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
Hi @AmitSahastra. Thanks for your PR. I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with Tip We noticed you've done this a few times! Consider joining the org to skip this step and gain Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
962cb7e to
ec3d921
Compare
ec3d921 to
10b2118
Compare
10b2118 to
02021ae
Compare
|
Hi @fiunchinho @serngawy the bot suggested I consider joining the kubernetes-sigs org given my contributions. Would either of you be willing to sponsor me for membership? I'm happy to open the formal request at https://github.com/kubernetes/org/issues/new?template=membership.md once I have two sponsors confirmed. Thanks! |
d6487c5 to
b292a1b
Compare
459ba3d to
07c03fa
Compare
|
/ok-to-test |
|
@AmitSahastra: The following tests failed, say
Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
9572a9f to
1efed4e
Compare
|
Adding label DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
8c80712 to
2b770c2
Compare
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Allow users to reference a pre-existing (BYO) AWS EC2 launch template by ID in AWSManagedMachinePoolSpec.AWSLaunchTemplate, instead of having CAPA create and manage one. When AWSLaunchTemplate.ID is set: - CAPA uses the referenced template directly and will not create, update, or delete it - versionNumber is required; CAPA-managed fields (ami, instanceType, sshKeyName, rootVolume, etc.) are forbidden - ID and Name are immutable after creation; versionNumber is mutable to support rolling out new template versions without replacing the nodegroup Key changes: - New ID field on exp/api/v1beta2.AWSLaunchTemplate with CEL immutability rule; v1beta1 is NOT modified (field is v1beta2-only per deprecation policy); ID survives v1beta2→v1beta1→v1beta2 round-trips via the CAPI MarshalData/UnmarshalData annotation mechanism, restored in ConvertTo for both AWSManagedMachinePool and AWSMachinePool - Generated files (CRDs, deepcopy, conversion) produced by make generate; v1beta1 CRD schema has no id field or XValidation rules - IsBYOLaunchTemplate() scope helper gates controller and nodegroup service to skip LT create/delete lifecycle for BYO templates - Webhook validates BYO-specific invariants and immutability - launchTemplateNeedsUpdate() helper covers ID changes and concrete version changes while skipping symbolic aliases ($Latest, $Default) to prevent endless reconcile loops - Guard against nil Version/ReleaseVersion on CREATE_FAILED/DELETE_FAILED nodegroups - LaunchTemplateNeedsUpdate returns a reason enum for improved log output - e2e test (BYOMachinePoolSpec): creates a real EC2 launch template, creates the nodegroup, and verifies the nodegroup uses the BYO template ID Signed-off-by: Amit Sahastrabuddhe <amit.sahastra@gmail.com>
- remove immutability markers from AWSLaunchTemplate.Name (kept on the new id field) since the shared type is also used by AWSMachinePool; regenerate CRDs - split BYO ami validation into separate ami.id and ami.eksLookupType errors - extract BYO-specific launch template validation into validateBYOLaunchTemplate - replace reflect.DeepEqual immutability helper with cmp.Equal-based checks inline - add IsCAPAManagedLaunchTemplate scope helper and use it in the controller - use ptr.Deref in IsBYOLaunchTemplate and reuse the scope helper in the nodegroup service - drop redundant nil guards when restoring the v1beta2-only id in conversion Signed-off-by: Amit Sahastrabuddhe <amit.sahastra@gmail.com>
2b770c2 to
23cfe6a
Compare
|
/test pull-cluster-api-provider-aws-test |
|
/test ? |
|
/test pull-cluster-api-provider-aws-e2e-eks |
There was a problem hiding this comment.
Pull request overview
This PR adds BYO (bring-your-own) EC2 launch template support for EKS managed node groups by introducing an awsLaunchTemplate.id field and updating reconciliation, validation, and tests so CAPA can reference an existing launch template without managing its lifecycle.
Changes:
- Add BYO launch template semantics to
AWSLaunchTemplateand update controller/service logic to skip LT lifecycle management whenidis set. - Update webhook validation/immutability checks and add focused unit tests for launch template version drift handling.
- Add new E2E flavor + helpers to exercise managed node groups using a pre-created EC2 launch template.
Reviewed changes
Copilot reviewed 17 out of 19 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| test/e2e/suites/managed/machine_pool.go | Adds a BYO-machine-pool spec helper and wires env var substitution for the new flavor. |
| test/e2e/suites/managed/helpers.go | Adds EC2 LT create/delete helpers and a verifier that the nodegroup uses the expected LT ID. |
| test/e2e/suites/managed/eks_test.go | Runs a new E2E scenario creating a managed node pool using a BYO launch template. |
| test/e2e/shared/defaults.go | Introduces env var keys for BYO LT ID/version template rendering. |
| test/e2e/data/eks/cluster-template-eks-managed-machinepool-with-byo-launch-template-only.yaml | New E2E flavor manifest for BYO LT managed machine pool. |
| test/e2e/data/e2e_eks_conf.yaml | Registers the new E2E flavor template for artifact publishing/consumption. |
| pkg/cloud/services/eks/nodegroup.go | Updates CreateNodegroup/UpdateNodegroupVersion behavior for BYO LT and symbolic versions; adds failed-nodegroup guards. |
| pkg/cloud/services/eks/nodegroup_test.go | Unit tests for symbolic LT version detection and LT drift logic. |
| pkg/cloud/scope/managednodegroup.go | Adds helpers to distinguish BYO vs CAPA-managed LTs and adjusts LT naming behavior. |
| exp/webhooks/awsmanagedmachinepool_webhook.go | Adds BYO-specific validation and immutability rules; aligns with EKS CreateNodegroup contract. |
| exp/webhooks/awsmanagedmachinepool_webhook_test.go | Adds webhook test coverage for BYO validations and immutability constraints. |
| exp/controllers/awsmanagedmachinepool_controller.go | Skips LT create/update/delete for BYO and sets LT status directly from spec. |
| exp/api/v1beta2/zz_generated.deepcopy.go | Regenerates deepcopy for the new AWSLaunchTemplate.ID field. |
| exp/api/v1beta2/types.go | Adds AWSLaunchTemplate.ID field (BYO LT reference) to v1beta2 API. |
| exp/api/v1beta2/awsmanagedmachinepool_types.go | Documents BYO LT behavior on AWSManagedMachinePoolSpec.AWSLaunchTemplate. |
| exp/api/v1beta1/zz_generated.conversion.go | Regenerated conversion code reflecting v1beta2-only fields. |
| exp/api/v1beta1/conversion.go | Preserves/drops v1beta2-only LT fields across hub/spoke conversions via annotations. |
| config/crd/bases/infrastructure.cluster.x-k8s.io_awsmanagedmachinepools.yaml | Regenerates CRD schema to include awsLaunchTemplate.id with immutability validation. |
| config/crd/bases/infrastructure.cluster.x-k8s.io_awsmachinepools.yaml | Regenerates CRD schema to include awsLaunchTemplate.id with immutability validation. |
Files not reviewed (2)
- exp/api/v1beta1/zz_generated.conversion.go: Generated file
- exp/api/v1beta2/zz_generated.deepcopy.go: Generated file
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…RD level Set amiType (AL2023_x86_64_STANDARD) and instanceType on the BYO launch template managed machine pool e2e flavor. Without an explicit amiType, EKS defaults to AL2_x86_64, which is rejected for Kubernetes 1.33+, so the nodegroup was never created and the e2e-eks suite timed out waiting for ready nodes. Setting these also exercises the supported path where amiType and instanceType are supplied alongside a BYO launch template. Add a CEL immutability rule for awsLaunchTemplate.name on the AWSManagedMachinePool spec so the immutability contract already enforced by the webhook is visible at the CRD level, consistent with awsLaunchTemplate.id. The rule is scoped to the AWSManagedMachinePool field rather than the shared AWSLaunchTemplate type because AWSMachinePool ignores awsLaunchTemplate.name and does not treat it as immutable. Signed-off-by: Amit Sahastrabuddhe <amit.sahastra@gmail.com>
|
@AmitSahastra: The following tests failed, say
Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
|
/retest pull-cluster-api-provider-aws-e2e-blocking |
|
/test pull-cluster-api-provider-aws-e2e-blocking |
|
Adding label DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
What type of PR is this?
/kind feature
What this PR does / why we need it:
Adds BYO (bring-your-own) launch template support for EKS managed node groups by introducing an
idfield onAWSLaunchTemplate(in bothv1beta1andv1beta2). Whenspec.awsLaunchTemplate.idis set, CAPA references the pre-existing AWS launch template directly instead of creating or managing one.This enables use cases where organizations manage launch templates externally (Terraform, CloudFormation, the AWS console, etc.) and want CAPA to adopt them as-is for EKS managed node groups.
Highlights:
idfield and immutablenamefield onAWSLaunchTemplate(v1beta1 + v1beta2), with CELself == oldSelfimmutability markers and regenerated CRDs/conversions.idis set, the reconciler skips LT create/update/delete and uses the referenced template directly; status (launchTemplateID/launchTemplateVersion) is populated from the spec. CAPA-managed (no-id) behaviour is unchanged.versionNumberrequired; LT-level fields that only make sense for CAPA-managed LTs (ami,instanceType,rootVolume,nonRootVolumes,sshKeyName, image-lookup fields,additionalSecurityGroups) are forbidden;id/nameimmutability. Plus cross-cutting gates aligned with the AWSCreateNodegroupcontract:spec.instanceTypeandspec.amiTypeare allowed alongside a BYO LT, whilespec.diskSizeandspec.remoteAccessare rejected whenever any LT is specified.CreateNodegrouppath —InstanceTypesandAmiTypeflow through for BYO LTs,RemoteAccessandDiskSizeare omitted when a LT is in use; symbolic LT versions ($Latest,$Default) are handled to avoid endless reconcile loops; guards added for nilVersion/ReleaseVersionon failed node groups.Which issue(s) this PR fixes (optional, in
fixes # (, fixes #, ...)format, will close the issue(s) when PR gets merged):Fixes #5896
Special notes for your reviewer:
VersionNumberis intentionally mutable onAWSLaunchTemplateso users can roll out a new LT version to the node group;idandnameare immutable (enforced via CEL and the webhook).amiType,amiVersion) and a LT that already pins a custom AMI are surfaced by the EKS API at create time rather than at admission.AWSManagedMachinePool:AI Usage:
AI assistance (Warp Agent Mode) was used for drafting webhook validation and writing unit tests. All generated code was reviewed, tested, and signed off by the author.
Checklist:
Release note: