Skip to content

Nodegroup version update incorrectly marked as terminal when cluster is upgrading #2957

Description

@blupman

Description

When upgrading an EKS cluster and its nodegroup(s) simultaneously (e.g., both from 1.32 to 1.33), the nodegroup update fails with a terminal error because the cluster hasn't finished upgrading yet. This prevents the controller from retrying, even though the condition is transient and will resolve once the cluster upgrade completes.

Steps to Reproduce

  1. Create an EKS cluster and nodegroup at version 1.32
  2. Update both the Cluster and Nodegroup CRs to version 1.33 at the same time, in my case this happens automaticaly since im using KRO to manage the cluster and nodegroup.
  3. The cluster starts upgrading (takes ~10-15 minutes)
  4. The nodegroup update is attempted before the cluster upgrade completes
  5. AWS returns an error because the nodegroup version must match the cluster version

Observed Behavior

The nodegroup resource enters a terminal state with this error:

status:
  conditions:
  - message: >-
      InvalidParameterException: Nodegroup Kubernetes version should be equal
      to Cluster kubernetes version 1.33 or NodeGroup kubernetes version 1.33
    status: 'True'
    type: ACK.Terminal

The controller stops reconciling and the nodegroup is never upgraded, even after the cluster upgrade completes.

Expected Behavior

The controller should recognize this as a transient condition and requeue the reconciliation. Once the cluster upgrade completes, the nodegroup update should succeed automatically.

An extra clue for me to assume this error could be a temporary one, is that restarting the ack controller helps, and the upgrade of the nodegroup is executed.

Metadata

Metadata

Assignees

Labels

kind/bugCategorizes issue or PR as related to a bug.service/eksIndicates issues or PRs that are related to eks-controller.

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions