Define Cloudwatch Alarms

[kaykhan]

Description

What problem are you trying to solve?

We would like to be able to define a set of cloudwatch alarms for ec2 instance.

spec:
  cloudWatchAlarms:
    - alarmName: "HighCPUUtilization"
      alarmDescription: "CPU utilization is too high"
      metricName: "CPUUtilization"
      namespace: "AWS/EC2"
      statistic: "Average"
      period: 300
      evaluationPeriods: 2
      threshold: 80
      comparisonOperator: "GreaterThanThreshold"
      alarmActions:
        - "arn:aws:sns:us-west-2:123456789012:MyTopic"
      actionsEnabled: true

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-cloudwatch-alarm.html

Currently we have to do this independently with a custom solution of listening to new ec2 instances. I don’t like this approach because it separates alarm creation from the definition of Karpenter node pools/classes. It makes more sense for these to be managed together since the entity defining the node pools/classes has better context on which CPU or other metrics to target and when to trigger alarms.

How important is this feature to you?

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

---

I would be interested in contributing, although it will be my first time working on an operator so i would need a lot of hand holding.

[jonathan-innis]

Currently we have to do this independently with a custom solution of listening to new ec2 instances

Are there alternatives that you could explore here that use search expressions for your alarming? I'm assuming that you want this created at runtime because you currently have to expand and retract the dimensions that you alarm on as nodes are added and deleted?

[kaykhan]

Currently we have to do this independently with a custom solution of listening to new ec2 instances

Are there alternatives that you could explore here that use search expressions for your alarming? I'm assuming that you want this created at runtime because you currently have to expand and retract the dimensions that you alarm on as nodes are added and deleted?

The challenge isn’t just dynamically adjusting alarms—it’s about ensuring alarms are defined alongside the Karpenter EC2NodeClass, so they are created and removed in sync with its node lifecycle. The engineer defining the node class has the best context on alarm settings, so it makes sense for alarms to be part of that definition.

I see that SEARCH() could be used to target a set of Karpenter nodes dynamically based on tags (e.g., karpenter.sh/provisioner-name=workload1) and create a single aggregated alarm for all matching instances. This removes the need to manually create alarms per instance.

However, this still requires us to define CloudWatch alarms separately, which adds extra management overhead. More importantly, our original goal is to have individual alarms for each EC2 instance, not a single aggregated alarm. For example, we may have a shared EC2NodeClass used across different workloads, but each instance still needs its own individual alarm rather than a single alarm covering all instances together. Even if the CPU utilization threshold remains the same, we want alarms that trigger per instance, not just at an aggregated level.

This is why SEARCH() doesn’t fully solve the problem—we need alarms that are tied to each instance and managed as part of the Karpenter lifecycle. Integrating alarm creation into Karpenter’s configuration would ensure that alarms are automatically created and cleaned up, rather than requiring separate management.

[github-actions]

This issue has been inactive for 14 days. StaleBot will close this stale issue after 14 more days of inactivity.

[kaykhan]

Not stale?

[github-actions]

This issue has been inactive for 14 days. StaleBot will close this stale issue after 14 more days of inactivity.