Account for AWS Savings Plan when choosing to deploy on-demand or spot

[lksvenoy-r7]

Tell us about your request

We need Karpenter to take into account discounts applied on compute as a result of AWS Savings Plans, and other types of discounts.

Having some mechanism to pass in configuration to inform Karpenter of discounts applicable to the specific environment it is running in, or have it somehow figure that out, would allow Karpenter to make better decisions with more accurate pricing information.

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?

Currently, Karpenter does not respect discounts that may be applicable as a result of AWS Savings Plan (and other discounts). Starting in April, we've noticed a significant increase in spot prices for certain regions, with for example us-east-1 spot instances that we are utilizing costing on average 50% more than 2 months prior.

This puts us in an awkward spot where running on on-demand may be significantly cheaper than running on spot pricing, due to discounts applied as a result of AWS Savings Plans.

To illustrate, consider the instance type m5d.12xlarge in us-east-1:

On the 16th of March, the spot price for this instance type was $1.06/hr. Since then, the price has been steadily increasing, currently at $1.56/hr.

This same instance type using a Compute Savings Plan, with a term length of 3 years, partial upfront, puts the price at $1.264/hr.

This is a ~21% difference in price.

Excluding other types of discounts, this means that we are now running into scenarios where running our workloads on on-demand is significantly cheaper than running on spot, not to mention more stable.

Are you currently working around this issue?

Short-term, we are seriously considering moving our workloads to on-demand, as this would significantly reduce the cost of our infrastructure. Long-term, we would like to see Karpenter be able to make better decisions around price performance.

Additional Context

No response

Attachments

No response

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[tzneal]

In case you didn't see it, the current workaround for savings plans is to use a weighted provisioner. This is documented at https://karpenter.sh/v0.27.3/concepts/scheduling/#savings-plans-and-reserved-instances

It isn't 100% ideal though and there is a gap here.

[runningman84]

The workaround assumes on demand pricing instead of having saving plan or private pricing.

[iandelahorne]

We're interested in this as AWS promotes savings plans heavily and we've noticed that spot pricing for SKUs we use have trended upward a lot recently.

There seems to be a limitation in the Pricing API that Karpenter uses to pull pricing data. According to the docs it does not take savings plans into account when returning prices.

[maximethebault]

Agreed. Lower compute cost is one of the most interesting Karpenter promise. Karpenter features around Spot/OD provisioning covered this really well, but global surge in spot prices has changed the landscape drastically.

As a user with renewed pressure on cost control, taking into account SP/RI in provisioning calculation has suddenly jumped from a "bonus" feature to a "top interest" feature.

And I suspect many users will feel the same.

[joshuapare]

So I will say, I would love to have this since we are also getting hit by the recent spot increases, but I can imagine a couple of things to take into account which @iandelahorne alluded to that likely need to get clarified:

• On-Demand and Spot are priced at the instance level, where the instances themselves have an associated price, while Savings Plan and Reserved Instances are discount applied to the account against On-Demand instances, so these aren't technically classified the same from a pricing perspective. Incorporating pricing into Karpener for these cases would likely involve calls and interpreting credits applied at the account level. This likely sounds out of scope, and I'm curious as to what someone from the core Karpenter team has to say here.
• If this is included without regard to active pricing and purely capacity based on available RI's, it would still need a different approach where the pricing discounts aren't taken into affect, but likely a priority is applied to provisioning instances that are covered under RI's. This would likely involve calls to the EC2 API to do calculations for current capacity in the account vs the number of provisioned instances. I'm imagining wrapping the On-Demand provider with an additional filter to provide a list of unused RI's. As far as the Savings Plan goes, this could likely be as simple as a further scope filter on the additional annotations that are already used.
• One more thing - this gets very tricky at the multi-account organization level, as RI's and Savings Plan can be used across accounts under the same organization that are within the same region, so how do we account for that? Do we note that Karpenter can only manage RI's and Savings plans that are initiated from the same account where Karpenter is installed? If we were to enable this, it's likely the scope would go way out of bounds, as it would require multi-account permissions to read EC2 provisioning data from other accounts and do comparisons and merges across all specified accounts and all RI's and pricing reductions at the account level. A lot of organizations have separate payer accounts from the accounts being used for operations and hosting, which may defeat the purpose.

If we're going for the simpler, single-account limitation approach, I would like to take a shot at the implementation of this, as I have written a few kubebuilder controllers, but before I do, it would be great to know what a core maintainer thinks of the approach as it pertains to the direction of Karpenter regarding RI and Savings Plan.

[Almenon]

nOps does this: https://www.nops.io/compute-copilot/
This is not an advertisement for nOps, I'm a minimalist and I would prefer just using karpenter if this feature existed.