Skip to content

Commit 3deed39

Browse files
docs: add conceptual documentation for NodeClaims (#9125)
Co-authored-by: Derek Frank <derekfrank@users.noreply.github.com>
1 parent 5ea4552 commit 3deed39

6 files changed

Lines changed: 1378 additions & 0 deletions

File tree

website/content/en/docs/concepts/nodeclaims.md

Lines changed: 230 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,9 @@ creates a NodeClaim, it asks the cloud provider to create the instance (launch),
1717
with the NodeClaim (registration), and wait for the node and its resources to be ready (initialization).
1818

1919
This page describes how NodeClaims integrate throughout Karpenter and the cloud provider implementation.
20+
NodeClaims are immutable, cluster-scoped resources that map one-to-one with a cloud provider instance and a Kubernetes Node.
21+
Each Karpenter-created NodeClaim is owned by a single [NodePool]({{< ref "./nodepools" >}}) and references a single [NodeClass]({{< ref "./nodeclasses" >}}).
22+
You can also create NodeClaims manually to request capacity outside of the standard provisioning workflow.
2023

2124
If you want to learn more about the nodes being managed by Karpenter, you can either look directly at the NodeClaim or at the nodes they are associated with:
2225

@@ -355,3 +358,230 @@ Events:
355358
Normal Initialized 36s karpenter Status condition transitioned, Type: Initialized, Status: Unknown -> True, Reason: Initialized
356359
Normal Ready 36s karpenter Status condition transitioned, Type: Ready, Status: Unknown -> True, Reason: Ready
357360
```
361+
362+
## NodeClaim API reference
363+
364+
The following is an annotated example of a NodeClaim resource.
365+
366+
```yaml
367+
apiVersion: karpenter.sh/v1
368+
kind: NodeClaim
369+
metadata:
370+
# NodeClaim names are auto-generated by Karpenter using the NodePool name as a prefix
371+
name: default-sfpsl
372+
# Labels are propagated from the NodePool's template and include well-known labels
373+
# resolved during scheduling (instance type, capacity type, zone, etc.)
374+
labels:
375+
billing-team: my-team
376+
karpenter.sh/capacity-type: spot
377+
karpenter.sh/nodepool: default
378+
node.kubernetes.io/instance-type: c5.2xlarge
379+
topology.kubernetes.io/zone: us-west-2b
380+
kubernetes.io/arch: amd64
381+
kubernetes.io/os: linux
382+
# Annotations are propagated from the NodePool's template
383+
annotations:
384+
example.com/owner: "my-team"
385+
# The NodeClaim is owned by the NodePool that created it
386+
ownerReferences:
387+
- apiVersion: karpenter.sh/v1
388+
kind: NodePool
389+
name: default
390+
blockOwnerDeletion: true
391+
# Karpenter adds a termination finalizer to ensure proper cleanup
392+
finalizers:
393+
- karpenter.sh/termination
394+
spec:
395+
# References the Cloud Provider's NodeClass resource
396+
nodeClassRef:
397+
group: karpenter.k8s.aws
398+
kind: EC2NodeClass
399+
name: default
400+
401+
# Taints applied to the node, propagated from the NodePool template
402+
taints:
403+
- key: example.com/special-taint
404+
effect: NoSchedule
405+
406+
# Startup taints applied to the node on creation, expected to be removed by an external system
407+
startupTaints:
408+
- key: example.com/another-taint
409+
effect: NoSchedule
410+
411+
# The amount of time a Node can live on the cluster before being removed
412+
# Inherited from the NodePool's spec.template.spec.expireAfter
413+
expireAfter: 720h
414+
415+
# The maximum duration the controller will wait before forcefully deleting pods on the node
416+
# Inherited from the NodePool's spec.template.spec.terminationGracePeriod
417+
terminationGracePeriod: 48h
418+
419+
# Requirements resolved during scheduling, combining NodePool requirements with pod constraints
420+
# These are the final, resolved requirements that were used to launch the instance
421+
requirements:
422+
- key: "kubernetes.io/arch"
423+
operator: In
424+
values: ["amd64"]
425+
- key: "kubernetes.io/os"
426+
operator: In
427+
values: ["linux"]
428+
- key: "karpenter.sh/capacity-type"
429+
operator: In
430+
values: ["spot"]
431+
- key: "karpenter.k8s.aws/instance-category"
432+
operator: In
433+
values: ["c", "m", "r"]
434+
- key: "karpenter.k8s.aws/instance-generation"
435+
operator: Gte
436+
values: ["3"]
437+
- key: "node.kubernetes.io/instance-type"
438+
operator: In
439+
values: ["c3.2xlarge", "c4.2xlarge", "c5.2xlarge", "c5.4xlarge", "m5.2xlarge"]
440+
- key: "topology.kubernetes.io/zone"
441+
operator: In
442+
values: ["us-west-2a", "us-west-2b"]
443+
444+
# Resource requests represent the minimum resources needed to schedule the pods
445+
# that triggered this NodeClaim
446+
resources:
447+
requests:
448+
cpu: "5150m"
449+
pods: "8"
450+
status:
451+
# The name of the Kubernetes Node object linked to this NodeClaim
452+
nodeName: ip-xxx-xxx-xx-xxx.us-west-2.compute.internal
453+
# The cloud provider identifier for the instance
454+
providerID: "aws:///us-west-2b/i-01234567adb205c7e"
455+
# The image (AMI) running on the instance
456+
imageID: ami-0ccbbed159cce4e37
457+
# Full capacity of the node
458+
capacity:
459+
cpu: "8"
460+
memory: "15155Mi"
461+
ephemeral-storage: "20Gi"
462+
pods: "58"
463+
# Allocatable resources available for pod scheduling
464+
allocatable:
465+
cpu: "7910m"
466+
memory: "14162Mi"
467+
ephemeral-storage: "17Gi"
468+
pods: "58"
469+
# Conditions track the lifecycle state of the NodeClaim
470+
conditions:
471+
- type: Launched
472+
status: "True"
473+
reason: Launched
474+
- type: Registered
475+
status: "True"
476+
reason: Registered
477+
- type: Initialized
478+
status: "True"
479+
reason: Initialized
480+
- type: Ready
481+
status: "True"
482+
reason: Ready
483+
- type: Consolidatable
484+
status: "True"
485+
reason: Consolidatable
486+
```
487+
488+
### metadata.labels
489+
490+
Labels on a NodeClaim come from two sources: labels defined in the NodePool's `spec.template.metadata.labels`, and well-known labels resolved during scheduling (such as `node.kubernetes.io/instance-type`, `topology.kubernetes.io/zone`, `karpenter.sh/capacity-type`, and `karpenter.sh/nodepool`). These labels are synced to the corresponding Kubernetes Node.
491+
492+
{{% alert title="Note" color="primary" %}}
493+
There is currently a limit of 100 on the total number of requirements on both the NodePool and the NodeClaim. Labels defined in `spec.template.metadata.labels` on the NodePool are propagated as requirements on the NodeClaim, meaning that you can't have more than 100 requirements and labels combined set on your NodePool.
494+
{{% /alert %}}
495+
496+
### metadata.annotations
497+
498+
Annotations on a NodeClaim are propagated from the NodePool's `spec.template.metadata.annotations`. Karpenter also adds internal annotations for tracking hash versions and cloud-provider-specific metadata. These annotations are synced to the corresponding Kubernetes Node.
499+
500+
### metadata.ownerReferences
501+
502+
Each Karpenter-created NodeClaim has an owner reference pointing to the NodePool that created it, with `blockOwnerDeletion: true`. This means deleting a NodePool will cascade-delete all of its NodeClaims.
503+
504+
### metadata.finalizers
505+
506+
Karpenter adds a `karpenter.sh/termination` finalizer to every NodeClaim. This finalizer ensures that when a NodeClaim is deleted, Karpenter properly cordons the node, drains pods, terminates the cloud provider instance, and cleans up the Node object before removing the finalizer.
507+
508+
### spec.nodeClassRef
509+
510+
This field references the cloud provider's NodeClass resource that defines provider-specific configuration for the instance. See [EC2NodeClasses]({{<ref "nodeclasses" >}}) for details.
511+
512+
The reference includes:
513+
* `group`: The API group of the NodeClass (e.g., `karpenter.k8s.aws`)
514+
* `kind`: The kind of the NodeClass (e.g., `EC2NodeClass`)
515+
* `name`: The name of the NodeClass resource
516+
517+
### spec.taints
518+
519+
Taints that are applied to the provisioned node. These are inherited from the NodePool's `spec.template.spec.taints`. Pods that don't tolerate these taints will not be scheduled on the node.
520+
See [Taints and Tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) for details.
521+
522+
### spec.startupTaints
523+
524+
Startup taints are applied to the node upon creation but are expected to be removed by an external system (e.g., a DaemonSet) after initialization completes. These are inherited from the NodePool's `spec.template.spec.startupTaints`. Pods do not need to tolerate startup taints to be considered for provisioning — Karpenter assumes they are temporary.
525+
526+
### spec.expireAfter
527+
528+
The duration a node can live on the cluster before Karpenter deletes it, inherited from the NodePool's `spec.template.spec.expireAfter`. Once the expiration time is reached, the node begins draining. This defaults to `720h` (30 days). Set to `Never` to disable expiration.
529+
530+
Changing `expireAfter` on the NodePool will cause existing NodeClaims to drift.
531+
532+
### spec.terminationGracePeriod
533+
534+
The maximum duration Karpenter will wait before forcefully deleting pods on a draining node, inherited from the NodePool's `spec.template.spec.terminationGracePeriod`. Once this period is reached, Karpenter will forcibly delete pods regardless of PodDisruptionBudgets or the `karpenter.sh/do-not-disrupt` annotation.
535+
536+
Karpenter will preemptively delete pods so their `terminationGracePeriodSeconds` aligns with the node's `terminationGracePeriod`. If left undefined, the controller will wait indefinitely for pods to drain.
537+
538+
Changing `terminationGracePeriod` on the NodePool will cause existing NodeClaims to drift.
539+
540+
### spec.requirements
541+
542+
The resolved scheduling requirements for the NodeClaim. These combine the NodePool's `spec.template.spec.requirements` with the scheduling constraints from the pods that triggered provisioning (nodeSelector, nodeAffinity, etc.). The same operators are supported: `In`, `NotIn`, `Exists`, `DoesNotExist`, `Gt`, `Lt`, `Gte`, and `Lte`.
543+
544+
These requirements represent the final constraints that were used to select the instance type and launch the node. They include well-known labels like `node.kubernetes.io/instance-type`, `topology.kubernetes.io/zone`, `kubernetes.io/arch`, and `karpenter.sh/capacity-type`.
545+
546+
### spec.resources
547+
548+
Resource requests represent the minimum resources needed to schedule the pods that triggered this NodeClaim. Karpenter uses these to right-size the instance.
549+
550+
* `spec.resources.requests`: The aggregate resource requests (e.g., `cpu`, `memory`, `pods`) from the pods being scheduled onto this NodeClaim.
551+
552+
### status.nodeName
553+
554+
The name of the Kubernetes Node object linked to this NodeClaim. This is populated after the instance registers with the cluster.
555+
556+
### status.providerID
557+
558+
The cloud provider identifier for the instance (e.g., `aws:///us-west-2b/i-01234567adb205c7e`). This is populated after the instance is launched.
559+
560+
### status.imageID
561+
562+
The identifier for the image running on the instance (e.g., an AMI ID like `ami-0ccbbed159cce4e37`).
563+
564+
### status.capacity
565+
566+
The estimated full capacity of the node, including resources reserved by the system. This includes `cpu`, `memory`, `ephemeral-storage`, `pods`, and any extended resources (e.g., `vpc.amazonaws.com/pod-eni`).
567+
568+
### status.allocatable
569+
570+
The estimated allocatable capacity of the node — the resources actually available for pod scheduling after system reservations. This is typically less than `status.capacity`.
571+
572+
### status.conditions
573+
574+
Conditions track the lifecycle state of the NodeClaim. Each condition has a `type`, `status` (`True`, `False`, or `Unknown`), `reason`, `message`, and `lastTransitionTime`.
575+
576+
| Condition Type | Description |
577+
|-----------------------|----------------------------------------------------------------------------------------------------------------------|
578+
| Launched | The cloud provider instance has been successfully created |
579+
| Registered | The instance has joined the cluster as a Kubernetes Node and Karpenter has synced labels, taints, and owner refs |
580+
| Initialized | The node is ready, startup taints have been removed, and all requested resources are registered |
581+
| Ready | Top-level condition indicating the NodeClaim is fully operational. True only when Launched, Registered, and Initialized are all True |
582+
| Consolidatable | The node is a candidate for consolidation (empty or underutilized) |
583+
| Drifted | The NodeClaim has drifted from its desired specification (e.g., NodePool or NodeClass changed) |
584+
| Drained | The node has been fully drained of pods during termination |
585+
| VolumesDetached | All volumes have been detached from the instance during termination |
586+
| InstanceTerminating | The cloud provider instance is in the process of being terminated |
587+
| ConsistentStateFound | Karpenter has verified the instance state is consistent with the NodeClaim |

0 commit comments

Comments
 (0)