diff --git a/website/content/en/docs/concepts/disruption.md b/website/content/en/docs/concepts/disruption.md index 81759b29ff3a..4054d9a021a2 100644 --- a/website/content/en/docs/concepts/disruption.md +++ b/website/content/en/docs/concepts/disruption.md @@ -334,7 +334,23 @@ Duration and Schedule must be defined together. When omitted, the budget is alwa ### Pod-Level Controls -You can block Karpenter from voluntarily disrupting and draining pods by adding the `karpenter.sh/do-not-disrupt: "true"` annotation to the pod. +Pods with blocking PDBs will not be evicted by the [Termination Controller]({{}}) or be considered for voluntary disruption actions. When multiple pods on a node have different PDBs, none of the PDBs may be blocking for Karpenter to voluntary disrupt a node. This can create complex eviction scenarios: + - If a pod matches multiple PDBs (via label selectors), ALL of these PDBs must allow for disruption + - When different pods on the same node belong to different PDBs, ALL PDBs must simultaneously permit eviction + - A single blocking PDB can prevent the entire node from being voluntary disrupted + +For example, consider a node with these pods and PDBs: +- Pod A: Matches PDB-1 (maxUnavailable: 0) and PDB-2 (maxUnavailable: 1) +- Pod B: Matches PDB-3 (minAvailable: 100%) +- Pod C: No PDB + +In this scenario, Karpenter cannot voluntary disrupt the node because: +1. Pod A is blocked by PDB-1 even though PDB-2 would allow disruption +2. Pod B is blocked by PDB-3's requirement for 100% availability + +As seen in this example, the more PDBs there are affecting a Node, the more difficult it will be for Karpenter to find an opportunity to perform voluntary disruption actions. + +Secondly, you can block Karpenter from voluntarily disrupting and draining pods by adding the `karpenter.sh/do-not-disrupt: "true"` annotation to the pod. You can treat this annotation as a single-pod, permanently blocking PDB. This has the following consequences: - Nodes with `karpenter.sh/do-not-disrupt` pods will be excluded from [Consolidation]({{}}), and conditionally excluded from [Drift]({{}}). diff --git a/website/content/en/preview/concepts/disruption.md b/website/content/en/preview/concepts/disruption.md index 81759b29ff3a..4054d9a021a2 100644 --- a/website/content/en/preview/concepts/disruption.md +++ b/website/content/en/preview/concepts/disruption.md @@ -334,7 +334,23 @@ Duration and Schedule must be defined together. When omitted, the budget is alwa ### Pod-Level Controls -You can block Karpenter from voluntarily disrupting and draining pods by adding the `karpenter.sh/do-not-disrupt: "true"` annotation to the pod. +Pods with blocking PDBs will not be evicted by the [Termination Controller]({{}}) or be considered for voluntary disruption actions. When multiple pods on a node have different PDBs, none of the PDBs may be blocking for Karpenter to voluntary disrupt a node. This can create complex eviction scenarios: + - If a pod matches multiple PDBs (via label selectors), ALL of these PDBs must allow for disruption + - When different pods on the same node belong to different PDBs, ALL PDBs must simultaneously permit eviction + - A single blocking PDB can prevent the entire node from being voluntary disrupted + +For example, consider a node with these pods and PDBs: +- Pod A: Matches PDB-1 (maxUnavailable: 0) and PDB-2 (maxUnavailable: 1) +- Pod B: Matches PDB-3 (minAvailable: 100%) +- Pod C: No PDB + +In this scenario, Karpenter cannot voluntary disrupt the node because: +1. Pod A is blocked by PDB-1 even though PDB-2 would allow disruption +2. Pod B is blocked by PDB-3's requirement for 100% availability + +As seen in this example, the more PDBs there are affecting a Node, the more difficult it will be for Karpenter to find an opportunity to perform voluntary disruption actions. + +Secondly, you can block Karpenter from voluntarily disrupting and draining pods by adding the `karpenter.sh/do-not-disrupt: "true"` annotation to the pod. You can treat this annotation as a single-pod, permanently blocking PDB. This has the following consequences: - Nodes with `karpenter.sh/do-not-disrupt` pods will be excluded from [Consolidation]({{}}), and conditionally excluded from [Drift]({{}}). diff --git a/website/content/en/v0.32/concepts/disruption.md b/website/content/en/v0.32/concepts/disruption.md index 1a7ef328cde5..31b67d853e91 100644 --- a/website/content/en/v0.32/concepts/disruption.md +++ b/website/content/en/v0.32/concepts/disruption.md @@ -180,7 +180,23 @@ To enable interruption handling, configure the `--interruption-queue` CLI argume ### Pod-Level Controls -You can block Karpenter from voluntarily choosing to disrupt certain pods by setting the `karpenter.sh/do-not-disrupt: "true"` annotation on the pod. This is useful for pods that you want to run from start to finish without disruption. By opting pods out of this disruption, you are telling Karpenter that it should not voluntarily remove a node containing this pod. +Pods with blocking PDBs will not be evicted by the [Termination Controller]({{}}) or be considered for voluntary disruption actions. When multiple pods on a node have different PDBs, none of the PDBs may be blocking for Karpenter to voluntary disrupt a node. This can create complex eviction scenarios: + - If a pod matches multiple PDBs (via label selectors), ALL of these PDBs must allow for disruption + - When different pods on the same node belong to different PDBs, ALL PDBs must simultaneously permit eviction + - A single blocking PDB can prevent the entire node from being voluntary disrupted + +For example, consider a node with these pods and PDBs: +- Pod A: Matches PDB-1 (maxUnavailable: 0) and PDB-2 (maxUnavailable: 1) +- Pod B: Matches PDB-3 (minAvailable: 100%) +- Pod C: No PDB + +In this scenario, Karpenter cannot voluntary disrupt the node because: +1. Pod A is blocked by PDB-1 even though PDB-2 would allow disruption +2. Pod B is blocked by PDB-3's requirement for 100% availability + +As seen in this example, the more PDBs there are affecting a Node, the more difficult it will be for Karpenter to find an opportunity to perform voluntary disruption actions. + +Secondly, you can block Karpenter from voluntarily choosing to disrupt certain pods by setting the `karpenter.sh/do-not-disrupt: "true"` annotation on the pod. This is useful for pods that you want to run from start to finish without disruption. By opting pods out of this disruption, you are telling Karpenter that it should not voluntarily remove a node containing this pod. Examples of pods that you might want to opt-out of disruption include an interactive game that you don't want to interrupt or a long batch job (such as you might have with machine learning) that would need to start over if it were interrupted. diff --git a/website/content/en/v1.0/concepts/disruption.md b/website/content/en/v1.0/concepts/disruption.md index 4d23048f63f4..e37077110c32 100644 --- a/website/content/en/v1.0/concepts/disruption.md +++ b/website/content/en/v1.0/concepts/disruption.md @@ -303,7 +303,23 @@ Duration and Schedule must be defined together. When omitted, the budget is alwa ### Pod-Level Controls -You can block Karpenter from voluntarily disrupting and draining pods by adding the `karpenter.sh/do-not-disrupt: "true"` annotation to the pod. +Pods with blocking PDBs will not be evicted by the [Termination Controller]({{}}) or be considered for voluntary disruption actions. When multiple pods on a node have different PDBs, none of the PDBs may be blocking for Karpenter to voluntary disrupt a node. This can create complex eviction scenarios: + - If a pod matches multiple PDBs (via label selectors), ALL of these PDBs must allow for disruption + - When different pods on the same node belong to different PDBs, ALL PDBs must simultaneously permit eviction + - A single blocking PDB can prevent the entire node from being voluntary disrupted + +For example, consider a node with these pods and PDBs: +- Pod A: Matches PDB-1 (maxUnavailable: 0) and PDB-2 (maxUnavailable: 1) +- Pod B: Matches PDB-3 (minAvailable: 100%) +- Pod C: No PDB + +In this scenario, Karpenter cannot voluntary disrupt the node because: +1. Pod A is blocked by PDB-1 even though PDB-2 would allow disruption +2. Pod B is blocked by PDB-3's requirement for 100% availability + +As seen in this example, the more PDBs there are affecting a Node, the more difficult it will be for Karpenter to find an opportunity to perform voluntary disruption actions. + +Secondly, you can block Karpenter from voluntarily disrupting and draining pods by adding the `karpenter.sh/do-not-disrupt: "true"` annotation to the pod. You can treat this annotation as a single-pod, permanently blocking PDB. This has the following consequences: - Nodes with `karpenter.sh/do-not-disrupt` pods will be excluded from [Consolidation]({{}}), and conditionally excluded from [Drift]({{}}). diff --git a/website/content/en/v1.1/concepts/disruption.md b/website/content/en/v1.1/concepts/disruption.md index 4d23048f63f4..e37077110c32 100644 --- a/website/content/en/v1.1/concepts/disruption.md +++ b/website/content/en/v1.1/concepts/disruption.md @@ -303,7 +303,23 @@ Duration and Schedule must be defined together. When omitted, the budget is alwa ### Pod-Level Controls -You can block Karpenter from voluntarily disrupting and draining pods by adding the `karpenter.sh/do-not-disrupt: "true"` annotation to the pod. +Pods with blocking PDBs will not be evicted by the [Termination Controller]({{}}) or be considered for voluntary disruption actions. When multiple pods on a node have different PDBs, none of the PDBs may be blocking for Karpenter to voluntary disrupt a node. This can create complex eviction scenarios: + - If a pod matches multiple PDBs (via label selectors), ALL of these PDBs must allow for disruption + - When different pods on the same node belong to different PDBs, ALL PDBs must simultaneously permit eviction + - A single blocking PDB can prevent the entire node from being voluntary disrupted + +For example, consider a node with these pods and PDBs: +- Pod A: Matches PDB-1 (maxUnavailable: 0) and PDB-2 (maxUnavailable: 1) +- Pod B: Matches PDB-3 (minAvailable: 100%) +- Pod C: No PDB + +In this scenario, Karpenter cannot voluntary disrupt the node because: +1. Pod A is blocked by PDB-1 even though PDB-2 would allow disruption +2. Pod B is blocked by PDB-3's requirement for 100% availability + +As seen in this example, the more PDBs there are affecting a Node, the more difficult it will be for Karpenter to find an opportunity to perform voluntary disruption actions. + +Secondly, you can block Karpenter from voluntarily disrupting and draining pods by adding the `karpenter.sh/do-not-disrupt: "true"` annotation to the pod. You can treat this annotation as a single-pod, permanently blocking PDB. This has the following consequences: - Nodes with `karpenter.sh/do-not-disrupt` pods will be excluded from [Consolidation]({{}}), and conditionally excluded from [Drift]({{}}). diff --git a/website/content/en/v1.2/concepts/disruption.md b/website/content/en/v1.2/concepts/disruption.md index 81759b29ff3a..bed634c024e9 100644 --- a/website/content/en/v1.2/concepts/disruption.md +++ b/website/content/en/v1.2/concepts/disruption.md @@ -334,12 +334,28 @@ Duration and Schedule must be defined together. When omitted, the budget is alwa ### Pod-Level Controls -You can block Karpenter from voluntarily disrupting and draining pods by adding the `karpenter.sh/do-not-disrupt: "true"` annotation to the pod. +Pods with blocking PDBs will not be evicted by the [Termination Controller]({{}}) or be considered for voluntary disruption actions. When multiple pods on a node have different PDBs, none of the PDBs may be blocking for Karpenter to voluntary disrupt a node. This can create complex eviction scenarios: + - If a pod matches multiple PDBs (via label selectors), ALL of these PDBs must allow for disruption + - When different pods on the same node belong to different PDBs, ALL PDBs must simultaneously permit eviction + - A single blocking PDB can prevent the entire node from being voluntary disrupted + +For example, consider a node with these pods and PDBs: +- Pod A: Matches PDB-1 (maxUnavailable: 0) and PDB-2 (maxUnavailable: 1) +- Pod B: Matches PDB-3 (minAvailable: 100%) +- Pod C: No PDB + +In this scenario, Karpenter cannot voluntary disrupt the node because: +1. Pod A is blocked by PDB-1 even though PDB-2 would allow disruption +2. Pod B is blocked by PDB-3's requirement for 100% availability + +As seen in this example, the more PDBs there are affecting a Node, the more difficult it will be for Karpenter to find an opportunity to perform voluntary disruption actions. + +Secondly, you can block Karpenter from voluntarily disrupting and draining pods by adding the `karpenter.sh/do-not-disrupt: "true"` annotation to the pod. You can treat this annotation as a single-pod, permanently blocking PDB. This has the following consequences: - Nodes with `karpenter.sh/do-not-disrupt` pods will be excluded from [Consolidation]({{}}), and conditionally excluded from [Drift]({{}}). - If the Node's owning NodeClaim has a [`terminationGracePeriod`]({{}}) configured, it will still be eligible for disruption via drift. -- Like pods with a blocking PDB, pods with the `karpenter.sh/do-not-disrupt` annotation will **not** be gracefully evicted by the [Termination Controller]({{ref "#terminationcontroller"}}). +- Like pods with a blocking PDB, pods with the `karpenter.sh/do-not-disrupt` annotation will **not** be gracefully evicted by the [Termination Controller]({{}}). Karpenter will not be able to complete termination of the node until one of the following conditions is met: - All pods with the `karpenter.sh/do-not-disrupt` annotation are removed. - All pods with the `karpenter.sh/do-not-disrupt` annotation have entered a [terminal phase](https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/#pod-phase) (`Succeeded` or `Failed`). diff --git a/website/content/en/v1.3/concepts/disruption.md b/website/content/en/v1.3/concepts/disruption.md index 81759b29ff3a..4054d9a021a2 100644 --- a/website/content/en/v1.3/concepts/disruption.md +++ b/website/content/en/v1.3/concepts/disruption.md @@ -334,7 +334,23 @@ Duration and Schedule must be defined together. When omitted, the budget is alwa ### Pod-Level Controls -You can block Karpenter from voluntarily disrupting and draining pods by adding the `karpenter.sh/do-not-disrupt: "true"` annotation to the pod. +Pods with blocking PDBs will not be evicted by the [Termination Controller]({{}}) or be considered for voluntary disruption actions. When multiple pods on a node have different PDBs, none of the PDBs may be blocking for Karpenter to voluntary disrupt a node. This can create complex eviction scenarios: + - If a pod matches multiple PDBs (via label selectors), ALL of these PDBs must allow for disruption + - When different pods on the same node belong to different PDBs, ALL PDBs must simultaneously permit eviction + - A single blocking PDB can prevent the entire node from being voluntary disrupted + +For example, consider a node with these pods and PDBs: +- Pod A: Matches PDB-1 (maxUnavailable: 0) and PDB-2 (maxUnavailable: 1) +- Pod B: Matches PDB-3 (minAvailable: 100%) +- Pod C: No PDB + +In this scenario, Karpenter cannot voluntary disrupt the node because: +1. Pod A is blocked by PDB-1 even though PDB-2 would allow disruption +2. Pod B is blocked by PDB-3's requirement for 100% availability + +As seen in this example, the more PDBs there are affecting a Node, the more difficult it will be for Karpenter to find an opportunity to perform voluntary disruption actions. + +Secondly, you can block Karpenter from voluntarily disrupting and draining pods by adding the `karpenter.sh/do-not-disrupt: "true"` annotation to the pod. You can treat this annotation as a single-pod, permanently blocking PDB. This has the following consequences: - Nodes with `karpenter.sh/do-not-disrupt` pods will be excluded from [Consolidation]({{}}), and conditionally excluded from [Drift]({{}}).