Skip to content

docs: expand Termination Controller section - #9510

Open
ketanjani21 wants to merge 4 commits into
aws:mainfrom
ketanjani21:docs/expand-termination-controller-9172
Open

docs: expand Termination Controller section#9510
ketanjani21 wants to merge 4 commits into
aws:mainfrom
ketanjani21:docs/expand-termination-controller-9172

Conversation

@ketanjani21

Copy link
Copy Markdown

Fixes #9172

Description

Expands the previously ~7-line Termination Controller section on the Disruption concepts page. The new section covers:

  • Actors involved (NodeClaim Lifecycle Controller, Node Termination Controller, Terminator, Eviction Queue, and the Node Health Controller that overrides terminationGracePeriod for unhealthy nodes)
  • Workflow steps annotated with NodeClaim status conditions (Drained, VolumesDetached, InstanceTerminating) and their reason strings
  • State diagram showing condition transitions, including the TerminationGracePeriodElapsed volume-detach skip
  • Pod eviction priority ordering (non-critical non-DaemonSet → non-critical DaemonSet → critical non-DaemonSet → critical DaemonSet)
  • Volume detachment waiting behavior and its terminationGracePeriod-forced skip
  • Fast path when the underlying instance disappears before drain completes
  • How terminationGracePeriod is determined: NodePool inheritance vs. Node Auto Repair override
  • Timeline showing how the Node's terminationGracePeriod, the pod's terminationGracePeriodSeconds, and the Node's DeletionTimestamp interact

Also deletes website/static/termination-state-machine.png (out of date and no longer referenced) and removes its lone reference from designs/termination.md.

How was this change tested?

Documentation-only change. Will verify rendering via the Amplify preview.

Does this change impact docs?

  • Yes, PR includes docs updates
  • Yes, issue opened: #
  • No

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@ketanjani21
ketanjani21 requested a review from a team as a code owner August 16, 2026 01:16
@ketanjani21
ketanjani21 requested a review from ryan-mist August 16, 2026 01:16
@ketanjani21
ketanjani21 force-pushed the docs/expand-termination-controller-9172 branch 3 times, most recently from 817c55c to f437c6a Compare August 20, 2026 14:39

@DerekFrank DerekFrank left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great!

NodeClaim status conditions progress through the following states during termination. Reason strings shown are the values set by the controller. Transitions also emit Kubernetes events on the Node.

```
Node.DeletionTimestamp set

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets follow the rest of the diagrams and make this a png if possible

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Converted to png and added.


Node termination is coordinated by several controllers, each with a distinct responsibility:

| Actor | Responsibility |

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if we need this level of fidelity into the core code for the end user. I love the state machine\workflow bits of this, but we should keep the docs opaque on implementation

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the feedback, updated the PR.

* **Node Auto Repair override.** When [Node Auto Repair]({{<ref "#node-auto-repair" >}}) determines a node is unhealthy, the Health Controller overwrites the annotation with the current time, forcing the Termination Controller to skip drain and volume-detachment waits. If the annotation is already in the past, it is not overwritten.
* **Unset.** If neither the NodePool nor the Health Controller sets a `terminationGracePeriod`, the annotation is absent and Karpenter waits indefinitely for drain and volume detachment.

#### terminationGracePeriod timeline

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd also love to see a diagram for this, with a worked through example

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added a diagram, Thoughts?

@ketanjani21

Copy link
Copy Markdown
Author

@DerekFrank - Thank you for the feedback, I've updated the PR and it's ready for another review.

@ketanjani21
ketanjani21 requested a review from DerekFrank August 20, 2026 21:02
@ketanjani21
ketanjani21 force-pushed the docs/expand-termination-controller-9172 branch 5 times, most recently from dafb21e to bfb1296 Compare August 25, 2026 14:24
@ketanjani21
ketanjani21 force-pushed the docs/expand-termination-controller-9172 branch from bfb1296 to fef5e49 Compare August 27, 2026 16:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Expand Node Termination Documentation

2 participants