🐛 Measure nodeDeletionTimeout from node deletion start - #14026
🐛 Measure nodeDeletionTimeout from node deletion start#14026saisankargochhayat wants to merge 1 commit into
Conversation
|
|
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
Welcome @saisankargochhayat! |
|
Hi @saisankargochhayat. Thanks for your PR. I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with Regular contributors should join the org to skip this step. Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
/area machine |
7f9fe40 to
f0f2d2f
Compare
nodeDeletionTimeoutSeconds was evaluated against the Machine's deletionTimestamp, so node drain, volume detach and infrastructure deletion all spent the same budget. Since nodeDrainTimeoutSeconds and nodeVolumeDetachTimeoutSeconds default to 0 (unlimited) and infrastructure deletion has no timeout, the budget is typically already exhausted by the time Node deletion is attempted, so raising nodeDeletionTimeoutSeconds has no effect. Track nodeDeletionStartTime in status.deletion and measure the timeout from it, consistent with how nodeDrainTimeoutSeconds and nodeVolumeDetachTimeoutSeconds were changed in kubernetes-sigs#11166.
f0f2d2f to
ee79bc9
Compare
| // if the nodeDeletionTimeoutSeconds is exceeded. | ||
| // Only present when the Machine has a deletionTimestamp and deleting the Node had been started. | ||
| // +optional | ||
| NodeDeletionStartTime metav1.Time `json:"nodeDeletionStartTime,omitempty,omitzero"` |
There was a problem hiding this comment.
Should we be adding this? Not sure but it seems like we would in effect be creating drift for an established api that already exists through several versions.
There was a problem hiding this comment.
I was taking some inspiration from #14024 111614024 6, finding a way to persist phase specific times.
We need a way to persist the timestamp for when Node deletion actually begins; otherwise nodeDeletionTimeoutSeconds includes time spent draining, waiting for hooks/volumes, and deleting infrastructure.
Any recommendations on how to persist this phase transition without extending MachineDeletionStatus?
|
/hold |
|
Also, please abide by the AI usage policy https://kubernetes.io/blog/2026/06/26/open-source-maintainership-in-the-age-of-ai/ |
What this PR does / why we need it:
nodeDeletionTimeoutSeconds is evaluated against the Machine's deletionTimestamp, so node drain, volume detach and infrastructure deletion all spend the same budget. Since nodeDrainTimeoutSeconds and nodeVolumeDetachTimeoutSeconds default to 0 (unlimited) and infrastructure deletion has no timeout, the budget is typically already exhausted by the time Node deletion is attempted — so raising nodeDeletionTimeoutSeconds has no effect unless the value exceeds the time those phases consume.
This tracks nodeDeletionStartTime in status.deletion and measures the timeout from it, consistent with how nodeDrainTimeoutSeconds and nodeVolumeDetachTimeoutSeconds were changed in #11166.
Scope and behaviour change:
Which issue(s) this PR fixes (optional, in
fixes #<issue number>(, fixes #<issue_number>, ...)format, will close the issue(s) when PR gets merged):Fixes #14024
This PR was written in part with the assistance of generative AI.