Skip to content

Don't delete resources unless field managers match #3407

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

blampe
Copy link
Contributor

@blampe blampe commented Jan 3, 2025

This fixes the delete-after-create problem by reviving #2999 using field managers (as Eron suggested) instead of annotations to confirm ownership before deleting a resource.

If the resource was created with SSA, and we don't find the expected field manager, then we no-op instead of deleting the resource. The intended effect is to remove it from our state so the "upserted" manager can continue owning it.

This is technically a breaking change in behavior for some edge cases, but I think it is justified by the potential severity of our delete-after-create bug.

@blampe blampe added the impact/no-changelog-required This issue doesn't require a CHANGELOG update label Jan 3, 2025
Copy link

github-actions bot commented Jan 3, 2025

Does the PR have any schema changes?

Looking good! No breaking changes found.
No new resources/functions.

Copy link

codecov bot commented Jan 3, 2025

Codecov Report

Attention: Patch coverage is 88.23529% with 2 lines in your changes missing coverage. Please review.

Project coverage is 41.12%. Comparing base (b0ab343) to head (c0839d3).

Files with missing lines Patch % Lines
provider/pkg/await/await.go 88.23% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3407      +/-   ##
==========================================
+ Coverage   41.08%   41.12%   +0.03%     
==========================================
  Files          87       87              
  Lines       12900    12917      +17     
==========================================
+ Hits         5300     5312      +12     
- Misses       7205     7208       +3     
- Partials      395      397       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@blampe blampe force-pushed the blampe/delete-after-create-field-manager branch from 0a9b5d5 to 4817d79 Compare January 3, 2025 23:20
@blampe blampe force-pushed the blampe/delete-after-create-field-manager branch from 4817d79 to c0839d3 Compare January 3, 2025 23:49
}
actualSSAManagers = actualSSAManagers.Insert(f.Manager)
}
if c.ServerSideApply && !actualSSAManagers.Has(c.FieldManager) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As written this would also no-op in cases where the object was updated with CSA.

We can narrow this by also checking actualSSAManagers is non-empty, so we know the live object was at least touched by pulumi.

@blampe blampe requested review from EronWright and rquitales January 4, 2025 00:30
@EronWright
Copy link
Contributor

Note that Helm uses labels to track ownership and to drive adoption.
helm/helm#7649

@EronWright
Copy link
Contributor

EronWright commented Apr 8, 2025

What makes me uncomfortable is the idea that Pulumi being a field manager is evidence of ownership. Imagine that some other manager has taken over all but one field, say foo which Pulumi still manages. Should we still proceed to delete the resource?

Skipping deletion when there's other field managers would not work either, because the autoscaler that is controlling the replicas has no intention of owning the object.

I would advocate for adopting a Helm-like strategy of using ownership labels. The use of the standard managed-by label is quite interesting, because Pulumi and Helm would helpfully detect each other.

Note that Helm recently added a flag called --take-ownership to skip the conflict check. I filed #3595 to track adding support to Release/v4.

Interesting to note that Helm has special treatment of CRDs, e.g. they're never deleted. Consider doing the same:

See also: #3000

@blampe are you alright with closing this PR?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
impact/no-changelog-required This issue doesn't require a CHANGELOG update
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants