feat: option to replicate shared resources to all project namespaces#5789
feat: option to replicate shared resources to all project namespaces#5789jessesuen wants to merge 8 commits intoakuity:mainfrom
Conversation
Adds a new shared-secrets replication reconciler to the management controller. Secrets in kargo-shared-resources annotated with kargo.akuity.io/replicate-to: "*" are automatically copied into every Project namespace, enabling Argo Rollouts AnalysisTemplates (and Job Pods) to consume them without cross-namespace secret references. Key behaviors: - Source secret gets kargo.akuity.io/replicated finalizer on first reconcile; deletion is blocked until all managed replicas are cleaned up. - Replicated secrets carry replicated-from and replicated-sha labels; the SHA label is a 16-char truncated SHA-256 of the secret data used to detect external modifications. - Externally modified replicas are never overwritten or deleted. - User-created secrets with the same name (no replicated-from label) are left untouched (conflict avoidance). - Removing the replicate-to annotation triggers the same cleanup path as source secret deletion. - A Project watch handler re-enqueues all annotated source secrets when a new Project appears, ensuring immediate replication. - Orphaned replicas in non-project namespaces are pruned on each reconcile pass. - kargo-shared-resources is now always included in the management controller's Secret namespace cache, independent of the legacy migration controller. Signed-off-by: Jesse Suen <jesse@akuity.io>
Signed-off-by: Jesse Suen <jesse@akuity.io>
- computeDataHash now factors in labels and annotations (excluding replication-managed labels and kubectl/replicate-to annotations) - syncToProjectNamespace carries source labels and annotations to replicas - List call in Reconcile filters to out-of-date replicas only (replicated-sha notin [sourceHash]) to skip unnecessary updates - Remove orphaned-namespace cleanup: Project deletion cascades to the namespace, which deletes replicated Secrets automatically Signed-off-by: Jesse Suen <jesse@akuity.io>
…nd ConfigMaps Signed-off-by: Jesse Suen <jesse@akuity.io>
…n on startup Signed-off-by: Jesse Suen <jesse@akuity.io>
Signed-off-by: Jesse Suen <jesse@akuity.io>
Signed-off-by: Jesse Suen <jesse@akuity.io>
Signed-off-by: Jesse Suen <jesse@akuity.io>
✅ Deploy Preview for docs-kargo-io ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
In addition to the unit tests, this was manually tested end-to-end to cover various corner cases, including:
|
| // AnnotationValueReplicateToAll is the annotation value for | ||
| // AnnotationKeyReplicateTo that causes a resource to be replicated to all | ||
| // Project namespaces. | ||
| AnnotationValueReplicateToAll = "*" |
There was a problem hiding this comment.
I am considering introducing an annotation, kargo.akuity.io/replicated-at: 2026-02-25T01:49:00Z timestamp to indicate when the resource was replicated. This might help end user (or even kargo devs) debug when things changed (e.g., if an admin updated the Secret/ConfigMap from underneath an AnalysisRun).
|
|
||
| if existing == nil { | ||
| // Either no replica exists yet, or it already has the current SHA | ||
| // (filtered out by the List). Try to create; handle AlreadyExists. |
There was a problem hiding this comment.
Is it necessary to call Create in the case where the replicated object exists with a matching SHA? If you were to not filter it, and compare the SHA here (which I see you do anyway later), could the Create call be avoided?
There was a problem hiding this comment.
I think you are right about this. I think this logic used to make sense before I added an optimization in the list query. I will revisit this.
|
|
||
| // Sync to each project namespace. | ||
| for ns := range projectNamespaces { | ||
| if err := r.syncToProjectNamespace(ctx, srcObj, ns, sourceHash, existingByNamespace[ns]); err != nil { |
| destObj.SetLabels(replicaLabels(src, sourceHash)) | ||
| destObj.SetAnnotations(replicaAnnotations(src)) | ||
| r.adapter.copyFields(destObj, src) | ||
| if err := r.client.Create(ctx, destObj); err != nil { |
There was a problem hiding this comment.
nit: set the FieldManager
Partially resolves #5788
Adds a new resource replication reconciler to the management controller.
Secrets&ConfigMapsin thekargo-shared-resourcesNamespace, annotated withkargo.akuity.io/replicate-to: "*"are automatically copied into every Project namespace. For example:This enables Argo Rollouts AnalysisTemplates (including Job Pods) to consume them, as it is not possible for cross-namespace secret references.
Key behaviors:
kargo.akuity.io/finalizerfinalizer on first reconcile; deletion is blocked until all managed replicas are cleaned up.replicated-fromandreplicated-shalabels; the SHA label is a 16-char truncated SHA-256 of the resource data.replicated-shalabel is used to detect and optimize when content updates are necessary to be copied to replicated resources. If sha is the same, the update is skipped.Remaining work: