Bug: Stale SA token Secrets from K8s < 1.24 survive the crane pipeline
Summary
When migrating from a Kubernetes cluster < 1.24, auto-generated ServiceAccount token Secrets (kubernetes.io/service-account-token type) pass through the entire crane pipeline (export → transform → apply) and are applied to
the target cluster with invalid source-cluster credentials. The KubernetesPlugin does not whiteout these Secrets because they lack ownerReferences on older K8s versions.
Background
In K8s < 1.24, creating a ServiceAccount automatically generates:
- A Secret of type
kubernetes.io/service-account-token containing a JWT token signed by the source cluster's signing key, the source cluster's CA certificate, and the source SA's UID in annotations
- These auto-generated Secrets have no ownerReferences on K8s < 1.24
In K8s >= 1.24, these Secrets are no longer auto-generated. Pods use projected tokens via the TokenRequest API instead.
On OpenShift (all versions), kubernetes.io/dockercfg Secrets are also auto-generated per SA with internal registry credentials. On modern OCP (4.16+), these have ownerReferences and are correctly whiteout-ed. On older OCP,
they may not.
Tested Scenario
Source: minikube K8s 1.23 (auto-generated token Secrets, no ownerReferences)
Target: minikube K8s 1.34 (uses projected tokens)
Steps:
# Source cluster (K8s 1.23) has:
# - ServiceAccount "app-sa" with secrets: [{name: app-sa-token-xwzl7}]
# - Secret "app-sa-token-xwzl7" type: kubernetes.io/service-account-token (no ownerReferences)
# - Deployment "myapp" using serviceAccountName: app-sa
crane export --context old-k8s -n app-test
crane transform -e export
crane apply -e export
kubectl --context tgt apply -f output/output.yaml
Current Behavior
| Stage |
Token Secret app-sa-token-xwzl7 |
SA secrets field |
| Export |
Exported as a regular namespaced Secret |
Exported with secrets: [{name: app-sa-token-xwzl7}] |
| Transform |
Survives — no ownerReferences, not in whiteout list |
Preserved — no SA-specific field stripping |
| Apply (render) |
Included in output.yaml with stale token, CA cert, SA UID |
SA included with dangling Secret reference |
| kubectl apply (target >= 1.24) |
Created then immediately deleted by token controller (UID mismatch). No error surfaced. |
SA created with dangling reference to non-existent Secret |
| kubectl apply (target < 1.24) |
Persists with stale source-cluster credentials. Pods get invalid JWT token. |
SA references Secret with wrong token |
Issues on target after migration:
- Stale token Secret silently created and deleted: kubectl apply reports secret/app-sa-token-xwzl7 created (exit code 0), but the token controller immediately deletes it because the annotation
kubernetes.io/service-account.uid doesn't match the target SA's UID. No error is surfaced to the user.
- ServiceAccount has dangling Secret reference: The SA is migrated with secrets: [{name: app-sa-token-xwzl7}] pointing to a Secret that no longer exists on the target.
- On older target clusters (K8s < 1.24): The stale token Secret would persist. Pods would mount the source cluster's JWT token — signed by the wrong key, with the wrong CA cert. All API calls from the pod would fail
authentication.
Expected Behavior
- Token Secrets should be whiteout-ed during transform. Secrets of type kubernetes.io/service-account-token and kubernetes.io/dockercfg are always cluster-specific and auto-generated — they should never be migrated. The
target cluster's token controller will generate fresh, valid Secrets for the SA.
- SA secrets field should be stripped during transform. The secrets: [{name: app-sa-token-xwzl7}] reference carries over a source-specific Secret name. On the target, this creates a dangling reference. The field should be
removed so the target SA starts clean.
- The migrated app should work cleanly on the target without dangling references, silently deleted resources, or stale credentials. The SA should exist, pods should use the target cluster's token mechanism (projected tokens
on >= 1.24, auto-generated Secrets on < 1.24), and no source-cluster artifacts should remain.
Bug: Stale SA token Secrets from K8s < 1.24 survive the crane pipeline
Summary
When migrating from a Kubernetes cluster < 1.24, auto-generated ServiceAccount token Secrets (
kubernetes.io/service-account-tokentype) pass through the entire crane pipeline (export → transform → apply) and are applied tothe target cluster with invalid source-cluster credentials. The KubernetesPlugin does not whiteout these Secrets because they lack
ownerReferenceson older K8s versions.Background
In K8s < 1.24, creating a ServiceAccount automatically generates:
kubernetes.io/service-account-tokencontaining a JWT token signed by the source cluster's signing key, the source cluster's CA certificate, and the source SA's UID in annotationsIn K8s >= 1.24, these Secrets are no longer auto-generated. Pods use projected tokens via the TokenRequest API instead.
On OpenShift (all versions),
kubernetes.io/dockercfgSecrets are also auto-generated per SA with internal registry credentials. On modern OCP (4.16+), these have ownerReferences and are correctly whiteout-ed. On older OCP,they may not.
Tested Scenario
Source: minikube K8s 1.23 (auto-generated token Secrets, no ownerReferences)
Target: minikube K8s 1.34 (uses projected tokens)
Steps:
Current Behavior
app-sa-token-xwzl7secretsfieldsecrets: [{name: app-sa-token-xwzl7}]output.yamlwith stale token, CA cert, SA UIDIssues on target after migration:
kubernetes.io/service-account.uid doesn't match the target SA's UID. No error is surfaced to the user.
authentication.
Expected Behavior
target cluster's token controller will generate fresh, valid Secrets for the SA.
removed so the target SA starts clean.
on >= 1.24, auto-generated Secrets on < 1.24), and no source-cluster artifacts should remain.