Add PSA configuration secret to provisioning resource set

[pedromfcarvalho]

When a k3s/RKE2 cluster is created with a non-default PSA configuration template, the configuration from that template is then stored in a secret, which is used by the control plane planner.

This secret is only updated by the webhook when the provisioning cluster object is updated, and it wasn't included in the backups created from the default resource set, so after a migration the planner would get stuck in an error state until the secret was re-created manually.

The secret name is defined in the webhook here.

This is not needed for RKE1 because in that case the configuration from the template is added to the management v3 cluster object, which is already backed-up, and not a secret.

Issue: rancher/rancher#48884

QA Suggestions

Suggested steps:

1. Install Rancher
2. Create a rke2 cluster using a non default PSACT configuration (cluster configuration -> basics -> security, select either rancher-privileged or rancher-restricted).
3. In the local cluster, note the name of the secret <downstream-cluster-name>-admission-configuration-psact (in the fleet-default namespace.
4. In the local cluster, install the new Rancher Backup chart with the fix.
5. Create two backups.
   1. One using the full resource set
   2. Another using a custom resource set, then selecting the preinstalled set called rancher-resource-set.
   3. If the UI has no option to select the resource set, edit the yaml directly (field resourceSetName, values rancher-resource-set-full and rancher-resource-set).
6. Get each backup file.
7. For each backup, install the new backup chart and restore the backup and then install rancher in a fresh kubernetes cluster. It's important to do a migration and not a regular restore in the original cluster, because the bug only happened in this case.
   • In each case, after restoring the backup, but before installing rancher, check with kubectl if the admission-configuration-psact secret is present.
   • The new chart can be installed from the rancher/charts repo.
8. In each case, check that after installing rancher:
   • The downstream cluster with the PSACT is not stuck in a provisioning state [1].
   • That the rancher logs don't show any logs of the type "error during plan processing: error retrieving secret ... admission-configuration-psact...".

[1]

[jbiers]

LGTM. Just for context, the default resourceSet will stop receiving any updates and be treated as deprecated once rancher/dashboard#12997 is completed. This is part of a larger effort described in #607.

[pedromfcarvalho]

Rebased because of a merge conflict.

[mallardduck]

/backport release/v6.x

[github-actions]

Not creating port PR, there was an error running git am -3:

Applying: Add PSA configuration secret to provisioning resource set
Using index info to reconstruct a base tree...
M	charts/rancher-backup/files/default-resourceset-contents/provisioningv2.yaml
A	charts/rancher-backup/files/sensitive-resourceset-contents/provisioningv2.yaml
A	e2e/test/data/rancher-resource-set-full.yaml
Falling back to patching base and 3-way merge...
CONFLICT (modify/delete): e2e/test/data/rancher-resource-set-full.yaml deleted in HEAD and modified in Add PSA configuration secret to provisioning resource set. Version Add PSA configuration secret to provisioning resource set of e2e/test/data/rancher-resource-set-full.yaml left in tree.
CONFLICT (modify/delete): charts/rancher-backup/files/sensitive-resourceset-contents/provisioningv2.yaml deleted in HEAD and modified in Add PSA configuration secret to provisioning resource set. Version Add PSA configuration secret to provisioning resource set of charts/rancher-backup/files/sensitive-resourceset-contents/provisioningv2.yaml left in tree.
Auto-merging charts/rancher-backup/files/default-resourceset-contents/provisioningv2.yaml
CONFLICT (content): Merge conflict in charts/rancher-backup/files/default-resourceset-contents/provisioningv2.yaml
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config set advice.mergeConflict false"
Patch failed at 0001 Add PSA configuration secret to provisioning resource set