Unable to pull images from Zarf image registry due to missing secret in namespace

[elvinasp]

Hello,

trying to package Rancher install with Zarf for an AirGapped install and struggling with pulling images from Zarf internal registry. Zarf does automatically inject imagePullSecret into Deployments for the images that were deployed by Zarf package. But secret is deployed only to namespaces created by Zarf. This makes sense until package with "shadow" dependencies as Rancher is deployed, i.e. package does post deployment actions after it is deployed and creates own namespaces.

Problem is that such namespaces must have Zarf secret too, but some namespaces CANNOT be created by Zarf in advance as this breaks Rancher component deployment. Such components like Rancher Turtles will attempt to create their own namespaces and will fail if the namespace is created by Zarf.

Are there any way to control secret insertion to all new namespaces? Or is there any other way?

So far I see only "try and fail approach" to insert missing annotations and labels for each such namespace and expect that it will not use some dynamically generated label or there will be no conflicting labels, which seems to be the case with app.kubernetes.io/managed-by.

[AustinAbro321]

This might be a bit hacky, but would things work if after you install the rancher comopnents you had a Zarf component like this which installed the namespaces as kubernetes files.

components:
  - name: rancher-namespaces
    required: true
    manifests:
      - name: rancher-namespaces
        namespace: rancher-namespace
        files:
          - rancher-namespace.yaml

Another cleaner option might be to change zarf tools update-creds registry to add credentials to every namespace with a label zarf.dev/mutate, then you could manually add that label and run the command. There is no operator from Zarf that scans and adds the secrets to namespaces, so we likely will need an imperative approach, unless you can contribute back to your tool to allow it to adopt an existing namespace

[elvinasp]

Hello,

for now I have resolved with dirty hack like this:

1. Use manifest to create all namespaces that can possibly be used by Rancher in advance. Lucky for me, Rancher have posted a list of such namespaces somewhere in their documentation portal. Namespaces in the manifest use all the necessary annotations and labels to keep Rancher deployment charts happy.
2. Deploy Rancher component via Helm chart.
3. Use onDeploy->after component action to patch namespaces that requires explicit labeling and replace label value inserted by Zarf to the one expected by Rancher.

This seems to work as Rancher dependency deployment starts after Rancher is initialized so there are time to run small inline cmd to run this:

             LIST="cattle-capi-system"
              for N in $LIST; do
                zarf tools kubectl patch namespace $N --patch '{"metadata": {"labels": {"app.kubernetes.io/managed-by": "Helm"}}}'
              done

Removal is also handled in similar way by running custom Rancher cleanup script which runs over the huge list of namespaces and cleans up CRDs, roles, api resources and etc. before deleting namespaces.

Not pretty, but so far it seems to be working.

But I am open for some more proper way of doing. :)