RKE2 privateRegistriesConfig with insecureSkipVerify

[rokoshy]

What happened:

Cluster provisioning fails with state pending ,.The RKE2ControlPlane configuration contains privateRegistriesConfig with tls config with only insecureSkipVerify true without any secrets information .

``

privateRegistriesConfig: 
    <private registry> : 
         tls: 
            insecureSkipVerify: true

E1031 04:22:39.518589       1 controller.go:347] "Reconciler error" err="resource name may not be empty" controller="rke2config" controllerGroup="bootstrap.cluster.x-k8s.io" controllerKind="RKE2Config" RKE2Config="tb3-mw-system/tb3-mw-rvsws" namespace="tb3-smi-blr-mw-system" name="tb3-smi-blr-mw-rvsws" reconcileID="d80bd6d0-af45-4a3f-a2fa-6142b0348fa3" 

What did you expect to happen:

Expected : Cluster provisioning is successful
How to reproduce it:

Create RKE2 cluster with RKE2ControlPlane having atleast one privateRegistriesConfig..tls.insecureSkipVerify set as true without providing any secret

Anything else you would like to add:

This workes directly with RKE2 as write_files ( i.e keeping only the insecureSkipVerify: true without tls files) but swtiching to rke2 crd approach failed as the code is expected a secret also . The requirement is to ignore the certificates as its in the same system

Environment:

• rke provider version: v0.20.1

[github-actions]

This issue is stale because it has been open 90 days with no activity.

[AMEvers]

So, tracked this down. First off, according to this
// InsecureSkipVerify may be set to false to skip verifying the registry's certificate, default is true.
Which is the exact opposite behavior of insecure_skip_verify in rke2's config. This is extremely confusing to say the least but I realized it was a typo because, if you actually set it to false

 spec:
   template:
     spec:
       privateRegistriesConfig:
         configs:
           management.registry:
             tls:
               insecureSkipVerify: false

the boostrapper triggers a mutating webook that defaults the custom resource and results in this:

 spec:
   template:
     spec:
       privateRegistriesConfig:
         configs:
           management.registry:
             authSecret: {}
             tls:
               tlsConfigSecret: {}

and the rke2 bootstrap controller infinite loops, as it's admission webhook rejects the manifest for having unfilled authsecrets and tlsconfigs. You probably should stop using the custom defaulter and just use the admission webhook to reject the crd up front if it has missing required fields. I only even figured out what was happening because I happened to check the RKE2ConfigTemplate after creation and see that it had been changed. It's kind of unpleasant to have to dig through the apiserver's logging because your RKE2ConfigTemplate is missing a field instead of just having it reject the RKE2ConfigTemplate up front and directly tell you the reason. Btw, bootstrapper pod just logs that it's defaulting the RKE2ConfigTemplate and validating it in an infinite loop but never says it's failing it or why. The apiserver had the real failure. The way the bootstrapper is logging currently makes it really hard to know that your RKE2ConfigTemplate is invalid or why.

Here's the interesting bit. if you set insecureSkipVerify: true the defaulter still triggers, adding empty authSecret and tlsConfigSecret blocks which is probably what @rokoshy was fighting

 privateRegistriesConfig:
      configs:
        management.registry:
          authSecret: {}
          tls:
            insecureSkipVerify: true
            tlsConfigSecret: {}

This runs afoul of rke2ConfigRegistry.Client.Get throwing an error. I could see it in the kube apiserver logs.

E0301 19:58:24.480127 1 registries.go:69] "Error fetching TLS Secret" err="resource name may not be empty" controller="rke2config" controllerGroup="bootstrap.cluster.x-k8s.io" controllerKind="RKE2Config" RKE2Config="cluster/cluster-wkr-p2xw8-42kb7" namespace="cluster" name="cluster-wkr-p2xw8-42kb7"

On a whim, I referenced your e2e registry test and tried listing out the secrets, even though I wasn't going to use them.

 privateRegistriesConfig:
     configs:
       management.registry:
         authSecret: 
           name: management-registry-auth-secret
           namespace: cluster
         tls:
           insecureSkipVerify: true
           tlsConfigSecret: 
             name: management-registry-tls-secret
             namespace: cluster

and generated empty secrets.
kubectl -n cluster create secret generic management-registry-tls-secret --from-literal=ca.crt="" --from-literal=tls.key="" --from-literal=tls.crt=""
kubectl -n cluster create secret generic management-registry-auth-secret --from-literal=username="" --from-literal=password=""
Which FINALLY let the cluster generate. This very much seems like a bug in the custom defaulter's implementation but, in the off chance that this behavior is intentional, this is completely undocumented as far as I can tell. I only figured it out through several hours of trial and error and tracing the code.