Calico chart template (v3.32.1) bundled with v3.29.3 images causes CRD/RBAC mismatches in k0s v1.36.3

[ZYWNB666]

Bug: Calico chart template (v3.32.1) bundled with v3.29.3 images causes CRD/RBAC mismatches in k0s v1.36.3

k0s version

k0s v1.36.3+k0s.2

Platform

• Linux (Ubuntu 24.04 / Debian 12)
• Multi-node cluster using Calico CNI with VXLAN mode
• Nodes connected via Tailscale (using Tailscale IP as node-ip)

What happened

k0s v1.36.3 bundles a Calico chart template from v3.32.1 but deploys Calico v3.29.3 images. This version mismatch causes multiple issues:

1. Missing adminnetworkpolicies CRD: The v3.32.1 felix configuration watches adminnetworkpolicies and baselineadminnetworkpolicies resources (in the policy.networking.k8s.io API group), but k0s only creates 19 Calico CRDs in calico_init/ — these two CRDs are missing. Felix repeatedly fails to list these resources and never becomes ready.

   Error in calico-node logs:

   error=resource does not exist: KubernetesAdminNetworkPolicy with error: the server could not find the requested resource (get adminnetworkpolicies.policy.networking.k8s.io)
   

2. Missing RBAC for calico-kube-controllers: The ClusterRole calico-kube-controllers (from the v3.32.1 template) is missing permissions that the v3.32 controller code requires:

   • serviceaccounts (core API group) — needed by the namespace controller
   • networkpolicies (networking.k8s.io API group) — needed by the policy controller

   Errors in calico-kube-controllers logs:

   serviceaccounts is forbidden: User "system:serviceaccount:kube-system:calico-kube-controllers" cannot list resource "serviceaccounts" in API group "" at the cluster scope
   networkpolicies.networking.k8s.io is forbidden: User "system:serviceaccount:kube-system:calico-kube-controllers" cannot list resource "networkpolicies" in API group "networking.k8s.io" at the cluster scope
   

3. Invalid loadbalancer controller: The v3.32.1 template enables the loadbalancer controller by default, but the v3.29.3 kube-controllers binary doesn't support it:

   FATAL: Invalid controller 'loadbalancer' provided.
   

How to reproduce

1. Install k0s v1.36.3+k0s.2 with Calico CNI (default or explicit provider: calico)
2. Deploy a multi-node cluster
3. Observe calico-node pods stuck in 0/1 (felix not ready) and calico-kube-controllers in CrashLoopBackOff

Expected behavior

The bundled Calico chart template version should match the deployed Calico image version, OR the chart template should be backwards-compatible with the image version. All required CRDs and RBAC rules should be present.

Workaround

Manual post-deployment fixes are required:

1. Create adminnetworkpolicies and baselineadminnetworkpolicies CRDs manually
2. Replace the calico-kube-controllers ClusterRole with a complete set of permissions (including serviceaccounts and networkpolicies)
3. Set ENABLED_CONTROLLERS=node,policy,profile,workloadendpoint environment variable on calico-kube-controllers to disable the unsupported loadbalancer controller

Additional context

The chart template source is annotated in the manifest files:

# Source: https://github.com/projectcalico/calico/blob/v3.32.1/charts/calico/templates/calico-kube-controllers.yaml

But the deployed image is:

quay.io/calico/node:v3.29.3
quay.io/calico/kube-controllers:v3.29.3

This version mismatch affects all k0s v1.36.x deployments using the built-in Calico provider.

[twz123]

I see the right Calico versions in the constants. Can you please check your k0s config, if you have the images pinned to a previous version?

[ZYWNB666]

Great, I found these configurations in the source code, but the features I found in the documentation are very limited. It would be great if you could improve the official documentation.

[ZYWNB666]

Are there any plans to support headscale/tailscale?

[twz123]

Which docs were you looking at, and what were you hoping to find?

How about adding your findings and opening a draft PR, so we can work it out from there?