"Error: Required CRDs are missing. Please install the corresponding CRD chart before installing this chart." when trying to add extra Traefik configuration

[proffalken]

Hey folks,

I'm using k3s to get up to speed on k8s.

I've run the installation script, however I discovered that the Traefik dashboard was not available on port 9000 as documented. kubectl describe shows that Traefik is only listening on port 80 and 443.

To "solve" this, I have created a file named traefik-config.yaml with the content below and placed it in /var/lib/rancher/k3s/server/manifests.

My understanding from the documentation is that anything in this file will "overwrite" the existing Traefik config and take effect however, instead, I just get a crashloop and the error in the titlebar.

Config file:

# /var/lib/rancher/k3s/server/manifests/traefik-config.yaml
---
apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
  name: traefik
  namespace: kube-system
spec:
  valuesContent: |-
    additionalArguments:
      - "--accesslog"
      - "--api"
      - "--api.dashboard=true"
      - "--api.insecure=true"
      - "--log.level=DEBUG"

Full logs:

kubectl -n kube-system logs helm-install-traefik-85qnv


if [[ ${KUBERNETES_SERVICE_HOST} =~ .*:.* ]]; then
	echo "KUBERNETES_SERVICE_HOST is using IPv6"
	CHART="${CHART//%\{KUBERNETES_API\}%/[${KUBERNETES_SERVICE_HOST}]:${KUBERNETES_SERVICE_PORT}}"
else
	CHART="${CHART//%\{KUBERNETES_API\}%/${KUBERNETES_SERVICE_HOST}:${KUBERNETES_SERVICE_PORT}}"
fi

set +v -x
+ [[ '' != \t\r\u\e ]]
+ export HELM_HOST=127.0.0.1:44134
+ HELM_HOST=127.0.0.1:44134
+ tiller --listen=127.0.0.1:44134 --storage=secret
+ helm_v2 init --skip-refresh --client-only --stable-repo-url https://charts.helm.sh/stable/
Creating /home/klipper-helm/.helm 
Creating /home/klipper-helm/.helm/repository 
Creating /home/klipper-helm/.helm/repository/cache 
Creating /home/klipper-helm/.helm/repository/local 
Creating /home/klipper-helm/.helm/plugins 
Creating /home/klipper-helm/.helm/starters 
Creating /home/klipper-helm/.helm/cache/archive 
Creating /home/klipper-helm/.helm/repository/repositories.yaml 
Adding stable repo with URL: https://charts.helm.sh/stable/ 
Adding local repo with URL: http://127.0.0.1:8879/charts 
$HELM_HOME has been configured at /home/klipper-helm/.helm.
Not installing Tiller due to 'client-only' flag having been set
[main] 2023/09/05 15:33:14 Starting Tiller v2.17.0 (tls=false)
[main] 2023/09/05 15:33:14 GRPC listening on 127.0.0.1:44134
[main] 2023/09/05 15:33:14 Probes listening on :44135
[main] 2023/09/05 15:33:14 Storage driver is Secret
[main] 2023/09/05 15:33:14 Max history per release is 0
++ timeout -s KILL 30 helm_v2 ls --all '^traefik$' --output json
++ jq -r '.Releases | length'
[storage] 2023/09/05 15:33:14 listing all releases with filter
+ V2_CHART_EXISTS=
+ [[ '' == \1 ]]
+ [[ '' == \v\2 ]]
+ shopt -s nullglob
+ [[ -f /config/ca-file.pem ]]
+ [[ -f /tmp/ca-file.pem ]]
+ [[ -n '' ]]
+ helm_content_decode
+ set -e
+ ENC_CHART_PATH=/chart/traefik.tgz.base64
+ CHART_PATH=/tmp/traefik.tgz
+ [[ ! -f /chart/traefik.tgz.base64 ]]
+ return
+ [[ install != \d\e\l\e\t\e ]]
+ helm_repo_init
+ grep -q -e 'https\?://'
+ echo 'chart path is a url, skipping repo update'
chart path is a url, skipping repo update
+ helm_v3 repo remove stable
Error: no repositories configured
+ true
+ return
+ helm_update install --set-string global.systemDefaultRegistry=
+ [[ helm_v3 == \h\e\l\m\_\v\3 ]]
++ helm_v3 ls --all -f '^traefik$' --namespace kube-system --output json
++ jq -r '"\(.[0].app_version),\(.[0].status)"'
++ tr '[:upper:]' '[:lower:]'
+ LINE=v2.9.10,deployed
+ IFS=,
+ read -r INSTALLED_VERSION STATUS _
+ VALUES=
+ for VALUES_FILE in /config/*.yaml
+ VALUES=' --values /config/values-01_HelmChart.yaml'
+ for VALUES_FILE in /config/*.yaml
+ VALUES=' --values /config/values-01_HelmChart.yaml --values /config/values-10_HelmChartConfig.yaml'
+ [[ install = \d\e\l\e\t\e ]]
+ [[ v2.9.10 =~ ^(|null)$ ]]
+ [[ deployed =~ ^(pending-install|pending-upgrade|pending-rollback)$ ]]
+ [[ deployed == \d\e\p\l\o\y\e\d ]]
+ echo 'Already installed traefik'
+ [[ helm_v3 == \h\e\l\m\_\v\3 ]]
+ helm_v3 mapkubeapis traefik --namespace kube-system
Already installed traefik
2023/09/05 15:33:15 Release 'traefik' will be checked for deprecated or removed Kubernetes APIs and will be updated if necessary to supported API versions.
2023/09/05 15:33:15 Get release 'traefik' latest version.
2023/09/05 15:33:15 Check release 'traefik' for deprecated or removed APIs...
2023/09/05 15:33:15 Finished checking release 'traefik' for deprecated or removed APIs.
2023/09/05 15:33:15 Release 'traefik' has no deprecated or removed APIs.
2023/09/05 15:33:15 Map of release 'traefik' deprecated or removed APIs to supported versions, completed successfully.
+ echo 'Upgrading helm_v3 chart'
+ echo 'Upgrading traefik'
+ shift 1
Upgrading traefik
+ helm_v3 upgrade --set-string global.systemDefaultRegistry= traefik https://10.43.0.1:443/static/charts/traefik-21.2.1+up21.2.0.tgz --values /config/values-01_HelmChart.yaml --values /config/values-10_HelmChartConfig.yaml
Error: UPGRADE FAILED: execution error at (traefik/templates/validate-install-crd.yaml:19:7): Required CRDs are missing. Please install the corresponding CRD chart before installing this chart.

[brandond]

That is normal to see on the initial install attempt. There is another chart for the CRDs; once that install has completed, the main traefik chart will retry and the installation will succeed.

[proffalken]

Ah, ok, thanks. It's been running for a few hours now, is there another command I should be running?

[brandond]

no. You might look at the logs for the traefik crd install to see why that is failing? Normally it's done within seconds of the first server starting up.

[proffalken]

OK, thanks, I'll take a look when I'm back at my desk tomorrow :)

[davidhiebert]

You back at your desk yet? 🤣

[proffalken]

@davidhiebert not right now, no, but I have been... 👀

I can't remember how I solved this, and my ADHD means I will have forgotten to look again when I'm there tomorrow, but if by some fluke I do remember, I'll post an answer!