Cluster failed to start with single or mutilples nodes and timeout with high CPU usage

[ckf-ateme]

What happened:

The cluster failed to start after 4 min of timeout. 100% CPU was used while the container was running.
Error extract (full logs attached)

[control-plane-check] kube-apiserver is not healthy after 4m0.000261894s

What you expected to happen:

The cluster shall start and not eat my CPU

How to reproduce it (as minimally and precisely as possible):
My config is the following. And I created the cluster with no further options

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
networking:
  # Make it accessible from the runner
  apiServerAddress: "0.0.0.0"
  # For simplicity, we use the default port
  apiServerPort: 6443
nodes:
  - role: control-plane

Anything else we need to know?:
Here you’ll find the attached logs
kind-logs.zip

Environment:

• kind version: (use kind version): kind v0.30.0 go1.24.6 linux/amd64
• Runtime info: (use docker info, podman info or nerdctl info): docker v 28.5.1
• OS (e.g. from /etc/os-release): Ubuntu 22.04.5 LTS"
• Kubernetes version: (use kubectl version): Client Version: v1.33.1
• Any proxies or other special environment settings?: No

Thank you for your help :)

[stmcginnis]

Runtime info: (use docker info, podman info or nerdctl info): docker v 28.5.1

Please include the full output from the example commands.

Attached zip file is empty, so not much to go on here. But if this just happened, right now there is an AWS outage that is impacting many different services like Docker Hub and others. It's possible this wasn't able to pull something. Try again after services are restored and see if you get the same failure.

https://health.aws.amazon.com/health/status

/remove-kind bug
/kind support

[ckf-ateme]

Sorry for the short logs, I thought my zip contained everything. I’ve just realized I forgot -r in my zip command.
Here is the full logs hopefully
kind-logs.zip

Here is the detailed log of the command

kind create cluster --config kind-config.yaml         
Creating cluster "kind" ...
 ✓ Ensuring node image (kindest/node:v1.34.0) 🖼 
 ✓ Preparing nodes 📦  
 ✓ Writing configuration 📜 
 ✗ Starting control-plane 🕹️ 
Deleted nodes: ["kind-control-plane"]
ERROR: failed to create cluster: failed to init node with kubeadm: command "docker exec --privileged kind-control-plane kubeadm init --config=/kind/kubeadm.conf --skip-token-print --v=6" failed with error: exit status 1
Command Output: I1020 12:24:28.755176     190 initconfiguration.go:260] loading configuration from "/kind/kubeadm.conf"
W1020 12:24:28.755787     190 common.go:100] your configuration file uses a deprecated API spec: "kubeadm.k8s.io/v1beta3" (kind: "ClusterConfiguration"). Please use 'kubeadm config migrate --old-config old-config-file --new-config new-config-file', which will write the new, similar spec using a newer API version.
W1020 12:24:28.756040     190 common.go:100] your configuration file uses a deprecated API spec: "kubeadm.k8s.io/v1beta3" (kind: "InitConfiguration"). Please use 'kubeadm config migrate --old-config old-config-file --new-config new-config-file', which will write the new, similar spec using a newer API version.
W1020 12:24:28.756240     190 common.go:100] your configuration file uses a deprecated API spec: "kubeadm.k8s.io/v1beta3" (kind: "JoinConfiguration"). Please use 'kubeadm config migrate --old-config old-config-file --new-config new-config-file', which will write the new, similar spec using a newer API version.
W1020 12:24:28.756370     190 initconfiguration.go:361] [config] WARNING: Ignored configuration document with GroupVersionKind kubeadm.k8s.io/v1beta3, Kind=JoinConfiguration
I1020 12:24:28.757624     190 certs.go:111] creating a new certificate authority for ca
[init] Using Kubernetes version: v1.34.0
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
I1020 12:24:29.158945     190 certs.go:472] validating certificate period for ca certificate
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [kind-control-plane kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local localhost] and IPs [10.96.0.1 172.240.1.2 0.0.0.0]
[certs] Generating "apiserver-kubelet-client" certificate and key
I1020 12:24:29.775129     190 certs.go:111] creating a new certificate authority for front-proxy-ca
[certs] Generating "front-proxy-ca" certificate and key
I1020 12:24:29.825167     190 certs.go:472] validating certificate period for front-proxy-ca certificate
[certs] Generating "front-proxy-client" certificate and key
I1020 12:24:30.347855     190 certs.go:111] creating a new certificate authority for etcd-ca
[certs] Generating "etcd/ca" certificate and key
I1020 12:24:30.580277     190 certs.go:472] validating certificate period for etcd/ca certificate
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [kind-control-plane localhost] and IPs [172.240.1.2 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [kind-control-plane localhost] and IPs [172.240.1.2 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
I1020 12:24:31.345643     190 certs.go:77] creating new public/private key files for signing service account users
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I1020 12:24:31.496988     190 kubeconfig.go:111] creating kubeconfig file for admin.conf
[kubeconfig] Writing "admin.conf" kubeconfig file
I1020 12:24:31.658333     190 kubeconfig.go:111] creating kubeconfig file for super-admin.conf
[kubeconfig] Writing "super-admin.conf" kubeconfig file
I1020 12:24:31.834607     190 kubeconfig.go:111] creating kubeconfig file for kubelet.conf
[kubeconfig] Writing "kubelet.conf" kubeconfig file
I1020 12:24:31.978633     190 kubeconfig.go:111] creating kubeconfig file for controller-manager.conf
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
I1020 12:24:32.065598     190 kubeconfig.go:111] creating kubeconfig file for scheduler.conf
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
I1020 12:24:32.217282     190 local.go:67] [etcd] wrote Static Pod manifest for a local etcd member to "/etc/kubernetes/manifests/etcd.yaml"
I1020 12:24:32.217303     190 manifests.go:125] [control-plane] getting StaticPodSpecs
I1020 12:24:32.217574     190 certs.go:472] validating certificate period for CA certificate
I1020 12:24:32.217623     190 manifests.go:151] [control-plane] adding volume "ca-certs" for component "kube-apiserver"
I1020 12:24:32.217629     190 manifests.go:151] [control-plane] adding volume "etc-ca-certificates" for component "kube-apiserver"
I1020 12:24:32.217633     190 manifests.go:151] [control-plane] adding volume "k8s-certs" for component "kube-apiserver"
I1020 12:24:32.217636     190 manifests.go:151] [control-plane] adding volume "usr-local-share-ca-certificates" for component "kube-apiserver"
I1020 12:24:32.217639     190 manifests.go:151] [control-plane] adding volume "usr-share-ca-certificates" for component "kube-apiserver"
I1020 12:24:32.218665     190 manifests.go:180] [control-plane] wrote static Pod manifest for component "kube-apiserver" to "/etc/kubernetes/manifests/kube-apiserver.yaml"
I1020 12:24:32.218679     190 manifests.go:125] [control-plane] getting StaticPodSpecs
[control-plane] Creating static Pod manifest for "kube-controller-manager"
I1020 12:24:32.218795     190 manifests.go:151] [control-plane] adding volume "ca-certs" for component "kube-controller-manager"
I1020 12:24:32.218799     190 manifests.go:151] [control-plane] adding volume "etc-ca-certificates" for component "kube-controller-manager"
I1020 12:24:32.218802     190 manifests.go:151] [control-plane] adding volume "flexvolume-dir" for component "kube-controller-manager"
I1020 12:24:32.218804     190 manifests.go:151] [control-plane] adding volume "k8s-certs" for component "kube-controller-manager"
I1020 12:24:32.218807     190 manifests.go:151] [control-plane] adding volume "kubeconfig" for component "kube-controller-manager"
I1020 12:24:32.218809     190 manifests.go:151] [control-plane] adding volume "usr-local-share-ca-certificates" for component "kube-controller-manager"
I1020 12:24:32.218812     190 manifests.go:151] [control-plane] adding volume "usr-share-ca-certificates" for component "kube-controller-manager"
I1020 12:24:32.219791     190 manifests.go:180] [control-plane] wrote static Pod manifest for component "kube-controller-manager" to "/etc/kubernetes/manifests/kube-controller-manager.yaml"
I1020 12:24:32.219799     190 manifests.go:125] [control-plane] getting StaticPodSpecs
[control-plane] Creating static Pod manifest for "kube-scheduler"
I1020 12:24:32.219886     190 manifests.go:151] [control-plane] adding volume "kubeconfig" for component "kube-scheduler"
I1020 12:24:32.220346     190 manifests.go:180] [control-plane] wrote static Pod manifest for component "kube-scheduler" to "/etc/kubernetes/manifests/kube-scheduler.yaml"
I1020 12:24:32.220356     190 kubelet.go:69] Stopping the kubelet
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/instance-config.yaml"
[patches] Applied patch of type "application/strategic-merge-patch+json" to target "kubeletconfiguration"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
I1020 12:24:32.343373     190 loader.go:402] Config loaded from file:  /etc/kubernetes/admin.conf
I1020 12:24:32.343740     190 envvar.go:172] "Feature gate default state" feature="ClientsAllowCBOR" enabled=false
I1020 12:24:32.343755     190 envvar.go:172] "Feature gate default state" feature="ClientsPreferCBOR" enabled=false
I1020 12:24:32.343760     190 envvar.go:172] "Feature gate default state" feature="InOrderInformers" enabled=true
I1020 12:24:32.343765     190 envvar.go:172] "Feature gate default state" feature="InformerResourceVersion" enabled=false
I1020 12:24:32.343769     190 envvar.go:172] "Feature gate default state" feature="WatchListClient" enabled=false
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests"
[kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s
[kubelet-check] The kubelet is healthy after 501.319429ms
[control-plane-check] Waiting for healthy control plane components. This can take up to 4m0s
[control-plane-check] Checking kube-apiserver at https://172.240.1.2:6443/livez
[control-plane-check] Checking kube-controller-manager at https://127.0.0.1:10257/healthz
[control-plane-check] Checking kube-scheduler at https://127.0.0.1:10259/livez
I1020 12:24:32.850950     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:24:33.350771     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:24:33.850753     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:24:34.350723     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:24:34.850666     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
[control-plane-check] kube-controller-manager is healthy after 3.017254923s
I1020 12:24:45.354772     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=10003
I1020 12:24:55.363915     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=10008
I1020 12:24:55.724073     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=359
I1020 12:24:56.724989     190 with_retry.go:234] "Got a Retry-After response" delay="1s" attempt=1 url="https://kind-control-plane:6443/livez?timeout=10s"
I1020 12:24:56.725509     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:24:56.725996     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
[control-plane-check] kube-scheduler is healthy after 23.894002627s
I1020 12:24:56.851123     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:24:57.351319     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:24:57.850859     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:24:58.351281     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:24:58.851600     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:24:59.350946     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:24:59.850854     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:00.351334     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:00.851479     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:01.351286     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:01.851489     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:02.350828     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:02.850731     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:03.350989     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:03.851491     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:04.351404     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:04.851916     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:05.351479     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:05.851519     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:06.350904     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:06.851169     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:07.351459     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:07.852198     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:08.350967     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:08.851223     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:09.351376     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:09.853425     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=1
I1020 12:25:20.351288     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=10000
I1020 12:25:30.361077     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=10009
I1020 12:25:31.676003     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=1314
I1020 12:25:32.676821     190 with_retry.go:234] "Got a Retry-After response" delay="1s" attempt=1 url="https://kind-control-plane:6443/livez?timeout=10s"
I1020 12:25:32.677369     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:32.677802     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:32.852554     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:33.351048     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:33.851405     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:34.350899     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:34.851315     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:35.351353     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:35.850924     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:36.351353     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:36.851361     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:37.352253     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:37.851840     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:38.351213     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:38.851390     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:39.351587     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:39.851041     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:40.351373     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:40.851390     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:41.353051     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=2
I1020 12:25:41.851359     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:42.351449     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:42.851414     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:43.352842     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=1
I1020 12:25:43.852174     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:44.351383     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:44.851319     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:45.351375     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:45.850899     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:46.351228     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:46.850888     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:47.350890     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:47.851146     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:48.351364     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:48.851290     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:49.351393     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:49.850907     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:50.351275     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:50.851758     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:25:51.350943     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:01.859761     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=10009
I1020 12:26:11.868803     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=10008
I1020 12:26:12.549076     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=680
I1020 12:26:13.549810     190 with_retry.go:234] "Got a Retry-After response" delay="1s" attempt=1 url="https://kind-control-plane:6443/livez?timeout=10s"
I1020 12:26:13.550394     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:13.550837     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:13.850854     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:14.351262     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:14.851320     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:15.351564     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:15.850974     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:16.352237     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:16.851361     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:17.351293     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:17.851010     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:18.356461     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:18.851327     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:19.351575     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:19.850849     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:20.351386     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:20.850930     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:21.351404     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:21.850776     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:22.351085     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:22.851477     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:23.351226     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:23.851399     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:24.352918     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=2
I1020 12:26:24.851009     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:25.351383     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:25.851355     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:26.351487     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:26.851006     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:27.354012     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=3
I1020 12:26:27.850990     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:28.350895     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:28.851302     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:29.353388     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:29.851355     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:30.351362     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:30.851340     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:31.351683     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:31.851018     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:32.351414     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:32.851632     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=1
I1020 12:26:33.350988     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:33.851267     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:34.351413     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:34.851513     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:35.350899     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:35.851186     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:36.351369     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:36.851347     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:37.351284     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:37.851349     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:38.351394     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:38.851335     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:39.351387     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:39.851312     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:40.351368     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:40.851297     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:41.351425     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:41.851338     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:42.351388     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:42.851438     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:43.351396     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:43.851370     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:44.351317     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:44.851322     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:45.351172     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:45.851594     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:46.351227     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:46.851324     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:47.350955     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:47.851211     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:48.351303     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:26:48.855785     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=4
I1020 12:26:59.358891     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=10008
I1020 12:27:09.368834     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=10009
I1020 12:27:10.477943     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=1108
I1020 12:27:11.479848     190 with_retry.go:234] "Got a Retry-After response" delay="1s" attempt=1 url="https://kind-control-plane:6443/livez?timeout=10s"
I1020 12:27:11.482041     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=2
I1020 12:27:11.482737     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:11.850862     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:12.351220     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:12.850865     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:13.351180     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:13.851352     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:14.351394     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:14.851349     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:15.350917     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:15.851327     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:16.362268     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:16.851587     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:17.350921     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:17.851299     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:18.351633     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:18.850779     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:19.351411     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:19.851106     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:20.351524     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:20.851343     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:21.351404     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:21.850965     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:22.354144     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=3
I1020 12:27:22.851362     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:23.351362     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:23.851335     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:24.351426     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:24.851320     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:25.351377     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:25.851348     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:26.350815     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:26.851120     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:27.351416     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:27.850829     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:28.351205     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:28.851605     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:29.351221     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:29.850927     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:30.354266     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:30.851334     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:31.351619     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:31.850914     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:32.351102     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:32.851366     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:33.351395     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:33.851372     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:34.350884     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:34.851329     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:35.351372     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:35.851343     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:36.351528     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:36.850873     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:37.351344     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:37.851360     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:38.351497     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:38.850895     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:39.351281     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:39.850976     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:40.350764     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:40.851011     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:41.351450     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:41.850853     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:42.350792     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:42.850957     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:43.351316     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:43.851359     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:44.351061     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:44.851360     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:45.351302     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:45.851415     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:46.351613     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:46.850905     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:47.351150     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:47.851647     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:48.351248     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:48.851437     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:49.350978     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:49.851256     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:50.351673     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:50.850953     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:51.351364     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:51.850931     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:52.351161     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:52.851337     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:53.351384     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:53.850987     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:54.351386     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:54.851242     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:55.350980     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:55.851379     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:56.351000     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:56.851474     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:57.351309     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:57.853111     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=2
I1020 12:27:58.351351     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:58.851494     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:59.351009     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:27:59.851296     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:28:00.351360     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:28:00.852048     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:28:01.351440     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:28:01.850872     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:28:12.359835     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=10008
I1020 12:28:22.368890     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=10008
I1020 12:28:22.789013     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=419
I1020 12:28:23.790879     190 with_retry.go:234] "Got a Retry-After response" delay="1s" attempt=1 url="https://kind-control-plane:6443/livez?timeout=10s"
I1020 12:28:23.791556     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:28:23.792060     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:28:23.851008     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:28:24.351459     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:28:24.851365     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:28:25.351704     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:28:25.851013     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:28:26.351765     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:28:26.851142     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:28:27.351384     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:28:27.851300     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:28:28.351519     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:28:28.850815     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:28:29.351113     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:28:29.851405     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:28:30.351407     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:28:30.851442     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:28:31.351580     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:28:31.850943     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
I1020 12:28:32.351245     190 round_trippers.go:632] "Response" verb="GET" url="https://kind-control-plane:6443/livez?timeout=10s" status="" milliseconds=0
[control-plane-check] kube-apiserver is not healthy after 4m0.000795045s

A control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
	- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
	Once you have found the failing container, you can inspect its logs with:
	- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock logs CONTAINERID'

error: error execution phase wait-control-plane: failed while waiting for the control plane to start: kube-apiserver check failed at https://172.240.1.2:6443/livez: client rate limiter Wait returned an error: rate: Wait(n=1) would exceed context deadline
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).Run.func1
	k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow/runner.go:262
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).visitAll
	k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow/runner.go:450
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).Run
	k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow/runner.go:234
k8s.io/kubernetes/cmd/kubeadm/app/cmd.newCmdInit.func1
	k8s.io/kubernetes/cmd/kubeadm/app/cmd/init.go:133
github.com/spf13/cobra.(*Command).execute
	github.com/spf13/cobra@v1.9.1/command.go:1015
github.com/spf13/cobra.(*Command).ExecuteC
	github.com/spf13/cobra@v1.9.1/command.go:1148
github.com/spf13/cobra.(*Command).Execute
	github.com/spf13/cobra@v1.9.1/command.go:1071
k8s.io/kubernetes/cmd/kubeadm/app.Run
	k8s.io/kubernetes/cmd/kubeadm/app/kubeadm.go:48
main.main
	k8s.io/kubernetes/cmd/kubeadm/kubeadm.go:25
runtime.main
	runtime/proc.go:283
runtime.goexit
	runtime/asm_amd64.s:1700

[stmcginnis]

Looks like a bunch of containerd errors starting pods. Try again after the AWS outage is fully resolved and see if that's it.

[ckf-ateme]

Thank you, I'll wait for now and work on something else. But I’m not sure it will resolve anything as I had the same problem last week.

[stmcginnis]

Ah, OK. Still need the runtime info added. If you want to grab that, it's possible that has some other clues.

Also as a sanity check, make sure you can just run regular containers directly (e.g. docker run --rm hello-world).

[ckf-ateme]

Sorry I though I was supposed to only give the docker version, here is the full output:

Client: Docker Engine - Community
 Version:    28.5.1
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.29.1
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.40.0
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 2
  Running: 2
  Paused: 0
  Stopped: 0
 Images: 13
 Server Version: 28.5.1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 CDI spec directories:
  /etc/cdi
  /var/run/cdi
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: b98a3aace656320842a23f4a392a33f46af97866
 runc version: v1.3.0-0-g4ca628d1
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.8.0-85-generic
 Operating System: Ubuntu 22.04.5 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 15.37GiB
 Name: RS-LPC205FR.corp.ateme.com
 ID: 1ef165db-a1d7-4fda-8118-3bc74e805c02
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  ::1/128
  127.0.0.0/8
 Live Restore Enabled: false
 Default Address Pools:
   Base: 172.17.0.0/16, Size: 24

EDIT:
Forgot to mention that the sanity check passed.

[ckf-ateme]

Another useful information. I have done the inotify fix that used to happen on ubuntu machines: https://kind.sigs.k8s.io/docs/user/known-issues/#pod-errors-due-to-too-many-open-files

[BenTheElder]

I took a quick look at the log zip ... etcd is failing to start but the etcd log files are empty.

containerd/containerd#8981 / containerd/containerd#9309 come to mind, possibly the host was otherwise under high I/O load?

[BenTheElder]

I assume it does, but does the absolute minimum config of just kind create cluster have the same issue? (no networking config)

[ckf-ateme]

The minimum config with kind create cluster does not work sadly.
I’ve just tried some benchmarks regarding the I/O load. However I only note few spikes during the first few seconds after the kind create command. (sorry about the poor annotation on the image, I’m not quite used to gimp)
In the image below create correspond to a kind create cluster command, and delete to a C-c followed by a kind delete cluster

[jahaniam]

Had very similar issue.

If you are on Mac M series, make sure your docker images is ARM64 and not AMD64. If you are forcing DOCKER_DEFAULT_PLATFORM to AMD64 remove it.