Skip to content

upgrade-cluster fails at control-plane : Remove binding to anonymous user (apiserver 127.0.0.1:6443 refused) at first run #13410

Open
#13417
Open
upgrade-cluster fails at control-plane : Remove binding to anonymous user (apiserver 127.0.0.1:6443 refused) at first run#13410
#13417
Assignees
suii2210
Labels
Ubuntu 22kind/bugCategorizes issue or PR as related to a bug.

Description

@YuryHrytsuk
YuryHrytsuk
opened on Aug 6, 2026

What happened?

Almost every upgrade-cluster fail at 1st run with

TASK [kubernetes/control-plane : Remove binding to anonymous user] ***********************************************************************************************
fatal: [prodnode1]: FAILED! => {"changed": true, "cmd": ["/usr/local/bin/kubectl", "--kubeconfig", "/etc/kubernetes/admin.conf", "-n", "kube-public", "delete", "rolebinding", "kubeadm:bootstrap-signer-clusterinfo", "--ignore-not-found"], "delta": "0:00:00.049898", "end": "2026-08-06 12:04:59.530895", "msg": "non-zero return code", "rc": 1, "start": "2026-08-06 12:04:59.480997", "stderr": "The connection to the server 127.0.0.1:6443 was refused - did you specify the right host or port?", "stderr_lines": ["The connection to the server 127.0.0.1:6443 was refused - did you specify the right host or port?"], "stdout": "", "stdout_lines": []

What did you expect to happen?

upgrade to run smoothly. Running upgrade-cluster 2nd time always succeeds.

How can we reproduce it (as minimally and precisely as possible)?

  1. Set remove_anonymous_access: true to true
  2. Create cluster with v2.28.1
  3. Upgrade cluster to v2.29.1 <--- should have the same failure

OS

Ubuntu 22

Version of Ansible

ansible [core 2.17.14]
config file = /etc/ansible/ansible.cfg
configured module search path = ['/home/hrytsuk/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /home/hrytsuk/fork-projects/osparc-infra/.venv/lib/python3.11/site-packages/ansible
ansible collection location = /home/hrytsuk/.ansible/collections:/usr/share/ansible/collections
executable location = ../.venv/bin/ansible
python version = 3.11.4 (main, Sep 4 2024, 08:48:30) [GCC 11.4.0] (/home/hrytsuk/fork-projects/osparc-infra/.venv/bin/python3)
jinja version = 3.1.4
libyaml = True

Version of Python

3.11.4

Version of Kubespray (commit)

v1.29.1 (and many previous)

Network plugin used

calico

Full inventory with variables

[all:vars]
ansible_user=ubuntu
ansible_become_pass=<redacted>

ansible_ssh_private_key_file="{{ lookup('env','ANSIBLE_SSH_PRIVATE_KEY_FILE', default='./sshkey.pem') }}"
ansible_ssh_common_args='-o StrictHostKeyChecking=no'

deployment_stage=prod
deployment_type=inhouse

# https://docs.tigera.io/calico/3.30/operations/install-apiserver
calico_apiserver_enabled=true

auto_renew_certificates=true

# keep in sync with value used in topolvm chart
topolvm_vg_name=topolvm

# CI variables

docker_image_tag="{{ lookup('env','DOCKER_IMAGE_TAG') }}"
config_branch="{{ lookup('env', 'CONFIG_BRANCH', default='configs') }}"
ops_env_branch="{{ lookup('env', 'OPS_ENV_BRANCH', default='main') }}"
simcore_branch="{{ lookup('env', 'SIMCORE_BRANCH', default='master') }}"
config_file="{{ lookup('env', 'CONFIG_FILE') }}"
ci_project_dir="{{ lookup('env', 'CI_PROJECT_DIR', default='.') }}"
ci_project_namespace="{{ lookup('env', 'CI_PROJECT_NAMESPACE', default='oSparc') }}"
depl_config_dir="{{ lookup('env', 'DEPL_CONFIG_DIR') }}"
shared_depl_config_dir="{{ lookup('env', 'SHARED_DEPL_CONFIG_DIR') }}"


[all]
prodnode1 ansible_host=<redacted> ip=172.16.8.137 node_labels='{"node-role.kubernetes.io/ops": ""}' topolvm_device='/dev/nvme1n1'

[kube_control_plane]
prodnode1

[etcd]
prodnode1

[kube_node]
prodnode1

[calico_rr]

[k8s_cluster:children]
kube_control_plane
kube_node
calico_rr

hardening vars

# These are native kubespray variables — kubespray applies them
# idempotently when running cluster.yml or upgrade-cluster.yml.
#
# Docs: https://github.com/kubernetes-sigs/kubespray/blob/master/docs/operations/hardening.md

# Disable anonymous authentication — all requests must present valid credentials
remove_anonymous_access: true

# Audit logging — record API requests for security review
kubernetes_audit: true
audit_log_path: "/var/log/kube-apiserver-log.json"
audit_log_maxage: 30
audit_log_maxbackups: 10
audit_log_maxsize: 100

# https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/
kube_apiserver_admission_control_config_file: true # enable Admission Controller in Kubernetes API-Server
kube_apiserver_enable_admission_plugins:
  - PodSecurity # https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#podsecurity
# https://kubernetes.io/docs/tasks/configure-pod-container/enforce-standards-admission-controller/
kube_pod_security_use_default: true
kube_pod_security_default_enforce: restricted
kube_pod_security_exemptions_namespaces:
  - kube-system
  - calico-apiserver # installed by kubespray and is not configurable

# kubelet
kubelet_authorization_mode_webhook: true # kubespray default
kubelet_authentication_token_webhook: true # kubespray default
kube_read_only_port: 0 # kubespray default
kubelet_rotate_certificates: true # kubespray default
kubelet_rotate_server_certificates: true # https://kubernetes.io/docs/reference/access-authn-authz/kubelet-tls-bootstrapping/
kubelet_csr_approver_values: # https://github.com/kubernetes-sigs/kubespray/pull/9877
  bypassDnsResolution: true # related to dns of node's hostname. unblocks `kubelet_rotate_server_certificates` setting
kubelet_protect_kernel_defaults: true # kubespray default
kubelet_event_record_qps: 50 # kubespray default is good enough
kubelet_streaming_connection_idle_timeout: "5m"
kubelet_make_iptables_util_chains: true # kubernetes default
kubelet_feature_gates: ["RotateKubeletServerCertificate=true"] # kubernetes default
kubelet_seccomp_default: true # enable seccomp by default for all pods
kubelet_systemd_hardening: true # stricten which from which IPs kubelet can be accessed

# etcd hardening
etcd_deployment_type: host # run in separate network (pod cannot access etcd directly)
etcd_secure_client: true # kubespray default
etcd_peer_client_auth: true # kubespray default
kube_encrypt_secret_data: true # encryption at rest for secrets
kube_encryption_resources: [secrets]
kube_encryption_algorithm: "secretbox"

# https://github.com/kubernetes-sigs/kubespray/pull/11296
ubuntu_kernel_unattended_upgrades_disabled: true

Anything else we need to know

I presume the problem is that restart kubectl restrats kube-apiserver and by the time kubernetes/control-plane : Remove binding to anonymous user task is executed kube-apiserver is not running yet

Command used to invoke ansible

ansible-playbook -i k8s_hosts.ini -e @k8s_hardening_vars.yml upgrade-cluster.yml --become --verbose

Output of ansible run

RUNNING HANDLER [kubernetes/node : Kubelet | restart kubelet] ****************************************************************************************************
changed: [prodnode1] => {"changed": true, "name": "kubelet", "state": "started", "status": {"ActiveEnterTimestamp": "Thu 2026-08-06 11:57:43 UTC", "ActiveEnterTimestampMonotonic": "8132197046498", "ActiveExitTimestamp": "Thu 2026-08-06 11:57:43 UTC", "ActiveExitTimestampMonotonic": "8132196997090", "ActiveState": "active", "After": "system.slice sysinit.target basic.target containerd.service systemd-journald.socket", "AllowIsolate": "no", "AssertResult": "yes", "AssertTimestamp": "Thu 2026-08-06 11:57:43 UTC", "AssertTimestampMonotonic": "8132197044744", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "yes", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "14028298000", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanFreeze": "yes", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read cap_perfmon cap_bpf cap_checkpoint_restore", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Thu 2026-08-06 11:57:43 UTC", "ConditionTimestampMonotonic": "8132197044744", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/kubelet.service", "ControlPID": "0", "CoredumpFilter": "0x33", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "Description": "Kubernetes Kubelet Server", "DevicePolicy": "auto", "Documentation": "https://github.com/GoogleCloudPlatform/kubernetes", "DynamicUser": "no", "EffectiveCPUs": "0-31", "EffectiveMemoryNodes": "0", "EnvironmentFiles": "/etc/kubernetes/kubelet.env (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestamp": "n/a", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "636728", "ExecMainStartTimestamp": "Thu 2026-08-06 11:57:43 UTC", "ExecMainStartTimestampMonotonic": "8132197046244", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/local/bin/kubelet ; argv[]=/usr/local/bin/kubelet $KUBE_LOGTOSTDERR $KUBE_LOG_LEVEL $KUBELET_API_SERVER $KUBELET_ADDRESS $KUBELET_PORT $KUBELET_HOSTNAME $KUBELET_ARGS $DOCKER_SOCKET $KUBELET_NETWORK_PLUGIN $KUBELET_VOLUME_PLUGIN $KUBELET_CLOUDPROVIDER ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/local/bin/kubelet ; argv[]=/usr/local/bin/kubelet $KUBE_LOGTOSTDERR $KUBE_LOG_LEVEL $KUBELET_API_SERVER $KUBELET_ADDRESS $KUBELET_PORT $KUBELET_HOSTNAME $KUBELET_ARGS $DOCKER_SOCKET $KUBELET_NETWORK_PLUGIN $KUBELET_VOLUME_PLUGIN $KUBELET_CLOUDPROVIDER ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/etc/systemd/system/kubelet.service", "FreezerState": "running", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "2", "IOSchedulingPriority": "4", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPAddressAllow": "127.0.0.0/8 ::1/128 169.254.0.0/16 fe80::/64 10.233.64.0/18 172.16.8.137/32", "IPAddressDeny": "0.0.0.0/0 ::/0", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressPackets": "[no data]", "Id": "kubelet.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestamp": "Thu 2026-08-06 11:57:43 UTC", "InactiveEnterTimestampMonotonic": "8132197044296", "InactiveExitTimestamp": "Thu 2026-08-06 11:57:43 UTC", "InactiveExitTimestampMonotonic": "8132197046498", "InvocationID": "dda201d535c34332992c20327d107041", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "524288", "LimitNOFILESoft": "1024", "LimitNPROC": "514458", "LimitNPROCSoft": "514458", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "514458", "LimitSIGPENDINGSoft": "514458", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "636728", "ManagedOOMMemoryPressure": "auto", "ManagedOOMMemoryPressureLimit": "0", "ManagedOOMPreference": "none", "ManagedOOMSwap": "auto", "MemoryAccounting": "yes", "MemoryAvailable": "infinity", "MemoryCurrent": "52076544", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAPolicy": "n/a", "Names": "kubelet.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMPolicy": "stop", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "OnSuccessJobMode": "fail", "Perpetual": "no", "PrivateDevices": "no", "PrivateIPC": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProcSubset": "all", "ProtectClock": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectProc": "default", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target system.slice", "Restart": "always", "RestartKillSignal": "15", "RestartUSec": "10s", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Thu 2026-08-06 11:57:43 UTC", "StateChangeTimestampMonotonic": "8132197046498", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "2147483646", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "39", "TasksMax": "154337", "TimeoutAbortUSec": "1min 30s", "TimeoutCleanUSec": "infinity", "TimeoutStartFailureMode": "terminate", "TimeoutStartUSec": "1min 30s", "TimeoutStopFailureMode": "terminate", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "Wants": "containerd.service", "WatchdogSignal": "6", "WatchdogTimestamp": "n/a", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}}
Donnerstag 06 August 2026 14:04:47 +0200 (0:00:00.354) 0:04:36.960 *****

TASK [kubernetes/node : Enable kubelet] **************************************************************************************************************************
ok: [prodnode1] => {"changed": false, "enabled": true, "name": "kubelet", "state": "started", "status": {"ActiveEnterTimestamp": "Thu 2026-08-06 12:04:47 UTC", "ActiveEnterTimestampMonotonic": "8132621331955", "ActiveExitTimestamp": "Thu 2026-08-06 12:04:47 UTC", "ActiveExitTimestampMonotonic": "8132621268341", "ActiveState": "active", "After": "system.slice sysinit.target basic.target containerd.service systemd-journald.socket", "AllowIsolate": "no", "AssertResult": "yes", "AssertTimestamp": "Thu 2026-08-06 12:04:47 UTC", "AssertTimestampMonotonic": "8132621330684", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "yes", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "473124000", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanFreeze": "yes", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read cap_perfmon cap_bpf cap_checkpoint_restore", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Thu 2026-08-06 12:04:47 UTC", "ConditionTimestampMonotonic": "8132621330682", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/kubelet.service", "ControlPID": "0", "CoredumpFilter": "0x33", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "Description": "Kubernetes Kubelet Server", "DevicePolicy": "auto", "Documentation": "https://github.com/GoogleCloudPlatform/kubernetes", "DynamicUser": "no", "EffectiveCPUs": "0-31", "EffectiveMemoryNodes": "0", "EnvironmentFiles": "/etc/kubernetes/kubelet.env (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestamp": "n/a", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "650788", "ExecMainStartTimestamp": "Thu 2026-08-06 12:04:47 UTC", "ExecMainStartTimestampMonotonic": "8132621331770", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/local/bin/kubelet ; argv[]=/usr/local/bin/kubelet $KUBE_LOGTOSTDERR $KUBE_LOG_LEVEL $KUBELET_API_SERVER $KUBELET_ADDRESS $KUBELET_PORT $KUBELET_HOSTNAME $KUBELET_ARGS $DOCKER_SOCKET $KUBELET_NETWORK_PLUGIN $KUBELET_VOLUME_PLUGIN $KUBELET_CLOUDPROVIDER ; ignore_errors=no ; start_time=[Thu 2026-08-06 12:04:47 UTC] ; stop_time=[n/a] ; pid=650788 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/local/bin/kubelet ; argv[]=/usr/local/bin/kubelet $KUBE_LOGTOSTDERR $KUBE_LOG_LEVEL $KUBELET_API_SERVER $KUBELET_ADDRESS $KUBELET_PORT $KUBELET_HOSTNAME $KUBELET_ARGS $DOCKER_SOCKET $KUBELET_NETWORK_PLUGIN $KUBELET_VOLUME_PLUGIN $KUBELET_CLOUDPROVIDER ; flags= ; start_time=[Thu 2026-08-06 12:04:47 UTC] ; stop_time=[n/a] ; pid=650788 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/etc/systemd/system/kubelet.service", "FreezerState": "running", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "2", "IOSchedulingPriority": "4", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPAddressAllow": "127.0.0.0/8 ::1/128 169.254.0.0/16 fe80::/64 10.233.64.0/18 172.16.8.137/32", "IPAddressDeny": "0.0.0.0/0 ::/0", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressPackets": "[no data]", "Id": "kubelet.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestamp": "Thu 2026-08-06 12:04:47 UTC", "InactiveEnterTimestampMonotonic": "8132621330193", "InactiveExitTimestamp": "Thu 2026-08-06 12:04:47 UTC", "InactiveExitTimestampMonotonic": "8132621331955", "InvocationID": "b38938abf91b438b9e4ae06938669356", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "524288", "LimitNOFILESoft": "1024", "LimitNPROC": "514458", "LimitNPROCSoft": "514458", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "514458", "LimitSIGPENDINGSoft": "514458", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "650788", "ManagedOOMMemoryPressure": "auto", "ManagedOOMMemoryPressureLimit": "0", "ManagedOOMPreference": "none", "ManagedOOMSwap": "auto", "MemoryAccounting": "yes", "MemoryAvailable": "infinity", "MemoryCurrent": "37888000", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAPolicy": "n/a", "Names": "kubelet.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMPolicy": "stop", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "OnSuccessJobMode": "fail", "Perpetual": "no", "PrivateDevices": "no", "PrivateIPC": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProcSubset": "all", "ProtectClock": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectProc": "default", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target system.slice", "Restart": "always", "RestartKillSignal": "15", "RestartUSec": "10s", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Thu 2026-08-06 12:04:47 UTC", "StateChangeTimestampMonotonic": "8132621331955", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "2147483646", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "39", "TasksMax": "154337", "TimeoutAbortUSec": "1min 30s", "TimeoutCleanUSec": "infinity", "TimeoutStartFailureMode": "terminate", "TimeoutStartUSec": "1min 30s", "TimeoutStopFailureMode": "terminate", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "Wants": "containerd.service", "WatchdogSignal": "6", "WatchdogTimestamp": "n/a", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}}
Donnerstag 06 August 2026 14:04:48 +0200 (0:00:00.348) 0:04:37.309 *****
Donnerstag 06 August 2026 14:04:48 +0200 (0:00:00.017) 0:04:37.326 *****
Donnerstag 06 August 2026 14:04:48 +0200 (0:00:00.023) 0:04:37.349 *****
Donnerstag 06 August 2026 14:04:48 +0200 (0:00:00.020) 0:04:37.369 *****
Donnerstag 06 August 2026 14:04:48 +0200 (0:00:00.033) 0:04:37.403 *****

TASK [kubernetes/control-plane : Pre-upgrade | Delete control plane manifests if etcd secrets changed] ***********************************************************
ok: [prodnode1] => (item=kube-apiserver) => {"ansible_loop_var": "item", "changed": false, "item": "kube-apiserver", "path": "/etc/kubernetes/manifests/kube-apiserver.manifest", "state": "absent"}
ok: [prodnode1] => (item=kube-controller-manager) => {"ansible_loop_var": "item", "changed": false, "item": "kube-controller-manager", "path": "/etc/kubernetes/manifests/kube-controller-manager.manifest", "state": "absent"}
ok: [prodnode1] => (item=kube-scheduler) => {"ansible_loop_var": "item", "changed": false, "item": "kube-scheduler", "path": "/etc/kubernetes/manifests/kube-scheduler.manifest", "state": "absent"}
Donnerstag 06 August 2026 14:04:48 +0200 (0:00:00.510) 0:04:37.914 *****
Donnerstag 06 August 2026 14:04:48 +0200 (0:00:00.040) 0:04:37.954 *****
Donnerstag 06 August 2026 14:04:48 +0200 (0:00:00.024) 0:04:37.978 *****
Donnerstag 06 August 2026 14:04:49 +0200 (0:00:00.024) 0:04:38.003 *****
Donnerstag 06 August 2026 14:04:49 +0200 (0:00:00.018) 0:04:38.022 *****

TASK [kubernetes/control-plane : Create kube-scheduler config] ***************************************************************************************************
ok: [prodnode1] => {"changed": false, "checksum": "ada0a8f90054287cb3941895dbdf91667eb9af3f", "dest": "/etc/kubernetes/kubescheduler-config.yaml", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/kubernetes/kubescheduler-config.yaml", "size": 199, "state": "file", "uid": 0}
Donnerstag 06 August 2026 14:04:49 +0200 (0:00:00.421) 0:04:38.443 *****

TASK [kubernetes/control-plane : Check if secret for encrypting data at rest already exist] **********************************************************************
ok: [prodnode1] => {"changed": false, "stat": {"atime": 1786017446.0069668, "block_size": 4096, "blocks": 8, "ctime": 1783583864.368541, "dev": 66306, "device_type": 0, "executable": false, "exists": true, "gid": 999, "gr_name": "kube-cert", "inode": 2621476, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mode": "0640", "mtime": 1783583864.1725366, "nlink": 1, "path": "/etc/kubernetes/ssl/secrets_encryption.yaml", "pw_name": "root", "readable": true, "rgrp": true, "roth": false, "rusr": true, "size": 258, "uid": 0, "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}}
Donnerstag 06 August 2026 14:04:49 +0200 (0:00:00.182) 0:04:38.626 *****

TASK [kubernetes/control-plane : Slurp secrets_encryption file if it exists] *************************************************************************************
ok: [prodnode1] => {"changed": false, "content": "YXBpVmVyc2lvbjogYXBpc2VydmVyLmNvbmZpZy5rOHMuaW8vdjEKa2luZDogRW5jcnlwdGlvbkNvbmZpZ3VyYXRpb24KcmVzb3VyY2VzOgogIC0gcmVzb3VyY2VzOgogICAgLSBzZWNyZXRzCgogICAgcHJvdmlkZXJzOgogICAgLSBzZWNyZXRib3g6CiAgICAgICAga2V5czoKICAgICAgICAtIG5hbWU6IGtleQogICAgICAgICAgc2VjcmV0OiBTVnA1TlhsVGIxQm1Wbm93ZGxreWExWkJjR05IUjFoaWRURTNOblpQY25JPQogICAgLSBpZGVudGl0eToge30K", "encoding": "base64", "source": "/etc/kubernetes/ssl/secrets_encryption.yaml"}
Donnerstag 06 August 2026 14:04:49 +0200 (0:00:00.196) 0:04:38.822 *****

TASK [kubernetes/control-plane : Base 64 Decode slurped secrets_encryption.yaml file] ****************************************************************************
ok: [prodnode1] => {"ansible_facts": {"secret_file_decoded": {"apiVersion": "apiserver.config.k8s.io/v1", "kind": "EncryptionConfiguration", "resources": [{"providers": [{"secretbox": {"keys": [{"name": "key", "secret": "SVp5NXlTb1BmVnowdlkya1ZBcGNHR1hidTE3NnZPcnI="}]}}, {"identity": {}}], "resources": ["secrets"]}]}}, "changed": false}
Donnerstag 06 August 2026 14:04:49 +0200 (0:00:00.029) 0:04:38.852 *****

TASK [kubernetes/control-plane : Extract secret value from secrets_encryption.yaml] ******************************************************************************
ok: [prodnode1] => {"ansible_facts": {"kube_encrypt_token_extracted": "IZy5ySoPfVz0vY2kVApcGGXbu176vOrr"}, "changed": false}
Donnerstag 06 August 2026 14:04:49 +0200 (0:00:00.034) 0:04:38.887 *****

TASK [kubernetes/control-plane : Set kube_encrypt_token across control plane nodes] ******************************************************************************
ok: [prodnode1] => (item=prodnode1) => {"ansible_facts": {"kube_encrypt_token": "IZy5ySoPfVz0vY2kVApcGGXbu176vOrr"}, "ansible_loop_var": "item", "changed": false, "item": "prodnode1"}
Donnerstag 06 August 2026 14:04:49 +0200 (0:00:00.030) 0:04:38.917 *****

TASK [kubernetes/control-plane : Write secrets for encrypting secret data at rest] *******************************************************************************
ok: [prodnode1] => {"changed": false, "checksum": "40c2b6c4c34bad81a2e0adfb809a7738164a3aa6", "dest": "/etc/kubernetes/ssl/secrets_encryption.yaml", "gid": 999, "group": "kube-cert", "mode": "0640", "owner": "root", "path": "/etc/kubernetes/ssl/secrets_encryption.yaml", "size": 258, "state": "file", "uid": 0}
Donnerstag 06 August 2026 14:04:50 +0200 (0:00:00.417) 0:04:39.335 *****

TASK [kubernetes/control-plane : Install | Copy kubectl binary from download dir] ********************************************************************************
changed: [prodnode1] => {"changed": true, "checksum": "d7dbf0539abe8a6c8b8868566d8c549b3e935c7e", "dest": "/usr/local/bin/kubectl", "gid": 0, "group": "root", "md5sum": "7bb294b845e9b05da04bc48b044c6bfc", "mode": "0755", "owner": "root", "size": 60133560, "src": "/tmp/releases/kubectl-1.33.7-amd64", "state": "file", "uid": 0}
Donnerstag 06 August 2026 14:04:50 +0200 (0:00:00.622) 0:04:39.957 *****

TASK [kubernetes/control-plane : Install kubectl bash completion] ************************************************************************************************
changed: [prodnode1] => {"changed": true, "cmd": "/usr/local/bin/kubectl completion bash >/etc/bash_completion.d/kubectl.sh", "delta": "0:00:00.030782", "end": "2026-08-06 12:04:51.163669", "msg": "", "rc": 0, "start": "2026-08-06 12:04:51.132887", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}
Donnerstag 06 August 2026 14:04:51 +0200 (0:00:00.225) 0:04:40.183 *****

TASK [kubernetes/control-plane : Set kubectl bash completion file permissions] ***********************************************************************************
ok: [prodnode1] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/bash_completion.d/kubectl.sh", "size": 12404, "state": "file", "uid": 0}
Donnerstag 06 August 2026 14:04:51 +0200 (0:00:00.214) 0:04:40.398 *****
Donnerstag 06 August 2026 14:04:51 +0200 (0:00:00.031) 0:04:40.430 *****

TASK [kubernetes/control-plane : Check which kube-control nodes are already members of the cluster] **************************************************************
fatal: [prodnode1]: FAILED! => {"changed": false, "cmd": ["/usr/local/bin/kubectl", "get", "nodes", "--selector=node-role.kubernetes.io/control-plane", "-o", "json"], "delta": "0:00:00.053111", "end": "2026-08-06 12:04:51.685470", "msg": "non-zero return code", "rc": 1, "start": "2026-08-06 12:04:51.632359", "stderr": "The connection to the server 127.0.0.1:6443 was refused - did you specify the right host or port?", "stderr_lines": ["The connection to the server 127.0.0.1:6443 was refused - did you specify the right host or port?"], "stdout": "{\n "apiVersion": "v1",\n "items": [],\n "kind": "List",\n "metadata": {\n "resourceVersion": ""\n }\n}", "stdout_lines": ["{", " "apiVersion": "v1",", " "items": [],", " "kind": "List",", " "metadata": {", " "resourceVersion": """, " }", "}"]}
...ignoring
Donnerstag 06 August 2026 14:04:51 +0200 (0:00:00.274) 0:04:40.704 *****
Donnerstag 06 August 2026 14:04:51 +0200 (0:00:00.027) 0:04:40.731 *****

TASK [kubernetes/control-plane : Set fact first_kube_control_plane] **********************************************************************************************
ok: [prodnode1] => {"ansible_facts": {"first_kube_control_plane": "prodnode1"}, "changed": false}
Donnerstag 06 August 2026 14:04:51 +0200 (0:00:00.029) 0:04:40.760 *****
Donnerstag 06 August 2026 14:04:51 +0200 (0:00:00.029) 0:04:40.790 *****

TASK [kubernetes/control-plane : Kubeadm | Check if kubeadm has already run] *************************************************************************************
ok: [prodnode1] => {"changed": false, "stat": {"atime": 1786017459.2912545, "block_size": 4096, "blocks": 8, "ctime": 1786017459.2992547, "dev": 66306, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 3802924, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mode": "0644", "mtime": 1786017459.2992547, "nlink": 1, "path": "/var/lib/kubelet/config.yaml", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 1179, "uid": 0, "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}}
Donnerstag 06 August 2026 14:04:52 +0200 (0:00:00.207) 0:04:40.997 *****

TASK [kubernetes/control-plane : Backup old certs and keys] ******************************************************************************************************
ok: [prodnode1] => (item=apiserver.crt) => {"ansible_loop_var": "item", "changed": false, "checksum": "bb918bbdba0734b144aa319287c901e0f9c8d4d4", "dest": "/etc/kubernetes/ssl/apiserver.crt.old", "gid": 0, "group": "root", "item": "apiserver.crt", "md5sum": "f62ebe290d96f6324c47e53fc7bedee7", "mode": "0644", "owner": "root", "size": 1383, "src": "/etc/kubernetes/ssl/apiserver.crt", "state": "file", "uid": 0}
ok: [prodnode1] => (item=apiserver.key) => {"ansible_loop_var": "item", "changed": false, "checksum": "ab6cb8f962dc9952e94e513914612198aeb84557", "dest": "/etc/kubernetes/ssl/apiserver.key.old", "gid": 0, "group": "root", "item": "apiserver.key", "md5sum": "dc9dc2dcdd23c0cc1326fc23ed6839e1", "mode": "0600", "owner": "root", "size": 1675, "src": "/etc/kubernetes/ssl/apiserver.key", "state": "file", "uid": 0}
ok: [prodnode1] => (item=apiserver-kubelet-client.crt) => {"ansible_loop_var": "item", "changed": false, "checksum": "ed4db4826e5699fdd25ebfda4c182d3d6f8506b1", "dest": "/etc/kubernetes/ssl/apiserver-kubelet-client.crt.old", "gid": 0, "group": "root", "item": "apiserver-kubelet-client.crt", "md5sum": "6eda3d8bb3ae73bab865f73d3c677b77", "mode": "0644", "owner": "root", "size": 1176, "src": "/etc/kubernetes/ssl/apiserver-kubelet-client.crt", "state": "file", "uid": 0}
ok: [prodnode1] => (item=apiserver-kubelet-client.key) => {"ansible_loop_var": "item", "changed": false, "checksum": "c9f91f34fea97ba21399922f0debffd7b761f647", "dest": "/etc/kubernetes/ssl/apiserver-kubelet-client.key.old", "gid": 0, "group": "root", "item": "apiserver-kubelet-client.key", "md5sum": "5cd199d6f2beb81c331fefde8accefcf", "mode": "0600", "owner": "root", "size": 1675, "src": "/etc/kubernetes/ssl/apiserver-kubelet-client.key", "state": "file", "uid": 0}
ok: [prodnode1] => (item=front-proxy-client.crt) => {"ansible_loop_var": "item", "changed": false, "checksum": "5ebf37057f961ad6a1b2bd4d7c124c7758dedeb4", "dest": "/etc/kubernetes/ssl/front-proxy-client.crt.old", "gid": 0, "group": "root", "item": "front-proxy-client.crt", "md5sum": "d8f1e1f58c3ea723e26be285dfafd3b4", "mode": "0644", "owner": "root", "size": 1119, "src": "/etc/kubernetes/ssl/front-proxy-client.crt", "state": "file", "uid": 0}
ok: [prodnode1] => (item=front-proxy-client.key) => {"ansible_loop_var": "item", "changed": false, "checksum": "cf99afce265c98bf07daa49ac5df25f7d03a2cc0", "dest": "/etc/kubernetes/ssl/front-proxy-client.key.old", "gid": 0, "group": "root", "item": "front-proxy-client.key", "md5sum": "6fef97b2f3cd7b16f1bd2481ffcd0c56", "mode": "0600", "owner": "root", "size": 1679, "src": "/etc/kubernetes/ssl/front-proxy-client.key", "state": "file", "uid": 0}
Donnerstag 06 August 2026 14:04:53 +0200 (0:00:01.558) 0:04:42.555 *****

TASK [kubernetes/control-plane : Backup old confs] ***************************************************************************************************************
changed: [prodnode1] => (item=admin.conf) => {"ansible_loop_var": "item", "changed": true, "checksum": "a207e641d79f5b58f66479f61f7771d8f405127f", "dest": "/etc/kubernetes/admin.conf.old", "gid": 0, "group": "root", "item": "admin.conf", "md5sum": "1f4a1f7a32d8589719c658de67971ee7", "mode": "0600", "owner": "root", "size": 5661, "src": "/etc/kubernetes/admin.conf", "state": "file", "uid": 0}
ok: [prodnode1] => (item=controller-manager.conf) => {"ansible_loop_var": "item", "changed": false, "checksum": "0dcc498a582fea8a41f7820cc83c55b18e00068d", "dest": "/etc/kubernetes/controller-manager.conf.old", "gid": 0, "group": "root", "item": "controller-manager.conf", "md5sum": "bb603e08542f2d2e745dd20bc1c90c46", "mode": "0600", "owner": "root", "size": 5689, "src": "/etc/kubernetes/controller-manager.conf", "state": "file", "uid": 0}
ok: [prodnode1] => (item=kubelet.conf) => {"ansible_loop_var": "item", "changed": false, "checksum": "4a8076bf070b5fcae21b4bf06b98b4d352b5e9bd", "dest": "/etc/kubernetes/kubelet.conf.old", "gid": 0, "group": "root", "item": "kubelet.conf", "md5sum": "4b27df7b1071ee4326bde54eafc45ac0", "mode": "0600", "owner": "root", "size": 1989, "src": "/etc/kubernetes/kubelet.conf", "state": "file", "uid": 0}
ok: [prodnode1] => (item=scheduler.conf) => {"ansible_loop_var": "item", "changed": false, "checksum": "d6229a95a8c03c5db9c3ef2eb2907c6905a0df8d", "dest": "/etc/kubernetes/scheduler.conf.old", "gid": 0, "group": "root", "item": "scheduler.conf", "md5sum": "ebcf03e10137c3b15ce41b5a3051ee75", "mode": "0600", "owner": "root", "size": 5637, "src": "/etc/kubernetes/scheduler.conf", "state": "file", "uid": 0}
Donnerstag 06 August 2026 14:04:54 +0200 (0:00:00.985) 0:04:43.541 *****

TASK [kubernetes/control-plane : Kubeadm | aggregate all SANs] ***************************************************************************************************
ok: [prodnode1] => {"ansible_facts": {"apiserver_sans": ["kubernetes", "kubernetes.default", "kubernetes.default.svc", "kubernetes.default.svc.cluster.local", "10.233.0.1", "localhost", "127.0.0.1", "::1", "prodnode1", "lb-apiserver.kubernetes.local", "172.16.8.137"]}, "changed": false}
Donnerstag 06 August 2026 14:04:54 +0200 (0:00:00.073) 0:04:43.614 *****

TASK [kubernetes/control-plane : Create audit-policy directory] **************************************************************************************************
ok: [prodnode1] => {"changed": false, "gid": 0, "group": "root", "mode": "0640", "owner": "root", "path": "/etc/kubernetes/audit-policy", "size": 4096, "state": "directory", "uid": 0}
Donnerstag 06 August 2026 14:04:54 +0200 (0:00:00.204) 0:04:43.819 *****

TASK [kubernetes/control-plane : Write api audit policy yaml] ****************************************************************************************************
ok: [prodnode1] => {"changed": false, "checksum": "ec9c4172b75a050c4ba2782c8965de1176f8d9f5", "dest": "/etc/kubernetes/audit-policy/apiserver-audit-policy.yaml", "gid": 0, "group": "root", "mode": "0640", "owner": "root", "path": "/etc/kubernetes/audit-policy/apiserver-audit-policy.yaml", "size": 3751, "state": "file", "uid": 0}
Donnerstag 06 August 2026 14:04:55 +0200 (0:00:00.473) 0:04:44.293 *****
Donnerstag 06 August 2026 14:04:55 +0200 (0:00:00.026) 0:04:44.319 *****
Donnerstag 06 August 2026 14:04:55 +0200 (0:00:00.016) 0:04:44.335 *****
Donnerstag 06 August 2026 14:04:55 +0200 (0:00:00.020) 0:04:44.355 *****
Donnerstag 06 August 2026 14:04:55 +0200 (0:00:00.021) 0:04:44.377 *****

TASK [kubernetes/control-plane : Kubeadm | Create kubeadm config] ************************************************************************************************
changed: [prodnode1] => {"changed": true, "checksum": "7f20117ce5928c3a5debee5eebba21ffac676d81", "dest": "/etc/kubernetes/kubeadm-config.yaml", "gid": 0, "group": "root", "md5sum": "d396713da76e78d028a364c99adba802", "mode": "0640", "owner": "root", "size": 5796, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1786017895.4207823-1131292-85471174803664/.source.yaml", "state": "file", "uid": 0}
Donnerstag 06 August 2026 14:04:55 +0200 (0:00:00.596) 0:04:44.974 *****

TASK [kubernetes/control-plane : Kubeadm | Create directory to store admission control configurations] ***********************************************************
ok: [prodnode1] => {"changed": false, "gid": 0, "group": "root", "mode": "0640", "owner": "root", "path": "/etc/kubernetes/admission-controls", "size": 4096, "state": "directory", "uid": 0}
Donnerstag 06 August 2026 14:04:56 +0200 (0:00:00.183) 0:04:45.157 *****

TASK [kubernetes/control-plane : Kubeadm | Push admission control config file] ***********************************************************************************
ok: [prodnode1] => {"changed": false, "checksum": "0f1b84f5b86e5b0e9417f476b277aceec228e3be", "dest": "/etc/kubernetes/admission-controls/admission-controls.yaml", "gid": 0, "group": "root", "mode": "0640", "owner": "root", "path": "/etc/kubernetes/admission-controls/admission-controls.yaml", "size": 138, "state": "file", "uid": 0}
Donnerstag 06 August 2026 14:04:56 +0200 (0:00:00.463) 0:04:45.620 *****

TASK [kubernetes/control-plane : Kubeadm | Push admission control config files] **********************************************************************************
changed: [prodnode1] => (item=PodSecurity) => {"ansible_loop_var": "item", "changed": true, "checksum": "afe87141feffaa5a4c8bdd5f188fa75683b66fa7", "dest": "/etc/kubernetes/admission-controls/podsecurity.yaml", "gid": 0, "group": "root", "item": "PodSecurity", "md5sum": "735e148115d82f6ab790c09439cafe55", "mode": "0640", "owner": "root", "size": 335, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1786017896.6655607-1131309-21613676079662/.source.yaml", "state": "file", "uid": 0}
Donnerstag 06 August 2026 14:04:57 +0200 (0:00:00.519) 0:04:46.140 *****

TASK [kubernetes/control-plane : Kubeadm | Check apiserver.crt SAN IPs] ******************************************************************************************
ok: [prodnode1] => (item=10.233.0.1) => {"ansible_loop_var": "item", "changed": false, "cmd": ["openssl", "x509", "-noout", "-in", "/etc/kubernetes/ssl/apiserver.crt", "-checkip", "10.233.0.1"], "delta": "0:00:00.004601", "end": "2026-08-06 12:04:57.339716", "failed_when_result": false, "item": "10.233.0.1", "msg": "", "rc": 0, "start": "2026-08-06 12:04:57.335115", "stderr": "", "stderr_lines": [], "stdout": "IP 10.233.0.1 does match certificate", "stdout_lines": ["IP 10.233.0.1 does match certificate"]}
ok: [prodnode1] => (item=127.0.0.1) => {"ansible_loop_var": "item", "changed": false, "cmd": ["openssl", "x509", "-noout", "-in", "/etc/kubernetes/ssl/apiserver.crt", "-checkip", "127.0.0.1"], "delta": "0:00:00.003695", "end": "2026-08-06 12:04:57.539639", "failed_when_result": false, "item": "127.0.0.1", "msg": "", "rc": 0, "start": "2026-08-06 12:04:57.535944", "stderr": "", "stderr_lines": [], "stdout": "IP 127.0.0.1 does match certificate", "stdout_lines": ["IP 127.0.0.1 does match certificate"]}
ok: [prodnode1] => (item=::1) => {"ansible_loop_var": "item", "changed": false, "cmd": ["openssl", "x509", "-noout", "-in", "/etc/kubernetes/ssl/apiserver.crt", "-checkip", "::1"], "delta": "0:00:00.003707", "end": "2026-08-06 12:04:57.698769", "failed_when_result": false, "item": "::1", "msg": "", "rc": 0, "start": "2026-08-06 12:04:57.695062", "stderr": "", "stderr_lines": [], "stdout": "IP ::1 does match certificate", "stdout_lines": ["IP ::1 does match certificate"]}
ok: [prodnode1] => (item=172.16.8.137) => {"ansible_loop_var": "item", "changed": false, "cmd": ["openssl", "x509", "-noout", "-in", "/etc/kubernetes/ssl/apiserver.crt", "-checkip", "172.16.8.137"], "delta": "0:00:00.005047", "end": "2026-08-06 12:04:57.872313", "failed_when_result": false, "item": "172.16.8.137", "msg": "", "rc": 0, "start": "2026-08-06 12:04:57.867266", "stderr": "", "stderr_lines": [], "stdout": "IP 172.16.8.137 does match certificate", "stdout_lines": ["IP 172.16.8.137 does match certificate"]}
Donnerstag 06 August 2026 14:04:57 +0200 (0:00:00.750) 0:04:46.890 *****

TASK [kubernetes/control-plane : Kubeadm | Check apiserver.crt SAN hosts] ****************************************************************************************
ok: [prodnode1] => (item=prodnode1) => {"ansible_loop_var": "item", "changed": false, "cmd": ["openssl", "x509", "-noout", "-in", "/etc/kubernetes/ssl/apiserver.crt", "-checkhost", "prodnode1"], "delta": "0:00:00.004629", "end": "2026-08-06 12:04:58.091155", "failed_when_result": false, "item": "prodnode1", "msg": "", "rc": 0, "start": "2026-08-06 12:04:58.086526", "stderr": "", "stderr_lines": [], "stdout": "Hostname prodnode1 does match certificate", "stdout_lines": ["Hostname prodnode1 does match certificate"]}
ok: [prodnode1] => (item=kubernetes.default.svc) => {"ansible_loop_var": "item", "changed": false, "cmd": ["openssl", "x509", "-noout", "-in", "/etc/kubernetes/ssl/apiserver.crt", "-checkhost", "kubernetes.default.svc"], "delta": "0:00:00.003814", "end": "2026-08-06 12:04:58.289883", "failed_when_result": false, "item": "kubernetes.default.svc", "msg": "", "rc": 0, "start": "2026-08-06 12:04:58.286069", "stderr": "", "stderr_lines": [], "stdout": "Hostname kubernetes.default.svc does match certificate", "stdout_lines": ["Hostname kubernetes.default.svc does match certificate"]}
ok: [prodnode1] => (item=kubernetes) => {"ansible_loop_var": "item", "changed": false, "cmd": ["openssl", "x509", "-noout", "-in", "/etc/kubernetes/ssl/apiserver.crt", "-checkhost", "kubernetes"], "delta": "0:00:00.005283", "end": "2026-08-06 12:04:58.489282", "failed_when_result": false, "item": "kubernetes", "msg": "", "rc": 0, "start": "2026-08-06 12:04:58.483999", "stderr": "", "stderr_lines": [], "stdout": "Hostname kubernetes does match certificate", "stdout_lines": ["Hostname kubernetes does match certificate"]}
ok: [prodnode1] => (item=lb-apiserver.kubernetes.local) => {"ansible_loop_var": "item", "changed": false, "cmd": ["openssl", "x509", "-noout", "-in", "/etc/kubernetes/ssl/apiserver.crt", "-checkhost", "lb-apiserver.kubernetes.local"], "delta": "0:00:00.004793", "end": "2026-08-06 12:04:58.667607", "failed_when_result": false, "item": "lb-apiserver.kubernetes.local", "msg": "", "rc": 0, "start": "2026-08-06 12:04:58.662814", "stderr": "", "stderr_lines": [], "stdout": "Hostname lb-apiserver.kubernetes.local does match certificate", "stdout_lines": ["Hostname lb-apiserver.kubernetes.local does match certificate"]}
ok: [prodnode1] => (item=localhost) => {"ansible_loop_var": "item", "changed": false, "cmd": ["openssl", "x509", "-noout", "-in", "/etc/kubernetes/ssl/apiserver.crt", "-checkhost", "localhost"], "delta": "0:00:00.005034", "end": "2026-08-06 12:04:58.839864", "failed_when_result": false, "item": "localhost", "msg": "", "rc": 0, "start": "2026-08-06 12:04:58.834830", "stderr": "", "stderr_lines": [], "stdout": "Hostname localhost does match certificate", "stdout_lines": ["Hostname localhost does match certificate"]}
ok: [prodnode1] => (item=kubernetes.default.svc.cluster.local) => {"ansible_loop_var": "item", "changed": false, "cmd": ["openssl", "x509", "-noout", "-in", "/etc/kubernetes/ssl/apiserver.crt", "-checkhost", "kubernetes.default.svc.cluster.local"], "delta": "0:00:00.004996", "end": "2026-08-06 12:04:59.008417", "failed_when_result": false, "item": "kubernetes.default.svc.cluster.local", "msg": "", "rc": 0, "start": "2026-08-06 12:04:59.003421", "stderr": "", "stderr_lines": [], "stdout": "Hostname kubernetes.default.svc.cluster.local does match certificate", "stdout_lines": ["Hostname kubernetes.default.svc.cluster.local does match certificate"]}
ok: [prodnode1] => (item=kubernetes.default) => {"ansible_loop_var": "item", "changed": false, "cmd": ["openssl", "x509", "-noout", "-in", "/etc/kubernetes/ssl/apiserver.crt", "-checkhost", "kubernetes.default"], "delta": "0:00:00.004886", "end": "2026-08-06 12:04:59.184190", "failed_when_result": false, "item": "kubernetes.default", "msg": "", "rc": 0, "start": "2026-08-06 12:04:59.179304", "stderr": "", "stderr_lines": [], "stdout": "Hostname kubernetes.default does match certificate", "stdout_lines": ["Hostname kubernetes.default does match certificate"]}
Donnerstag 06 August 2026 14:04:59 +0200 (0:00:01.317) 0:04:48.208 *****
Donnerstag 06 August 2026 14:04:59 +0200 (0:00:00.027) 0:04:48.236 *****
Donnerstag 06 August 2026 14:04:59 +0200 (0:00:00.019) 0:04:48.256 *****
Donnerstag 06 August 2026 14:04:59 +0200 (0:00:00.016) 0:04:48.272 *****
Donnerstag 06 August 2026 14:04:59 +0200 (0:00:00.018) 0:04:48.291 *****
Donnerstag 06 August 2026 14:04:59 +0200 (0:00:00.017) 0:04:48.308 *****

TASK [kubernetes/control-plane : Remove binding to anonymous user] ***********************************************************************************************
fatal: [prodnode1]: FAILED! => {"changed": true, "cmd": ["/usr/local/bin/kubectl", "--kubeconfig", "/etc/kubernetes/admin.conf", "-n", "kube-public", "delete", "rolebinding", "kubeadm:bootstrap-signer-clusterinfo", "--ignore-not-found"], "delta": "0:00:00.049898", "end": "2026-08-06 12:04:59.530895", "msg": "non-zero return code", "rc": 1, "start": "2026-08-06 12:04:59.480997", "stderr": "The connection to the server 127.0.0.1:6443 was refused - did you specify the right host or port?", "stderr_lines": ["The connection to the server 127.0.0.1:6443 was refused - did you specify the right host or port?"], "stdout": "", "stdout_lines": []}

Metadata

Metadata

Assignees

Labels

Ubuntu 22kind/bugCategorizes issue or PR as related to a bug.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions