Skip to content

Script to renew cluster certificates #9709

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 3, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
title: "Certificate management"
linkTitle: "Certificate management"
weight: 50
aliases:
/docs/tasks/cluster/cluster-certificates/
description: >
How to renew EKS Anywhere cluster certificates
---
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
---
title: "Certificate rotation"
linkTitle: "Certificate rotation"
title: "Manual steps to renew certificates"
linkTitle: "Manual steps to renew certificates"
weight: 31
aliases:
/docs/tasks/cluster/manually-renew-certs/
/docs/tasks/cluster/manual-steps-renew-certs/
date: 2021-11-04
description: >
How to rotate certificates for etcd and control plane nodes
Expand Down Expand Up @@ -36,7 +36,7 @@ echo | openssl s_client -connect ${CONTROL_PLANE_IP}:6443 2>/dev/null | openssl
echo | openssl s_client -connect ${EXTERNAL_ETCD_IP}:2379 2>/dev/null | openssl x509 -noout -dates
```

You can rotate certificates by following the steps given below. You cannot rotate the `ca` certificate because it is the root certificate. Note that the commands used for Bottlerocket nodes are different than those for Ubuntu and RHEL nodes.
>**_NOTE:_** You can rotate certificates by following the steps given below. You cannot rotate the `ca` certificate because it is the root certificate. Note that the commands used for Bottlerocket nodes are different than those for Ubuntu and RHEL nodes.

#### External etcd nodes

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
---
title: "Script to renew cluster certificates"
linkTitle: "Script to renew certificates"
description: >
Step-by-step guide to renew Kubernetes certificates on EKS Anywhere clusters using a script
weight: 20
---

Get more information on EKS Anywhere cluster certificates from [here]({{< relref "manual-steps-renew-certs.md" >}})

This script automates:

- Certificate renewal for etcd and control plane nodes
- Cleanup of temporary files if certificates are renewed and cluster is healthy

### Prerequisites

- Admin machine with:
- `kubectl`, `yq`, `jq`, `scp`, `ssh`, and `sudo` installed
- SSH access to all control plane and etcd nodes

### Steps

1. Setup environment variable:

```bash
export KUBECONFIG=<path-to-management-cluster-kubeconfig>
```

2. Prepare a `keys-config.yaml` file

Add node and private key information of your control plane and/or external etcd to a file, such as `keys-config.yaml`:

```bash
clusterName: <cluster-name>
controlPlane:
nodes:
- <control-plane-1-ip>
- <control-plane-2-ip>
- <control-plane-3-ip>
sshKey: <complete-path-to-private-ssh-key>
sshUser: <ssh-user>
etcd:
nodes:
- <external-etcd-1-ip>
- <external-etcd-2-ip>
- <external-etcd-3-ip>
sshKey: <complete-path-to-private-ssh-key>
sshUser: <ssh-user>
```

3. Download the Script

{{< tabpane >}}
{{< tab header="Ubuntu or RHEL" lang="bash" >}}
```bash
curl -O https://raw.githubusercontent.com/aws/eks-anywhere/refs/heads/main/scripts/renew_certificates.sh
chmod +x renew_certificates.sh
```
{{< /tab >}}
{{< tab header="Bottlerocket" lang="bash" >}}
```bash
curl -O https://raw.githubusercontent.com/aws/eks-anywhere/refs/heads/main/scripts/renew_certificates_bottlerocket.sh
chmod +x renew_certificates_bottlerocket.sh
```
{{< /tab >}}
{{< /tabpane >}}

4. Run the Script as a `sudo` user

```bash
sudo ./renew_certificates.sh -f keys-config.yaml
```


### What the Script Does

- Backs up:
- All etcd certificates (in case of external etcd)
- Control plane certificates
- Renews external etcd certificates
- Updates the Kubernetes secret `apiserver-etcd-client` if api server is reachable
- Renews all kubeadm certificates
- Restarts static control plane pods
- Cleans up temporary certs and backup folders (only if certificates are renewed successfully and cluster is healthy)
4 changes: 2 additions & 2 deletions docs/content/en/docs/troubleshooting/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -332,7 +332,7 @@ ${IMAGE_ID} tmp-certs-check \
{{< /tab >}}
{{< /tabpane >}}

EKS Anywhere typically renews certificates when upgrading a cluster. However, if a cluster has not been upgraded for over a year, then it is necessary to manually renew these certificates. Please see [Certificate rotation]({{< relref "../clustermgmt/security/manually-renew-certs.md" >}}) to manually rotate expired certificates.
EKS Anywhere typically renews certificates when upgrading a cluster. However, if a cluster has not been upgraded for over a year, then it is necessary to manually renew these certificates. Please see [Certificate rotation]({{< relref "../clustermgmt/certificate-management/manual-steps-renew-certs.md" >}}) to manually rotate expired certificates.

### Bootstrap cluster fails to come up

Expand Down Expand Up @@ -630,7 +630,7 @@ kubectl get nodes --no-headers -l '!node-role.kubernetes.io/control-plane' -o js
```

### No static pods is running in container runtime on control plane nodes
When the kubelet systemd service is running on a control plane node and you notice that no [static pods](https://kubernetes.io/docs/concepts/workloads/pods/#static-pods) are running, this could be due to a certificate issue. Check the kubelet service logs. If you see error messages like the ones below, renew the kubelet-client-current.pem certificate by following [these steps]({{< relref "../clustermgmt/security/manually-renew-certs/#kubelet" >}}):
When the kubelet systemd service is running on a control plane node and you notice that no [static pods](https://kubernetes.io/docs/concepts/workloads/pods/#static-pods) are running, this could be due to a certificate issue. Check the kubelet service logs. If you see error messages like the ones below, renew the kubelet-client-current.pem certificate by following [these steps]({{< relref "../clustermgmt/certificate-management/manual-steps-renew-certs/#kubelet" >}}):
```
part of the existing bootstrap client certificate in /etc/kubernetes/kubelet/kubeconfig
Loading cert/key pair from "/var/lib/kubelet/pki/kubelet-client-current.pem
Expand Down
Loading
Loading