Script to renew cluster certificates #9709

panktishah26 · 2025-05-03T23:49:23Z

Description of changes:
The PR includes script to renew external etcd (if present) and control plane certificates for Bottlerocket, Ubuntu, RHEL OS.

Testing (if applicable):

Documentation added/planned (if applicable):

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

eks-distro-bot · 2025-05-03T23:49:27Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from panktishah26. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

codecov · 2025-05-03T23:54:42Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 69.86%. Comparing base (cb875e8) to head (4409889).
Report is 2 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #9709      +/-   ##
==========================================
+ Coverage   69.84%   69.86%   +0.01%     
==========================================
  Files         672      672              
  Lines       49810    49831      +21     
==========================================
+ Hits        34791    34812      +21     
  Misses      13236    13236              
  Partials     1783     1783

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

vivek-koppuru

Let's have one of the doc writers review the changes to the docs too as they are major ones

vivek-koppuru · 2025-05-07T06:54:45Z

docs/content/en/docs/clustermgmt/security/certificate-management/script-renew-certs.md

+
+Use this script **if your cluster certificates are nearing expiration**.
+
+If any certificate is **already expired**, follow the [manual renewal steps](https://anywhere.eks.amazonaws.com/docs/clustermgmt/security/manually-renew-certs/) instead.


This link would change after you moving it right? Can you see where else this link needs to change, and how the security page will look now?

vivek-koppuru · 2025-05-07T06:57:39Z

docs/content/en/docs/clustermgmt/security/certificate-management/script-renew-certs.md

+
+## Overview
+
+Use this script **if your cluster certificates are nearing expiration**.


Will we have a way to check this? We can use kubeadm certs check-expiration on the node to get this information, or if there is a golang api for this

vivek-koppuru · 2025-05-07T07:00:03Z

docs/content/en/docs/clustermgmt/security/certificate-management/script-renew-certs.md

+
+- Certificate renewal for etcd and control plane nodes.
+- Safe backup of existing certificates and the `kubeadm-config` ConfigMap.
+- Cleanup of temporary files if cetificates are renewed and cluster is healthy.


Suggested change

- Cleanup of temporary files if cetificates are renewed and cluster is healthy.

- Cleanup of temporary files if certificates are renewed and cluster is healthy.

vivek-koppuru · 2025-05-07T07:00:23Z

docs/content/en/docs/clustermgmt/security/certificate-management/script-renew-certs.md

+- Admin machine with:
+  - `kubectl`, `jq`, `scp`, `ssh`, and `sudo` installed
+- SSH access to all control plane and etcd nodes


There is a mix of punctation and not

vivek-koppuru · 2025-05-07T07:01:01Z

docs/content/en/docs/clustermgmt/security/certificate-management/script-renew-certs.md

+### 3. Run the Script
+
+```bash
+./renew_certificates.sh <cluster-name> <ssh-user> <path-to-ssh-private-key>


No flags? This might make it confusing

vivek-koppuru · 2025-05-07T07:01:38Z

docs/content/en/docs/troubleshooting/troubleshooting.md

@@ -630,7 +630,7 @@ kubectl get nodes --no-headers -l '!node-role.kubernetes.io/control-plane' -o js
 ```

 ### No static pods is running in container runtime on control plane nodes
-When the kubelet systemd service is running on a control plane node and you notice that no [static pods](https://kubernetes.io/docs/concepts/workloads/pods/#static-pods) are running, this could be due to a certificate issue. Check the kubelet service logs. If you see error messages like the ones below, renew the kubelet-client-current.pem certificate by following [these steps]({{< relref "../clustermgmt/security/manually-renew-certs/#kubelet" >}}):
+When the kubelet systemd service is running on a control plane node and you notice that no [static pods](https://kubernetes.io/docs/concepts/workloads/pods/#static-pods) are running, this could be due to a certificate issue. Check the kubelet service logs. If you see error messages like the ones below, renew the kubelet-client-current.pem certificate by following [these steps]({{< relref "../clustermgmt/security/certificate-management/manually-renew-certs.md/#kubelet" >}}):


The .md looks wrong here, will it work?

vivek-koppuru · 2025-05-07T07:03:21Z

scripts/renew_certificates.sh

+function renew_etcd_certs() {
+    cat <<EOF
+# Renew certificates
+sudo etcdadm join phase certificates http://eks-a-etcd-dumb-url


Is this url configurable?

vivek-koppuru · 2025-05-07T07:05:08Z

scripts/renew_certificates_bottlerocket.sh

+cp -r ${ETCD_CERT_DIR}/pki ${ETCD_CERT_DIR}/pki.bak_${BACKUP_DATE}
+rm ${ETCD_CERT_DIR}/pki/*
+cp ${ETCD_CERT_DIR}/pki.bak_${BACKUP_DATE}/ca.* ${ETCD_CERT_DIR}/pki
+echo "✅ Certs backedup--"


Suggested change

echo "✅ Certs backedup--"

echo "✅ Certs backed up--"

vivek-koppuru · 2025-05-27T20:56:25Z

docs/content/en/docs/clustermgmt/security/certificate-management/script-renew-certs.md

+echo | openssl s_client -connect ${EXTERNAL_ETCD_IP}:2379 2>/dev/null | openssl x509 -noout -dates
+```
+
+You can rotate certificates by following the steps given below. You cannot rotate the `ca` certificate because it is the root certificate. Note that the commands used for Bottlerocket nodes are different than those for Ubuntu and RHEL nodes.


Make this a Note or something

vivek-koppuru · 2025-05-27T20:56:52Z

docs/content/en/docs/clustermgmt/security/certificate-management/script-renew-certs.md

+## Overview
+This script automates:
+
+- Certificate renewal for etcd and control plane nodes.


Can we just remove the punctuation all throughout for bullet points? Not sure if we do that in other sections

vivek-koppuru · 2025-05-27T20:59:47Z

docs/content/en/docs/clustermgmt/security/certificate-management/script-renew-certs.md

@@ -0,0 +1,111 @@
+---
+title: "Manually Renew Cluster Certificates"
+linkTitle: "Script To Renew Certificates"


I am curious if these instructions should just be in the README of the repo and we can just refer the readme in the official docs. But I would defer to docs writer on that

github-actions bot added area/docs Documentation documentation labels May 3, 2025

eks-distro-bot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label May 3, 2025

panktishah26 force-pushed the script-renew-certs branch 2 times, most recently from 2caa5b2 to 9912a3b Compare May 5, 2025 17:00

vivek-koppuru reviewed May 7, 2025

View reviewed changes

panktishah26 force-pushed the script-renew-certs branch from 9912a3b to 52a269c Compare May 21, 2025 21:41

Script to renew cluster certificates

4409889

panktishah26 force-pushed the script-renew-certs branch from 52a269c to 4409889 Compare May 21, 2025 22:22

vivek-koppuru reviewed May 27, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Script to renew cluster certificates #9709

Script to renew cluster certificates #9709

panktishah26 commented May 3, 2025

Uh oh!

eks-distro-bot commented May 3, 2025

Uh oh!

codecov bot commented May 3, 2025 •

edited

Loading

Uh oh!

vivek-koppuru left a comment

Uh oh!

vivek-koppuru May 7, 2025

Uh oh!

vivek-koppuru May 7, 2025

Uh oh!

vivek-koppuru May 7, 2025

Uh oh!

vivek-koppuru May 7, 2025

Uh oh!

vivek-koppuru May 7, 2025

Uh oh!

vivek-koppuru May 7, 2025

Uh oh!

vivek-koppuru May 7, 2025

Uh oh!

vivek-koppuru May 7, 2025

Uh oh!

vivek-koppuru May 27, 2025

Uh oh!

vivek-koppuru May 27, 2025

Uh oh!

vivek-koppuru May 27, 2025

Uh oh!

Uh oh!


		Use this script if your cluster certificates are nearing expiration.

		If any certificate is already expired, follow the [manual renewal steps](https://anywhere.eks.amazonaws.com/docs/clustermgmt/security/manually-renew-certs/) instead.


		## Overview

		Use this script if your cluster certificates are nearing expiration.

	- Cleanup of temporary files if cetificates are renewed and cluster is healthy.
	- Cleanup of temporary files if certificates are renewed and cluster is healthy.

Script to renew cluster certificates #9709

Are you sure you want to change the base?

Script to renew cluster certificates #9709

Conversation

panktishah26 commented May 3, 2025

Uh oh!

eks-distro-bot commented May 3, 2025

Uh oh!

codecov bot commented May 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

vivek-koppuru left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

codecov bot commented May 3, 2025 •

edited

Loading