Strimzi
[Bug]: CertificateExpiredException for certificate (.crt) on strimzi kafka component #12629
Replies: 1 comment 4 replies
-
|
If the operator does not work, there is nobody to renew the certificates. So you have to fix that first. Second, if the certifciates already expired, the operator won't be able to fix it on its own. You would need to help it - typically by manually deleting the ZooKeeper / Kafka Pods depending on the situation. But first, you need to get the operator into the working state. So you should focus on it. From the logs, I have no idea why it is crash looping as none of the logs you shared is really complete or explain it. So you should probably start there and provide more information, full logs, full configuration, etc. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @scholzj
Typically, it will trigger by recreating all strimzi kafka pod components, right ? But it doesn't renew other ca certificate and then make strimzi other pods (kafka broker, cruise control, etc.) Not recreated *cmiiw
And current strimzi-cluster-operator POD is on running state with log as attachment above. So, Do we need to ensure there are no error in strimzi-cluster-operator log pod even though the pod is successfully on running state, then we can proceed to remediate the rest of CLBO error on other strimzi kafka pod ? |
Beta Was this translation helpful? Give feedback.
-
|
That does not help if the CAs are already expired. But in any case, as I said, if the operator is not running properly and is crash-looping, you need to fix that first. The log you shared does not correspond to you saying that the Cluster Operator pod is crash-looping, so not sore what it really does. But the error you have there would block the certificate renewal. So you need to address it in any case. The error suggests some forbidden maniapulation with the storage configuration. So you would need to find out by looking into what your storage configuration is and what your PVs / PVCs say and you would need to somehow reconcile it manually. Probably by increasing your stroage capacity in the Kafka/KafkaNodePool CRs. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @scholzj . Absolutely, you're right !🔥 Since strimzi-cluster-operator pod is running but somehow, there are still some error related to PVC on kafka strimzipodset, as below:
SOLUTION
Thanks alot for your help @scholzj ✨️ |
Beta Was this translation helpful? Give feedback.
-
|
Great. Glad we managed to fix it. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Bug Description
Hi,
We have EXISITING Kafka Cluster within EKS Cluster (AWS, using version: 1.33), installing by using strimzi-kafka-operator Helm Chart (version: 0.45.2) on namespace
cdcand cluster namecdc-cluster.The following are some of the components installed within EKS cluster:
a) topic operator
b) user operator
for those component, we deploy by applying custom kafka-persistent.yaml.
Chronology
USING Helm chart 0.45.0 Version
strimzi-cluster-operatorlogs, it said as below:STEP 1: Upgrading Helm chart 0.45.2 Version
please see selected log for strimzi-kafka-operator (Since 24h) as below:
based on error above, we notice that:
even though, we have upgraded strimzi-cluster-operator into 0.45.2, the rest of pods component still has error pod log as below:
STEP 2: Trying to ANNOTATE force-renew Cluster Operator-managed CA certificates
seems, one of certificate strimzi was expired since Dec 17 23:30:03 GMT 2025 and not created automatically due to strimzi operator CLBO error before.
then, based on official docs (0.45.2):
https://strimzi.io/docs/operators/0.45.2/deploying.html#proc-recovering-expired-ca-certs-str, we tried to renew the ca-certificate for cluster operator, by using kubectl annotate command as below:cdc-cluster-clients-ca-certandcdc-cluster-cluster-ca-certalready renewed. As below:Only zookeeper pod already recreated. But the others, not created
and also, the errors for zookeeper are STILL similar as before:
cluster name: cdc-cluster
cdc-cluster-kafka-brokersthat already expired. And we're confirmed that current kafka-broker certificate not renewed, by tried to exec through the pod. as below:for example
cdc-cluster-kafka-0.crtSTEP 3: Trying to Kubectl restart for selected pod (kafka-broker)
kafka-broker-0 POD, to tried, will cdc-cluster-kafka-brokers secret be renewed into new kafka-broker pod., but we got CLBO error and cdc-cluster-kafka-broker not renewed, here is the error log for kafka-broker-0 pod as below:and now, here are pod condition in our cluster
Any kind of help is greatly appreciated. let me know if there are some logs missing.
Thankyou.
Steps to reproduce
helm upgrade --install --wait --debug --namespace=cdc strimzi-kafka-cluster-operator --version 0.45.2 oci://quay.io/strimzi-helm/strimzi-kafka-operatorExpected behavior
All of strimzi kafka operator component are running with renewed all certificates.
a) topic operator
b) user operator
kafka connect (kind: kafkaconnect) (strimzipodset) 3 replicas
certificate secret renewed
Strimzi version
0.45.2
Kubernetes version
Kubernetes 1.33
Installation method
Helm Chart
Infrastructure
AWS EKS 1.33
Configuration files and logs
kafka-persistent.yaml (modified)
kafka-connect.yaml (modified)
Additional context
No response
Beta Was this translation helpful? Give feedback.
All reactions