Issue with Changing StorageClass in Strimzi Kafka Helm Chart

[rizsk1220]

Strimzi Kafka installed via Helm chart
Current configuration: 3 ZooKeeper and 3 kafka pods

we have Strimzi deployed on GKE and they were using non-cmek disks.
we tried to migrate data on the non-cmek disk to cmek disk using volumesnapshot tool on GKE cluster.
after the snapshot migration, we tried to attach the new cmek disk to the strimzi operator and tried to bring up but strimzi operator is overwriting the cmek disk with no data.

old
storage:
type: persistent-claim

• class: "premium-rwo"
  size: "20Gi"
  deleteClaim: true

New

{{- if .Values.storageClassCreate -}}
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: {{ .Values.storageClass | default "csi-strimzi-pd-cmek" | quote }}
provisioner: pd.csi.storage.gke.io
volumeBindingMode: {{ .Values.volumeBindingMode | default "Immediate" | quote }}
allowVolumeExpansion: true
parameters:
type: {{ .Values.storageClassParametersType | default "pd-ssd" | quote }}
disk-encryption-kms-key: {{ ......................................... }}
{{- end -}}

How can I properly perform disk maintenance on the cluster to change the StorageClass?

Specifically, I want to achieve a rolling update that would:

Replace all PVCs and PVs in the ZooKeeper and kafka pods
Use the new default StorageClass (or any other class)
Preserve all data across the cluster
Maintain cluster stability during the transition (no downtime)

[kyguy]

Hi @rizsk1220, changing the StorageClass by updating the class field in the Kafka resource is not supported by Strimzi. From what I understand, to migrate to new disks using the new StorageClass you can add the new disks to your JBOD configuration and leverage Cruise Control to migrate the partitions off of the old disks and onto the new disks [1]. For more information on how to do this checkout the blog post here [1]

[1] https://strimzi.io/blog/2025/02/13/moving-data-between-jbod-disks-using-cruise-control/

[rizsk1220]

We took a volume snapshot of the old Persistent Volume Claim (PVC) and restored that snapshot to a newly created PVC. However, when we attached the new PVCs to the operator, the operator seemingly overwrote them without retaining the data, leading to data loss.
We would like to understand the correct way to use snapshots for restoring data onto newly created PVCs. Specifically:

1. What happens to the data in the restored PVC when the operator attaches to it?
2. How does the operator handle restored PVCs, and why did it overwrite the data?
3. What is the best approach recommended by Strimzi to restore data during disk failures, ensuring that the operator does not overwrite the restored PVCs?”

[kyguy]

What happens to the data in the restored PVC when the operator attaches to it?

That data is restored.

How does the operator handle restored PVCs, and why did it overwrite the data?

Following the guidelines documented here [1] should allow for data recovery. If there was any data loss I imagine it could have been caused by a misconfiguration in the restored PVC. From the guide:

"it is essential that PVs are mounted into the correct PVC to avoid data corruption. A volumeName is specified for the PVC and this must match the name of the PV."

What is the best approach recommended by Strimzi to restore data during disk failures, ensuring that the operator does not overwrite the restored PVCs?”

Follow the guidelines here [1]

[1] https://strimzi.io/docs/operators/latest/full/deploying.html#assembly-cluster-recovery-volume-str