I’ve observed the following behavior in Vertica version 24.1. When a configuration change triggers a rolling restart, the operator generates a brand-new secret containing the NMA certificates. This happens in clusters where no custom secret has been specified, i.e.:
During the rolling restart, the first new pod is issued the new secret, while the remaining (older) pods still hold the previous version. This leads to a mismatch in TLS certificates across the cluster, resulting in SSL failures on Kubernetes probes. As a result, the new pod never becomes healthy, and the operator is also unable to communicate properly with the nodes.
The only workaround we’ve found is to ensure that all pods use the same secret, either by manually recreating the StatefulSet or by providing a pre-defined secret with consistent TLS certificates.
It seems odd that new certificates are generated on every rolling restart. Is there a specific reason for this behavior? Would it be possible to modify the logic to reuse an existing secret if one is already present, rather than always creating a new one?
I’ve observed the following behavior in Vertica version 24.1. When a configuration change triggers a rolling restart, the operator generates a brand-new secret containing the NMA certificates. This happens in clusters where no custom secret has been specified, i.e.:
During the rolling restart, the first new pod is issued the new secret, while the remaining (older) pods still hold the previous version. This leads to a mismatch in TLS certificates across the cluster, resulting in SSL failures on Kubernetes probes. As a result, the new pod never becomes healthy, and the operator is also unable to communicate properly with the nodes.
The only workaround we’ve found is to ensure that all pods use the same secret, either by manually recreating the StatefulSet or by providing a pre-defined secret with consistent TLS certificates.
It seems odd that new certificates are generated on every rolling restart. Is there a specific reason for this behavior? Would it be possible to modify the logic to reuse an existing secret if one is already present, rather than always creating a new one?