Skip to content

PerconaPGCluster standby removal not propagated to inner PostgresCluster — cluster stuck in standby after spec.standby removed #1687

Description

@fabriceellti8m

Report

When a PerconaPGCluster is configured with spec.standby and then the spec.standby block is removed (to promote the cluster to primary), the Percona operator reconciles the outer CR but does not update the inner upstream.pgv2.percona.com/v1beta1 PostgresCluster object it manages. The inner object retains spec.standby.enabled: true, which keeps the Patroni DCS standby_cluster config in place and leaves PostgreSQL in recovery mode indefinitely.

More about the problem

Expected behavior

Removing spec.standby from PerconaPGCluster should cause the operator to update the inner PostgresCluster to remove spec.standby, which in turn clears standby_cluster from the Patroni DCS and triggers promotion to primary.

Actual behavior

The inner PostgresCluster is not updated. The operator appears to use a merge patch that omits absent fields rather than explicitly nulling them out, so the existing standby field on the inner object persists across reconciles.

Workaround

Manually patch the inner PostgresCluster to remove the standby field:

oc patch postgrescluster.upstream.pgv2.percona.com
-n --type=merge
-p '{"spec":{"standby":null}}'
This clears the DCS standby_cluster config and Patroni promotes to primary within ~30 seconds.

Steps to reproduce

  1. Create a PerconaPGCluster with spec.standby.enabled: true pointing to a pgBackRest repo
  2. Confirm the cluster is in standby (pg_is_in_recovery() = true)
  3. Remove the spec.standby block from the PerconaPGCluster CR and apply it
  4. Observe that the outer CR reconciles successfully (observedGeneration matches generation)
  5. Check the inner PostgresCluster:
    oc get postgrescluster.upstream.pgv2.percona.com -n -o jsonpath='{.spec.standby}'
    → standby.enabled: true is still present, generation unchanged
  6. Check Patroni DCS:
    oc get endpoints -ha-config -n -o jsonpath='{.metadata.annotations.config}' | python3 - m json.tool | grep -A5 standby
    → standby_cluster block is still present in DCS
  7. pg_is_in_recovery() still returns true
  8. Restarting the operator pod does not resolve the issue

Versions

  1. OpenShift 4.20.26
  2. Operator 3.0.0
  3. Database Postgres 16

Anything else?

Besides this issue, which can be worked around quite effortlessly, so far the migrations from crunchy to this operator have been going well, great work 👍

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions