Skip to content

Commit 6e04454

Browse files
mitchrossclaude
andcommitted
cnpg: flip all 4 DBs out of recovery -> initdb (post-nuke hygiene)
After the 2026-06-28 kopiur-only nuke, all four CNPG DBs recovered from their prior lineage and write forward to the bumped target (gitea v10, immich v6, paperless v6, temporal v8). Flip the root kustomizations overlays/recovery -> overlays/initdb so a future PVC loss can't auto-restore the stale source lineage; recovery is re-enabled per-DB only as an explicit DR action. No-op on the running healthy clusters (CNPG reads bootstrap only at creation); serverName unchanged. Also refresh the lineage table + mark the Barman-plugin migration done in infrastructure/database/CLAUDE.md. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent fedcc60 commit 6e04454

5 files changed

Lines changed: 32 additions & 18 deletions

File tree

infrastructure/database/CLAUDE.md

Lines changed: 20 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -56,10 +56,18 @@ The `serverName` values below live in each DB's `base/cluster.yaml` and
5656

5757
| Database | Current write target (base) | Prior lineage (recovery source) |
5858
|-----------|------------------------------|---------------------------------|
59-
| gitea | `gitea-database-v9` | `gitea-database-v6` |
60-
| immich | `immich-database-v5` | `immich-database-v4` |
61-
| paperless | `paperless-database-v5` | `paperless-database-v4` |
62-
| temporal | `temporal-database-v7` | `temporal-database-v6` |
59+
| gitea | `gitea-database-v10` | `gitea-database-v9` |
60+
| immich | `immich-database-v6` | `immich-database-v5` |
61+
| paperless | `paperless-database-v6` | `paperless-database-v5` |
62+
| temporal | `temporal-database-v8` | `temporal-database-v7` |
63+
64+
2026-06-28 (first kopiur-only full nuke): all four recovered from their prior
65+
lineage and write forward to the bumped target above (gitea v9→v10, immich
66+
v5→v6, paperless v5→v6, temporal v7→v8). After recovery completed and the
67+
primaries went healthy, all four root kustomizations were **flipped back
68+
`overlays/recovery` → `overlays/initdb`** (steady state) so a future PVC loss
69+
can't auto-restore a stale lineage. Recovery is re-enabled per-DB only as an
70+
explicit DR action.
6371

6472
All four bumped TWICE on 2026-06-11: once for the Longhorn V2 rebuild nuke,
6573
and again for the same-day re-nuke (SPDK cpu-mask validation run) because the
@@ -129,9 +137,14 @@ See the full runbook in [`docs/domains/cnpg/disaster-recovery.md`](../../docs/do
129137

130138
## Deprecation warnings
131139

132-
- **Native `spec.backup.barmanObjectStore`** — will be removed in CNPG 1.30.0.
133-
Migrate to the Barman Cloud Plugin (already installed at
134-
`infrastructure/database/cnpg-barman-plugin/`). Not urgent; track release notes.
140+
- **Native `spec.backup.barmanObjectStore` — MIGRATION DONE.** All four DBs use
141+
the Barman Cloud Plugin (`infrastructure/database/cnpg-barman-plugin/`):
142+
live backup config is `spec.plugins[]` (name `barman-cloud.cloudnative-pg.io`,
143+
`isWALArchiver: true`, `parameters.barmanObjectName` → a sibling `ObjectStore`
144+
CR + `parameters.serverName`), and recovery is `externalClusters[].plugin`.
145+
The in-tree `barmanObjectStore` field is removed in CNPG 1.30.0 — do not
146+
reintroduce it. (When adding a NEW DB, set serverName via the plugin
147+
`parameters`, not `backup.barmanObjectStore`.)
135148
- **`spec.monitoring.enablePodMonitor`** — deprecated, replace with manually-
136149
managed `PodMonitor` resources per cluster.
137150

infrastructure/database/cloudnative-pg/gitea/kustomization.yaml

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -20,10 +20,11 @@ commonLabels:
2020
commonAnnotations:
2121
argocd.argoproj.io/sync-wave: "-5"
2222
resources:
23-
- overlays/recovery # ← ACTIVE: restore gitea DB FROM v9 (current data,
24-
# pre-nuke). Writes forward to v10 (clean, unused
25-
# prefix) to pass barman-cloud-check-wal-archive.
26-
# - overlays/initdb # fresh empty DB (1 admin, 0 repos). Was active
27-
# while v5 was (wrongly) assumed unrestorable.
23+
# - overlays/recovery # DR-only: restore gitea DB FROM v9 → write v10.
24+
# Before a planned nuke, bump base serverName and
25+
# set this overlay's source, then re-enable.
26+
- overlays/initdb # ← ACTIVE (steady state; recovery completed
27+
# 2026-06-28). bootstrap.initdb is a no-op on the
28+
# existing cluster; writes forward to base v10.
2829
- externalsecret.yaml
2930
- scheduled-backup.yaml

infrastructure/database/cloudnative-pg/immich/kustomization.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ commonLabels:
1212
commonAnnotations:
1313
argocd.argoproj.io/sync-wave: "-5"
1414
resources:
15-
- overlays/recovery # ← ACTIVE: restore from Barman v5 → write v6 (nuke recovery)
16-
# - overlays/initdb # fresh DB / normal operation
15+
# - overlays/recovery # DR-only: restore from Barman v5 → write v6. Re-enable before a nuke.
16+
- overlays/initdb # ← ACTIVE (steady state; recovered 2026-06-28). No-op on existing cluster; writes v6.
1717
- externalsecret.yaml
1818
- scheduled-backup.yaml

infrastructure/database/cloudnative-pg/paperless/kustomization.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ commonLabels:
1111
commonAnnotations:
1212
argocd.argoproj.io/sync-wave: "-5"
1313
resources:
14-
- overlays/recovery # ← ACTIVE: restore from Barman v5 → write v6 (nuke recovery)
15-
# - overlays/initdb # fresh DB / normal operation
14+
# - overlays/recovery # DR-only: restore from Barman v5 → write v6. Re-enable before a nuke.
15+
- overlays/initdb # ← ACTIVE (steady state; recovered 2026-06-28). No-op on existing cluster; writes v6.
1616
- externalsecret.yaml
1717
- scheduled-backup.yaml

infrastructure/database/cloudnative-pg/temporal/kustomization.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ commonLabels:
1111
commonAnnotations:
1212
argocd.argoproj.io/sync-wave: "-5"
1313
resources:
14-
- overlays/recovery # ← ACTIVE: restore from Barman v7 → write v8 (nuke recovery)
15-
# - overlays/initdb # steady state. bootstrap.initdb is a no-op on existing Cluster.
14+
# - overlays/recovery # DR-only: restore from Barman v7 → write v8. Re-enable before a nuke.
15+
- overlays/initdb # ← ACTIVE (steady state; recovered 2026-06-28). bootstrap.initdb is a no-op on existing Cluster; writes v8.
1616
- externalsecret.yaml
1717
- scheduled-backup.yaml

0 commit comments

Comments
 (0)