Skip to content

Latest commit

 

History

History
76 lines (52 loc) · 5.18 KB

File metadata and controls

76 lines (52 loc) · 5.18 KB
name certifying-backups
description Use when backups exist but nobody has proven they restore, when a backup job should be verified on a schedule instead of trusted, or when you would otherwise discover a corrupt backup on the day you need it.

Certifying Backups

Overview

Restore one backup per iteration into a scratch location and prove it is usable: the archive opens, the data loads, and a checksum or row count matches expectations. The loop replaces the hope that backups work with a rolling certificate that they do.

Core principle: one backup certified per iteration, every iteration starts by checking the exit conditions and ends by writing the ledger.

When to Use

  • A nightly dump, snapshot, or archive job runs and is assumed to work
  • A compliance or disaster-recovery review needs evidence of restorability
  • After changing backup tooling, retention, or encryption settings

When NOT to use: production restore drills that need a maintenance window and a human on call, run those as planned exercises instead; backups you lack credentials to read, report the gap rather than looping on failures.

The Ledger

All state lives in .loop/certify-backups.md, called the ledger. Every entry records: backup id and date, source system, restore target path, verification method and its result (checksum, row count, or sample query), duration, and running counters: consecutive failures per source, backups certified this cycle.

Counter rules: a passing certification resets that source's consecutive-failure counter to zero. A cycle is one pass over every backup in scope; the certified-this-cycle counter resets when the success exit fires or when a new scheduled run starts.

Iteration 1 bootstrap: if the ledger does not exist, create .loop/ and the file, list the backups in scope (aws s3 ls s3://backups/db/ | tail -30 or the equivalent for your store), and write a baseline entry: backup inventory, newest certified id per source (none), each source's verification method and expected ranges as given in the loop instruction or the repo's backup docs, and all counters zero. If a source's verification method is given nowhere, stop and report it rather than inventing one. That is the whole first iteration, always; certification starts at iteration 2.

One Iteration

  1. Read the ledger (bootstrap it if missing).
  2. List backups newer than the newest certified id per source, plus the oldest backup still inside the retention window that has never been certified.
  3. Check EVERY exit condition in the table below against the fresh listing and the ledger counters. Any row matches: do its action and stop. This happens before any work.
  4. Nothing uncertified: write a status entry, end the iteration.
  5. Pick the FIRST uncertified backup in the listing, oldest first, ties broken by source name alphabetically (deterministic order beats clever prioritization). Restore it into a scratch path or scratch database that is wiped first. Never restore toward any production path.
  6. Verify by the method recorded for that source: compare the stored checksum, or load and count rows against the expected range, or run the source's sample query. Success requires the verification output, not just a completed restore command.
  7. Clean up the scratch location, then write the ledger entry: pass or fail, evidence, counters updated.

Exit Conditions (checked at step 3, every iteration)

Condition Action
Every backup in scope certified this cycle Summarize the certificate (ids, dates, methods), stop
Same source failed verification 2 consecutive times Stop, report: backups for that source are not restorable, with both failure outputs
A restore wrote outside the scratch path Stop immediately, report exactly what was written where; do not clean up
Budget reached (default 10 certifications per run) Stop, report what remains uncertified

Any stop-and-report lists every source with uncertified or failed backups, not just the one being worked.

Interval

Daily or weekly via a scheduler; certification follows the backup cadence, not a tight loop. Use /schedule or cron. A timed /loop only makes sense when burning down a backlog of never-certified backups in one sitting, and self-paced is better for that.

Common Mistakes

  • Certifying only the newest backup: the old one you will actually reach for during an incident stays unproven. The oldest-first pick order exists for this.
  • Treating a completed restore command as success: a restore that exits zero can still produce an empty database. Verification output is the certificate.
  • Restoring toward any path a production system reads: scratch means scratch, and the exit table treats a stray write as an emergency stop.
  • Letting counters live in your head: every counter the exit table needs must be in the ledger.

Example

/schedule weekly Certify this week's database backups using certifying-backups.
Budget: 10 certifications.

Ledger entry the loop should produce:

[2026-06-08 02:14] backup db-2026-06-07T01:00 (postgres/main), restored to
/scratch/certify/db-main in 4m12s. Verification: row count 1,482,330 within
expected 1.4M-1.6M, sample query ok. PASS. Counters: main failures 0,
certified this cycle 6/10.