Skip to content

Commit 1a4a69d

Browse files
committed
fix(retention): apply CodeRabbit review fixes
- Apply post-review fixes to the retention cycle. - Order chain members topologically (base first, each child after its parent via ParentID), with Created only as a tie-breaker. Ordering by Created alone could place a child ahead of its base on tied or skewed timestamps, and the leaf-inward delete then removes the base before a surviving incremental — orphaning it. Stragglers from broken/cyclic links are appended, never dropped. - Stop deleting a chain's members at the first failure (return, not continue): once a leaf delete fails, deleting its ancestors would orphan it, so abort the chain and leave a complete shorter prefix. Other chains still proceed. - Make chain ordering deterministic: break equal newest() timestamps by Root so keep_last / GFS selection is reproducible (map iteration is randomized; a planner must not be). Same comparator used in Plan. - Reject negative retention.max_age in config validation: ParseDuration accepts "-1h", which would push the cutoff into the future and make the whole catalog deletable. - Reject negative CLI flag overrides before planning, mirroring the config-load validation that flags otherwise bypass. - Label a partially deleted chain "partially pruned", not "pruned". - Fix the command name in README and docs/RETENTION.md: it is "siphon dumps prune", not "siphon prune". - Add tests: topological member order on tied/skewed timestamps, deterministic chain order on tied timestamps, and that a chain's deletion stops (base untouched) when the leaf delete fails.
1 parent ce22097 commit 1a4a69d

8 files changed

Lines changed: 222 additions & 22 deletions

File tree

README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -181,7 +181,7 @@ storage:
181181

182182
Credentials are resolved from the standard AWS chain (env vars, `~/.aws`, instance role) — never stored in the config file. See [docs/STORAGE.md](docs/STORAGE.md) for details.
183183

184-
A `retention:` block (default + optional per-profile override) drives `siphon prune`, which deletes old backups as whole chains so an incremental is never orphaned:
184+
A `retention:` block (default + optional per-profile override) drives `siphon dumps prune`, which deletes old backups as whole chains so an incremental is never orphaned:
185185

186186
```yaml
187187
defaults:
@@ -191,7 +191,7 @@ defaults:
191191
gfs: { daily: 7, weekly: 4, monthly: 6 }
192192
```
193193

194-
`siphon prune` is dry-run by default; pass `--apply` to delete. Flags override the configured policy per run. See [docs/RETENTION.md](docs/RETENTION.md) for details.
194+
`siphon dumps prune` is dry-run by default; pass `--apply` to delete. Flags override the configured policy per run. See [docs/RETENTION.md](docs/RETENTION.md) for details.
195195

196196
## Architecture
197197

docs/RETENTION.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Retention & pruning
22

3-
`siphon prune` applies a retention policy to the dump catalog, deleting old
3+
`siphon dumps prune` applies a retention policy to the dump catalog, deleting old
44
backups while guaranteeing it never orphans an incremental from its base. It
55
works against any storage backend (local or S3), since it prunes through the
66
same `Store.Delete` the catalog already uses.
@@ -70,13 +70,13 @@ duration) fails fast at config load.
7070
7171
```bash
7272
# Dry-run (default): show which chains the policy would prune, for one profile.
73-
siphon prune --profile prod
73+
siphon dumps prune --profile prod
7474

7575
# Override the configured policy for this run, then actually delete.
76-
siphon prune --profile prod --keep-last 14 --apply
76+
siphon dumps prune --profile prod --keep-last 14 --apply
7777

7878
# Pure flag-driven policy (no config), GFS only.
79-
siphon prune --gfs-daily 7 --gfs-weekly 4 --gfs-monthly 6 --apply
79+
siphon dumps prune --gfs-daily 7 --gfs-weekly 4 --gfs-monthly 6 --apply
8080
```
8181

8282
`--apply` performs deletions; without it, prune only prints the plan. Flags

internal/app/prune.go

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -78,16 +78,19 @@ func Prune(ctx context.Context, d Deps, opt PruneOpts) (*PruneResult, error) {
7878
}
7979

8080
// applyChainDeletion deletes a pruned chain's members leaf-inward (incrementals
81-
// before the base). A per-dump failure is collected and does not abort the rest
82-
// of the chain or the run — a single stuck object should not block reclaiming
83-
// the others.
81+
// before the base) and STOPS at the first failure within the chain. Stopping is
82+
// the invariant: if a leaf delete fails (its meta may be gone but the dump still
83+
// catalogued), continuing to delete its ancestors would orphan that leaf —
84+
// exactly what the chain-aware design prevents. Aborting the chain leaves a
85+
// complete, still-restorable prefix. Other chains are unaffected; the failure is
86+
// recorded and the run continues with them.
8487
func applyChainDeletion(ctx context.Context, d Deps, ch dumps.Chain, oc *ChainOutcome, result *PruneResult) {
8588
for i := len(ch.Members) - 1; i >= 0; i-- {
8689
m := ch.Members[i]
8790
if err := d.Dumps.Delete(ctx, m.ID); err != nil {
8891
oc.Errors = append(oc.Errors, m.ID+": "+err.Error())
8992
result.Failed++
90-
continue
93+
return // do not delete this member's ancestors — would orphan it
9194
}
9295
oc.Deleted = append(oc.Deleted, m.ID)
9396
result.Reclaimed += m.SizeBytes

internal/app/prune_test.go

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -193,6 +193,36 @@ func TestPrune_CollectsDeletionFailures(t *testing.T) {
193193
}
194194
}
195195

196+
// TestPrune_StopsChainOnFirstFailure proves the orphan-prevention invariant:
197+
// when deleting the leaf incremental of a pruned chain fails, the prune must NOT
198+
// proceed to delete the base — otherwise the surviving leaf is orphaned.
199+
func TestPrune_StopsChainOnFirstFailure(t *testing.T) {
200+
store := newRecordingStore()
201+
cat := dumps.New(store)
202+
old := time.Now().AddDate(0, 0, -40)
203+
seedDump(t, cat, "base", "p", old, "base", "")
204+
seedDump(t, cat, "inc", "p", old.Add(time.Hour), "base", "base")
205+
// The leaf (inc) is deleted first (leaf-inward). Make its meta delete fail —
206+
// Catalog.Delete removes meta first, so this is the first Delete call.
207+
store.failKeys["inc.meta.json"] = true
208+
209+
res, err := Prune(context.Background(), pruneDeps(store), PruneOpts{
210+
Policy: dumps.RetentionPolicy{MaxAge: time.Hour}, Apply: true,
211+
})
212+
if err != nil {
213+
t.Fatalf("Prune: %v", err)
214+
}
215+
if res.Failed != 1 {
216+
t.Errorf("Failed = %d, want 1", res.Failed)
217+
}
218+
// The base must NOT have been deleted — neither its meta nor its body.
219+
for _, k := range store.deletes {
220+
if strings.HasPrefix(k, "base") {
221+
t.Errorf("base was deleted after leaf failure (orphans the leaf): deletes=%v", store.deletes)
222+
}
223+
}
224+
}
225+
196226
// helpers
197227

198228
func firstIndexWithPrefix(s []string, p string) int {

internal/cli/dumps.go

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -100,6 +100,13 @@ func dumpsPruneCmd() *cobra.Command {
100100
if gfsMSet {
101101
policy.GFS.Monthly = gfsMonthly
102102
}
103+
// Config-backed policy is validated at load; flag overrides bypass that,
104+
// so reject negatives here too before a destructive run sees an
105+
// undefined policy.
106+
if policy.KeepLast < 0 || policy.MaxAge < 0 ||
107+
policy.GFS.Daily < 0 || policy.GFS.Weekly < 0 || policy.GFS.Monthly < 0 {
108+
return &errs.Error{Op: "prune", Code: errs.CodeUser, Hint: "retention values must be non-negative"}
109+
}
103110

104111
deps, err := buildDeps()
105112
if err != nil {
@@ -169,7 +176,13 @@ func renderPruneResult(w io.Writer, res *app.PruneResult) {
169176
continue
170177
}
171178
prunedChains++
172-
_, _ = fmt.Fprintf(w, "%s chain %s (%d dump(s), %d bytes)\n", verb, oc.Root, len(oc.DumpIDs), oc.SizeBytes)
179+
// On apply, a chain with errors was only partially deleted (the delete
180+
// aborted leaf-inward at the failure), so don't label it fully "pruned".
181+
label := verb
182+
if res.Apply && len(oc.Errors) > 0 {
183+
label = "partially pruned"
184+
}
185+
_, _ = fmt.Fprintf(w, "%s chain %s (%d dump(s), %d bytes)\n", label, oc.Root, len(oc.DumpIDs), oc.SizeBytes)
173186
for _, e := range oc.Errors {
174187
_, _ = fmt.Fprintf(w, " ! %s\n", e)
175188
}

internal/config/config.go

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -59,9 +59,16 @@ func (r *RetentionConfig) Validate() error {
5959
return errors.New("retention.gfs tiers must be >= 0")
6060
}
6161
if r.MaxAge != "" {
62-
if _, err := time.ParseDuration(r.MaxAge); err != nil {
62+
d, err := time.ParseDuration(r.MaxAge)
63+
if err != nil {
6364
return fmt.Errorf("retention.max_age %q is not a valid duration: %w", r.MaxAge, err)
6465
}
66+
// time.ParseDuration accepts signed inputs ("-1h"); a negative max-age
67+
// would push the cutoff into the future and make the whole catalog
68+
// eligible for deletion, so reject it.
69+
if d < 0 {
70+
return fmt.Errorf("retention.max_age must be >= 0, got %q", r.MaxAge)
71+
}
6572
}
6673
return nil
6774
}

internal/dumps/retention.go

Lines changed: 92 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -66,8 +66,14 @@ type RetentionPlan struct {
6666
// backup (BaseID == ID, or legacy empty BaseID) is a singleton chain; its
6767
// incrementals attach to it via their BaseID. A dump whose root is missing from
6868
// the set anchors its own chain rather than being dropped, so a partially
69-
// present catalog never loses entries silently. Within each chain, members are
70-
// ordered by Created (base first).
69+
// present catalog never loses entries silently.
70+
//
71+
// Within each chain, members are ordered by ParentID/BaseID TOPOLOGY (base
72+
// first, then each child after its parent), with Created only as a tie-breaker.
73+
// This is the contract the leaf-inward delete path relies on: deleting members
74+
// last-to-first must remove every descendant before its ancestor. Ordering by
75+
// Created alone would break that on tied or skewed timestamps — a child could
76+
// sort ahead of its parent and the base could be deleted while a leaf survives.
7177
func GroupChains(dumps []Meta) []Chain {
7278
byRoot := map[string][]Meta{}
7379
for _, m := range dumps {
@@ -79,18 +85,88 @@ func GroupChains(dumps []Meta) []Chain {
7985
}
8086
chains := make([]Chain, 0, len(byRoot))
8187
for root, members := range byRoot {
82-
sort.SliceStable(members, func(i, j int) bool {
83-
return members[i].Created.Before(members[j].Created)
84-
})
85-
chains = append(chains, Chain{Root: root, Members: members})
88+
chains = append(chains, Chain{Root: root, Members: orderMembers(root, members)})
8689
}
87-
// Deterministic order: newest chain (by newest member) first.
90+
// Deterministic order: newest chain first, Root breaking ties so the plan is
91+
// reproducible across runs even when timestamps collide (map iteration is
92+
// randomized; a destructive planner must not be).
8893
sort.SliceStable(chains, func(i, j int) bool {
89-
return chains[i].newest().After(chains[j].newest())
94+
ti, tj := chains[i].newest(), chains[j].newest()
95+
if ti.Equal(tj) {
96+
return chains[i].Root < chains[j].Root
97+
}
98+
return ti.After(tj)
9099
})
91100
return chains
92101
}
93102

103+
// orderMembers returns a chain's members in topological apply order: the base
104+
// (root) first, then each incremental placed immediately it can be (its parent
105+
// already emitted). Created breaks ties among siblings and gives a stable order.
106+
// Any members left unplaceable by broken/cyclic ParentID links are appended in
107+
// Created order so nothing is dropped — a corrupt chain still surfaces all its
108+
// dumps for the caller to handle.
109+
func orderMembers(root string, members []Meta) []Meta {
110+
// Stable Created order first, so ties resolve deterministically.
111+
sort.SliceStable(members, func(i, j int) bool {
112+
if members[i].Created.Equal(members[j].Created) {
113+
return members[i].ID < members[j].ID
114+
}
115+
return members[i].Created.Before(members[j].Created)
116+
})
117+
118+
byID := make(map[string]Meta, len(members))
119+
for _, m := range members {
120+
byID[m.ID] = m
121+
}
122+
123+
ordered := make([]Meta, 0, len(members))
124+
emitted := make(map[string]bool, len(members))
125+
126+
// Emit the base first if present (its ID == root, or it self-roots / is a
127+
// legacy empty-BaseID full backup).
128+
if base, ok := byID[root]; ok {
129+
ordered = append(ordered, base)
130+
emitted[base.ID] = true
131+
}
132+
133+
// attachable reports whether m can be emitted now: its parent is already
134+
// emitted, or its parent is not part of this chain (absent/self/empty), in
135+
// which case it attaches at the root level. The base itself is handled above.
136+
attachable := func(m Meta) bool {
137+
if m.ID == root {
138+
return false // already emitted (or will be appended as a straggler)
139+
}
140+
p := m.ParentID
141+
return p == "" || p == m.ID || emitted[p] || byID[p].ID == ""
142+
}
143+
144+
// Iterate to a fixed point; members are Created-sorted, so each pass emits the
145+
// earliest now-attachable child.
146+
for {
147+
progressed := false
148+
for _, m := range members {
149+
if emitted[m.ID] || !attachable(m) {
150+
continue
151+
}
152+
ordered = append(ordered, m)
153+
emitted[m.ID] = true
154+
progressed = true
155+
}
156+
if !progressed {
157+
break
158+
}
159+
}
160+
161+
// Append any stragglers (cycles) in Created order — never drop a member.
162+
for _, m := range members {
163+
if !emitted[m.ID] {
164+
ordered = append(ordered, m)
165+
}
166+
}
167+
return ordered
168+
}
169+
94170
// Plan decides which chains to keep vs prune under p, as of now. It is pure: no
95171
// I/O, no clock — now is injected — so every rule and edge case is unit-testable
96172
// with synthetic fixtures. An empty policy keeps everything.
@@ -99,10 +175,16 @@ func Plan(chains []Chain, p RetentionPolicy, now time.Time) RetentionPlan {
99175
return RetentionPlan{Keep: append([]Chain(nil), chains...)}
100176
}
101177

102-
// chains is already sorted newest-first by GroupChains; copy defensively.
178+
// chains is already sorted newest-first by GroupChains; copy and re-sort with
179+
// the SAME deterministic comparator (Root breaks ties) so callers that build
180+
// a Chain slice by hand still get reproducible keep_last / GFS selection.
103181
ordered := append([]Chain(nil), chains...)
104182
sort.SliceStable(ordered, func(i, j int) bool {
105-
return ordered[i].newest().After(ordered[j].newest())
183+
ti, tj := ordered[i].newest(), ordered[j].newest()
184+
if ti.Equal(tj) {
185+
return ordered[i].Root < ordered[j].Root
186+
}
187+
return ti.After(tj)
106188
})
107189

108190
keep := map[string]bool{} // union of every active rule's keep set

internal/dumps/retention_test.go

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,71 @@ func TestGroupChains(t *testing.T) {
5858
t.Fatalf("orphan grouping = %+v, want one chain rooted at missingbase", chains)
5959
}
6060
})
61+
62+
t.Run("members ordered by topology, not Created, on tied timestamps", func(t *testing.T) {
63+
// Base and incremental share the SAME Created time (clock skew / coarse
64+
// granularity). Topological order MUST still put the base first, because
65+
// the leaf-inward delete path deletes last-to-first and would otherwise
66+
// remove the base before its surviving child.
67+
same := daysAgo(2)
68+
ds := []Meta{
69+
dump("inc", "p", same, "base", "base"), // listed first, same timestamp
70+
dump("base", "p", same, "base", ""),
71+
}
72+
chains := GroupChains(ds)
73+
if len(chains) != 1 {
74+
t.Fatalf("got %d chains, want 1", len(chains))
75+
}
76+
got := chains[0].Members
77+
if len(got) != 2 || got[0].ID != "base" || got[1].ID != "inc" {
78+
t.Errorf("member order = [%s %s], want [base inc] (base first despite tie)", got[0].ID, got[1].ID)
79+
}
80+
})
81+
82+
t.Run("members topologically ordered when a child predates its parent", func(t *testing.T) {
83+
// Pathological: the incremental's Created is BEFORE the base's (skew).
84+
// Topology must still win — base first.
85+
ds := []Meta{
86+
dump("base", "p", daysAgo(1), "base", ""),
87+
dump("inc", "p", daysAgo(5), "base", "base"), // older than its base
88+
}
89+
chains := GroupChains(ds)
90+
got := chains[0].Members
91+
if got[0].ID != "base" {
92+
t.Errorf("member order = [%s ...], want base first despite older child", got[0].ID)
93+
}
94+
})
95+
}
96+
97+
func TestGroupChains_DeterministicOnTiedChainTimestamps(t *testing.T) {
98+
// Two independent chains with identical newest() timestamps must sort in a
99+
// stable order (by Root) across runs — a destructive planner cannot depend on
100+
// randomized map iteration. Run the grouping repeatedly and assert identical
101+
// output.
102+
same := daysAgo(3)
103+
ds := []Meta{
104+
dump("zzz", "p", same, "", ""),
105+
dump("aaa", "p", same, "", ""),
106+
dump("mmm", "p", same, "", ""),
107+
}
108+
first := roots(GroupChains(ds))
109+
for i := 0; i < 20; i++ {
110+
if got := roots(GroupChains(ds)); !equalStrings(got, first) {
111+
t.Fatalf("non-deterministic chain order: run %d = %v, first = %v", i, got, first)
112+
}
113+
}
114+
}
115+
116+
func equalStrings(a, b []string) bool {
117+
if len(a) != len(b) {
118+
return false
119+
}
120+
for i := range a {
121+
if a[i] != b[i] {
122+
return false
123+
}
124+
}
125+
return true
61126
}
62127

63128
func TestPlan_EmptyPolicyKeepsEverything(t *testing.T) {

0 commit comments

Comments
 (0)