fix(outcome): diagnose and repair completed-ledger digest chain breaks - #645
Conversation
A broken completed outcome ledger used to hard-block every capture with a raw traceback and no recovery path. Doctor now surfaces the first chain break, capture degrades to a bounded repair hint, and outcome repair quarantines the original ledger before re-chaining from the valid prefix. Co-authored-by: Cursor <cursoragent@cursor.com>
|
Important Review skippedAuto reviews are limited based on label configuration. 🏷️ Required labels (at least one) (1)
Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository: escoffier-labs/coderabbit/.coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Grader review: APPROVE-WITH-CHANGES - and one finding is a hard stop before this ever runs on the live ledger. The mechanics survive attack (healthy-ledger no-op under the append lock, O_EXCL quarantine keyed on content, correct crash ordering, byte-identical ledger after failed capture). Required changes:
Non-blocking: format_ledger_corrupt_error is dead code; add a directory fsync for the quarantine dir before the ledger rewrite (run_redaction does this); doctor rc=0 on corrupt prevents automation gating; the duplicate-writer heuristic requires payload equality minus digests so the live near-identical rows (differing ts) will report plain discontinuity; verify_run ignores capture's rc so degradation is stderr-only. Ping for re-grade; the live repair happens only after re-grade, operator-run, with the dropped-record count on screen. |
Co-authored-by: Codex <codex@openai.com>
|
Implemented in 4a8110f (
Verification passed: Ready for re-review. |
|
Grader: APPROVE, merging. The hard-stop finding is properly fixed: rather than the minimum (warn before mutating), the repair now re-signs self-consistent records after the first break and discloses salvaged counts and line ranges in the repair record, audit file, and both output modes. The four new tests cover exactly the shapes that were untested and that the live ledger actually exhibits: records after the break, a first-line break, multiple breaks, and binary bytes. Quarantine-first is restored for incomplete trailing bytes, and quarantine publication is now binary O_EXCL with directory fsync before the ledger rewrite, closing the Windows byte-fidelity gap. Receipt 20260801-202819-work-verify-19fcc2, CI green. Note for the live repair: this now preserves the ~633 post-break records on our corrupted ledger instead of discarding them. Run it operator-supervised with the salvaged count on screen. |
Summary
brigade outcome doctornow validates the completed ledger chain and reports the first break with line number, expected/actual digests, and suspected cause (including duplicate-writer).brigade outcome repair --operator-confirm: quarantine write-once under.brigade/outcome/repairs/, preserve the invalid segment, keep the valid prefix byte-identical, append a neutralledger-repairrecord, and re-verify.ledger corrupt at line N … run outcome repair) instead of a raw traceback, and write nothing while the chain is broken.Closes #639.
Verify receipt
20260801-034309-work-verify-12d572Command:
Acceptance criteria → tests
test_doctor_reports_completed_ledger_chain_breaktest_diagnose_duplicate_record_chain_breaktest_diagnose_truncated_line_breaktest_repair_requires_operator_confirmationtest_repair_quarantines_rechains_and_reverifytest_capture_degrades_with_bounded_error_and_writes_nothingtest_post_repair_capture_succeedstest_repair_of_truncated_line_recovers_via_lock_path_and_stays_healthyOperator note
Do not run repair against the live corrupted ledger from this PR worktree. After merge, repair the live checkout manually:
Test plan
brigade work verify run --target . --command "./scripts/verify" --capture brigade-worktests/test_outcome_repair.pygreenMade with Cursor