@@ -34,75 +34,65 @@ then produces a clean parking catalog for stage 2 to measure.
3434 durable catalog cannot reproduce and reports success. Compounded by #48 today,
3535 but independent: any failed/interrupted ` add ` (which ` checkpoint_catalog ` already
3636 anticipates, ` catalog_state.py:388-390 ` ) triggers it after #48 is fixed.
37- - ** #49 ** — authored state (charts, post_processing, …) stored as extra
38- ` catalog.yaml ` keys is dropped when xorq rebuilds ` catalog.yaml ` from
39- entries+aliases on a merge. ** Deprioritised out of stage 1's critical path:**
40- latent only under multi-user ` pull ` /sync, which does not happen single-user /
41- ` --no-sync ` . Stays filed; not fixed this stage.
37+ - ** #49 / #54 ** — authored state (charts, post_processing, stats, notebook, …)
38+ stored as ` catalog.yaml ` keys is dropped when xorq rebuilds ` catalog.yaml ` from
39+ entries+aliases on a merge. ** Resolution direction settled by #54 (and the #57
40+ precedent):** this state is shareable catalog content and * belongs* in
41+ ` catalog.yaml ` so it clones and versions with the catalog — #49 's proposed
42+ relocation to an out-of-repo store would stop that. So the fix is on the
43+ * durability* side (make the keys survive xorq's merge-rebuild — e.g. per-entry
44+ state into the entry metadata sidecar, which xorq treats as opaque), not
45+ relocation. ** Deprioritised out of stage 1's critical path:** the drop is latent
46+ only under multi-user ` pull ` /sync, which does not happen single-user /
47+ ` --no-sync ` . Stays filed (#49 , #54 ); not fixed this stage.
4248
4349### Work
4450
45- 1 . ** #48 fix — fold the bookkeeping into ` catalog.yaml ` (decided 2026-06-14).**
46- Stop writing ` aliases.json ` / ` alias_history.json ` as separate files; store
47- their content as tallyman keys in ` catalog.yaml ` , the same way every other
48- tallyman-owned thing already is (` post_processing ` , ` stats ` , ` charts ` ,
49- ` display_configs ` , ` notebook ` , ` entry_hashes ` , ` result_cache ` ,
50- ` compute_cache ` ). The two files are the lone anomaly that breaks #48 ; folding
51- them in removes them from the tracked file set, so ` assert_consistency ` (which
52- checks the file * set* , not ` catalog.yaml ` 's contents, catalog.py:768-785)
53- passes and ` xorq catalog add ` stops no-opping. xorq's add/add-alias
54- read-modify-write the whole dict and round-trip unknown keys on the ` --no-sync `
55- path tallyman always uses.
56- - ** Why this over a separate file/repo.** ` catalog.yaml ` is git-tracked and
57- checkpoint-committed, so ` git reset --hard ` already rolls alias state back —
58- no sibling repo, no lockstep commits, no bespoke reconciliation, and
59- ` test_reset_rolls_back_alias_state ` stays green via git. Strictly less code
60- than the alternatives.
61- - ** Known caveat (deferred, not regressed).** ` catalog.yaml ` keys are what xorq
62- drops when it rebuilds the file from entries+aliases on a merge-conflict
63- ` pull ` (#49 ). Latent under single-user / ` --no-sync ` , and the same exposure
64- the authored state already accepts. The multi-user fix is to move all
65- project-global state out of ` catalog.yaml ` together (#48 + #49 jointly) —
66- ` catalog.yaml ` -now defers that without foreclosing it.
67- - ** Optional refinement.** ` set_alias ` already calls ` _xcat.add_alias `
68- (` aliases.py:111 ` ), so xorq's own ` aliases ` key holds ` {alias: latest_hash} `
69- and ` aliases.json ` partly duplicates it. Folding only ` alias_history ` into a
70- tallyman key (and reading current aliases from xorq's native ` aliases ` )
71- shrinks tallyman bookkeeping and survives merges for the current-alias half.
72- - Audit the full set of non-xorq files the checkpoint commits under
73- ` catalog_dir ` , not just the two named, in case anything else needs the same
74- treatment.
75- - The deferred ` test_reset_rolls_back_alias_state ` guard (PR #50 ) ships * with*
76- this fix — the rollback path changes, so the guard travels with the change
77- per the TDD rule.
78-
79- 2 . ** #52 fix.** Add an explicit cross-view reconciliation + assertion at the end of
80- ` reset_to ` : compare the tracked ` entries/*.zip ` set against ` entry_hashes ` ,
81- re-register (or fail loudly on) any pointer whose recipe is missing, and assert
82- ` Catalog.from_kwargs(init=False) ` opens. Turns the silent divergence observable.
83-
84- 3 . ** #48 secondary — surface the silent swallow.** ` add_entry ` is best-effort and
85- discards its result (` xorq_catalog.py:63-68 ` , ` build.py ` ); expose a
86- ` BuildResult.catalog_registered ` -style signal, log failures at ERROR, and emit a
87- one-time companion warning. PR #50 's Group D pins this.
88-
89- 4 . ** Reproducible parking-catalog rebuild script.** No migration/repair of the
90- existing broken catalog — rebuild from scratch through the * fixed* machinery.
91- One script drives the parking sources through ` catalog_create ` /` catalog_revise `
92- to produce the canonical catalog, and emits both corpus variants for stage 2
93- (full ≈10M+ rows, truncated ≈1M). This replaces "repair" on the task list and
94- doubles as the stage-2 corpus generator. Depends on #48 + #52 being fixed first
95- (otherwise the rebuild is born broken again).
51+ 1 . ** ✅ DONE (PR #57 , merged) — #48 fix, alias bookkeeping in ` catalog.yaml ` .**
52+ ` aliases.py ` now reads/writes ` alias_map ` / ` alias_history ` as ` catalog.yaml `
53+ keys via ` catalog_state ` (no ` aliases.json ` , no out-of-repo sidecar — #53 's
54+ alternative was rejected). The tracked file set is back to xorq's expected set,
55+ ` assert_consistency ` passes, ` xorq catalog add ` stops no-opping. ` reset_to ` 's
56+ ` git reset ` rolls the alias keys back with the rest of ` catalog.yaml ` — no
57+ separate reconcile path. ** #48 closed.** ` test_reset_rolls_back_alias_state `
58+ shipped with the fix and guards the rollback; the non-xorq-fileset audit
59+ landed too (` _TRACKED ` is ` catalog.yaml ` only).
60+
61+ 2 . ** OPEN — #52 fix (the live stage-1 work item).** Not touched by #57 : that fix
62+ reconciles * alias state* , not the xorq ` .zip ` -vs-` entry_hashes ` divergence. Add
63+ an explicit cross-view reconciliation + assertion at the end of ` reset_to ` :
64+ compare the tracked ` entries/*.zip ` set against ` entry_hashes ` , re-register (or
65+ fail loudly on) any pointer whose recipe is missing, and assert
66+ ` Catalog.from_kwargs(init=False) ` opens. Turns the silent divergence
67+ observable. (PR #50 's Group B is the failing-test half, already on ` main ` .)
68+
69+ 3 . ** PARTLY DONE — surface the silent swallow.** #57 exposed
70+ ` BuildResult.catalog_registered ` and logs the failed entry-` add ` at ERROR. The
71+ remainder is ** #56 (open):** ` catalog_registered ` is computed but dies in
72+ ` _run_and_record ` (never propagated to the caller / companion), and ` set_alias `
73+ still discards ` add_alias ` 's bool, so a failed * alias* add stays swallowed.
74+ Finish #56 to actually surface both failures.
75+
76+ 4 . ** OPEN — reproducible parking-catalog rebuild script.** No migration/repair —
77+ rebuild from scratch through the now-fixed machinery. One script drives the
78+ parking sources through ` catalog_create ` /` catalog_revise ` to produce the
79+ canonical catalog and emits both corpus variants for stage 2 (full ≈10M+ rows,
80+ truncated ≈1M). Doubles as the stage-2 corpus generator. Now unblocked on the
81+ #48 side; still wants #52 fixed first so a back/forward rebuild stays durable.
82+ ** Blocking input:** confirm the parking source parquet(s) still exist / where.
9683
9784### Tests / acceptance
9885
99- - PR #50 's failing integration tests (Groups A/B/D) go green.
100- - One end-to-end demo-walk invariant test: genesis → multiple creates → revise →
101- reset back → forward → back, asserting at every step that the catalog opens,
102- every recipe is durable as a tracked ` .zip ` , alias state is correct, and (where
103- in scope) authored state survives.
104- - TDD split per house rule: failing tests (PR #50 ) seen red on CI first, then the
105- fixes; the alias-rollback guard ships with the fix commit.
86+ - ✅ PR #50 's failing integration tests (Groups A/B/D) landed on ` main `
87+ (` 80d2e9b ` ); #57 made the #48 ones (A, and D rewritten) pass and added Group F
88+ (alias registration durability). The TDD red-then-green split is satisfied for
89+ #48 .
90+ - ** Still red / unwritten:** Group B (reset divergence) stays failing until #52 is
91+ fixed — that's the next fix to land.
92+ - One end-to-end demo-walk invariant test still wanted: genesis → multiple creates
93+ → revise → reset back → forward → back, asserting at every step that the catalog
94+ opens, every recipe is durable as a tracked ` .zip ` , alias state is correct, and
95+ (where in scope) authored state survives. Lands with the #52 fix.
10696
10797---
10898
0 commit comments