Problem
reset_to reconciles two different views of "what entries exist" with two
different mechanisms that are never checked against each other
(src/tallyman_core/catalog_state.py:413-429):
- The git-tracked xorq catalog (
entries/<hash>.zip +
metadata/<hash>.zip.metadata.yaml, plus catalog.yaml's entries/aliases)
is reconciled solely by git reset --hard <commit> (:422). git can only
restore a recipe zip that was actually committed.
- The untracked tallyman artifacts (the
entries/<hash>/ build dirs,
result_cache, compute_cache) are reconciled against the entry_hashes /
result_cache / compute_cache pointer lists in catalog.yaml via the
bullpen: prune_entries (:248), prune_result_cache/prune_compute_cache
(:280-285) retire to the bullpen, restore_from_bullpen (:288-314) copies
back.
reset_to runs materialize → prune_* → restore_from_bullpen and never
re-validates the git-tracked xorq recipe set against the tallyman pointers, nor
re-runs xorq catalog add for an entry whose pointer is present but whose .zip
is not. So when the recipe set is incomplete, the tallyman view (restored by
copy from the bullpen) claims entries that the durable xorq catalog cannot
reproduce, and reset returns success.
Trace from the real demo flow (genesis → catalog_create → catalog_create → reset_to), driven through the MCP tools so the checkpoint wrapper fires as in
production:
[after both] dirs=[023fb..,49b47..] tracked_zips=[49b47..zip] head=2
[reset->s1] dirs=[49b47..] tracked_zips=[49b47..zip] head=1
[forward->s2] dirs=[023fb..,49b47..] tracked_zips=[49b47..zip] head=2
After back→forward the tallyman view shows two entries; the durable xorq catalog
still holds one zip. The 023fb recipe round-trips through the bullpen as a
build-dir copy but is never regained as a committed recipe.
Impact
The bullpen masks the loss instead of surfacing it: the tallyman entry comes
back, so reset looks correct, while "git-backed, recompute from history" is
quietly false for that entry — its only copy is the evictable bullpen dir, and
the content-addressed .zip it should recompute from does not exist. The
recovery model the bullpen exists to support is the thing being undermined.
This is most often triggered today by #48 (every xorq catalog add after the
first silently no-ops, so the zips were never committed). It is a distinct cause,
not the same bug: even with #48 fixed, checkpoint_catalog already anticipates a
commit that does not land (returns None on index.lock contention with xorq's
own add, :388-390) and add_entry is best-effort (xorq_catalog.py:63-68).
Any interrupted or failed add leaves a dir + pointer with no zip, and
reset_to will mask it the same way, because the missing-cross-view-check is the
real gap.
Suggested fix
Give reset_to an explicit reconciliation between the git-tracked xorq catalog
and the tallyman pointers after git reset --hard, before it reports success:
compare the tracked entries/*.zip set against entry_hashes, and for any
pointer whose recipe is missing, either re-register it (xorq catalog add from
the restored build dir) or fail loudly rather than returning a catalog whose two
views disagree. A consistency assertion at the end of reset_to (tracked-zip set
== entry_hashes, and Catalog.from_kwargs(init=False) opens) would turn the
silent divergence into an observable error.
Context
Found while writing the failing integration tests in #50 (catalog↔xorq
coexistence and reset round-trips), main @ a18b14f, xorq 0.3.29 (1f46392e).
Related to #48 (recipes never committed) and #49 (authored state dropped on
merge) — all three are tallyman and the xorq catalog disagreeing about what is
durable — but a distinct cause: reset-time reconciliation has no cross-view
check here, vs consistency-check rejection (#48) and merge-time reconstruction
(#49). Group B of #50 pins this end to end.
Problem
reset_toreconciles two different views of "what entries exist" with twodifferent mechanisms that are never checked against each other
(
src/tallyman_core/catalog_state.py:413-429):entries/<hash>.zip+metadata/<hash>.zip.metadata.yaml, pluscatalog.yaml'sentries/aliases)is reconciled solely by
git reset --hard <commit>(:422). git can onlyrestore a recipe zip that was actually committed.
entries/<hash>/build dirs,result_cache,compute_cache) are reconciled against theentry_hashes/result_cache/compute_cachepointer lists incatalog.yamlvia thebullpen:
prune_entries(:248),prune_result_cache/prune_compute_cache(
:280-285) retire to the bullpen,restore_from_bullpen(:288-314) copiesback.
reset_torunsmaterialize → prune_* → restore_from_bullpenand neverre-validates the git-tracked xorq recipe set against the tallyman pointers, nor
re-runs
xorq catalog addfor an entry whose pointer is present but whose.zipis not. So when the recipe set is incomplete, the tallyman view (restored by
copy from the bullpen) claims entries that the durable xorq catalog cannot
reproduce, and reset returns success.
Trace from the real demo flow (
genesis → catalog_create → catalog_create → reset_to), driven through the MCP tools so the checkpoint wrapper fires as inproduction:
After back→forward the tallyman view shows two entries; the durable xorq catalog
still holds one zip. The
023fbrecipe round-trips through the bullpen as abuild-dir copy but is never regained as a committed recipe.
Impact
The bullpen masks the loss instead of surfacing it: the tallyman entry comes
back, so reset looks correct, while "git-backed, recompute from history" is
quietly false for that entry — its only copy is the evictable bullpen dir, and
the content-addressed
.zipit should recompute from does not exist. Therecovery model the bullpen exists to support is the thing being undermined.
This is most often triggered today by #48 (every
xorq catalog addafter thefirst silently no-ops, so the zips were never committed). It is a distinct cause,
not the same bug: even with #48 fixed,
checkpoint_catalogalready anticipates acommit that does not land (returns
Noneon index.lock contention with xorq'sown add,
:388-390) andadd_entryis best-effort (xorq_catalog.py:63-68).Any interrupted or failed
addleaves a dir + pointer with no zip, andreset_towill mask it the same way, because the missing-cross-view-check is thereal gap.
Suggested fix
Give
reset_toan explicit reconciliation between the git-tracked xorq catalogand the tallyman pointers after
git reset --hard, before it reports success:compare the tracked
entries/*.zipset againstentry_hashes, and for anypointer whose recipe is missing, either re-register it (
xorq catalog addfromthe restored build dir) or fail loudly rather than returning a catalog whose two
views disagree. A consistency assertion at the end of
reset_to(tracked-zip set==
entry_hashes, andCatalog.from_kwargs(init=False)opens) would turn thesilent divergence into an observable error.
Context
Found while writing the failing integration tests in #50 (catalog↔xorq
coexistence and reset round-trips), main @ a18b14f, xorq 0.3.29 (1f46392e).
Related to #48 (recipes never committed) and #49 (authored state dropped on
merge) — all three are tallyman and the xorq catalog disagreeing about what is
durable — but a distinct cause: reset-time reconciliation has no cross-view
check here, vs consistency-check rejection (#48) and merge-time reconstruction
(#49). Group B of #50 pins this end to end.