Skip to content

test: doc-driven edge-case tests for the reactive system and caching#160

Open
paddymul wants to merge 3 commits into
mainfrom
test/reactive-caching-doc-edges
Open

test: doc-driven edge-case tests for the reactive system and caching#160
paddymul wants to merge 3 commits into
mainfrom
test/reactive-caching-doc-edges

Conversation

@paddymul

@paddymul paddymul commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

What

23 new tests exercising doc-promised edges of the reactive system (staleness axes, recalc cone, auto-recalc) and the caching stack. They were written from docs/reactive-recalc.md, docs/caching.md, docs/architecture.md, docs/expression-lifecycle.md, and docs/mcp-server.md first, then reconciled against the API only — the point is to pin what the docs promise, not what the code happens to do.

20 pass. 3 fail, each documenting a real doc-vs-code hole (failing tests pushed first per the TDD workflow; whether each fix is code-side or doc-side is a design call):

Failing tests (the findings)

  1. test_recalc.py::test_explicit_husk_root_is_dropped_from_the_conerecalc: superseded historical entries treated as live — cascade replays them (manufactured junk entries) and orphan-classifier flags them "UNEXPLAINED / file a bug" #154 head-gated the scan-driven root set and cone membership, but an explicitly-passed husk root bypasses the gate: recalc(project, [husk], dry_run=False) replays the superseded version. Under source drift the replay mints a junk sibling entry of the live head — the same junk-replay shape recalc: superseded historical entries treated as live — cascade replays them (manufactured junk entries) and orphan-classifier flags them "UNEXPLAINED / file a bug" #154 fixed on the scan-driven path. mcp-server.md says "Roots naming no live entry are silently dropped"; the scan's own live=False verdict for the husk says it should be.

  2. test_recalc.py::test_scratch_follower_rebuilds_with_remap_but_no_alias_revision — reactive-recalc.md: a rebuilt entry with no alias ("an unnamed scratch entry") "produces a new hash and a remap entry but zero alias revisions". In practice the head-gate marks a never-aliased catalog_run follower live=False, so it is never actionably stale (stale=False even with the alias reason recorded) and never recalcs. Either the gate should treat never-aliased scratch entries as live, or the doc's scratch-rebuild semantics are dead and should say so.

  3. test_recalc.py::test_alias_pin_is_not_silently_advanced_by_a_replay — the docs promise a pin (by alias or hash) means the child "stays on that exact revision" and is "never disturbed by recalc". True when the pin is the child's only edge; but a child with one tracked parent and one alias-pinned parent gets replayed when the tracked parent advances, and the replay re-runs pinned_expr_from_alias("p"), which re-resolves the name to p's new head — the pin silently advances. Hash pins are immune. Fix is either replay-time pin freezing or a doc caveat ("pin by hash if you need replay durability").

Passing tests worth a note

  • test_revise_carry_over.py::test_cascade_carries_chart_to_rebuilt_follower — cascade carry-forward of chart/display config works but is undocumented (caching.md only names catalog_revise/put_code as carry-forward callers).
  • test_staleness.py::test_inplace_edit_preserving_mtime_and_size_is_detected — the "forces a faithful re-read" claim holds even when the edit preserves mtime and byte length (the stat-keyed memo's blind spot).
  • test_staleness.py::test_cheap_parent_sources_inline_into_child_and_both_go_directly_stale / test_expensive_parent_shields_child_from_source_axis — the cheap-inline vs expensive-shield source-axis behavior, including the doc's checkable invariant (child sources ⊇ cheap parent sources).
  • test_recalc.py::test_diamond_join_child_rebuilds_once_against_both_new_parents / test_diamond_failure_skips_the_join_child — diamond replay: join child rebuilds once against both advanced heads; stop-and-report skips it when an arm fails.
  • test_recalc.py::test_empty_roots_list_recomputes_nothingroots=[] is an explicit empty selection, not a falsy fallback to the stale set.
  • test_source_identity.py::test_salt_mode_bakes_no_snapshot_and_reads_still_work / test_salt_rebuild_over_edited_source_forks_hash — first coverage of salt mode's documented behavior.
  • test_auto_recalc.py::test_revise_identical_code_keeps_version_and_cascades_nothing — byte-identical revise dedupes: same hash, still version 1, empty cascade.

Doc inconsistencies found during the review (no test possible / already covered)

🤖 Generated with Claude Code

paddymul and others added 3 commits July 1, 2026 16:23
Twenty new tests written from docs/reactive-recalc.md, caching.md,
architecture.md, expression-lifecycle.md, and mcp-server.md, covering
edges the suite did not exercise:

- staleness: alias-form pins, deleted alias/source inputs (unknown_axes),
  alias recreated at a new head, both axes stale at once, cheap-parent
  source inlining vs expensive-parent shielding (incl. the sources-union
  invariant), and an in-place edit that preserves mtime and byte length
  (the faithful re-read claim)
- recalc: diamond replay (join child rebuilds once against both advanced
  heads), stop-and-report through a diamond, explicit roots on
  transitively-stale or fresh entries, roots=[] as an explicit empty
  selection, two aliases on one head advancing together, replay after a
  followed alias was deleted
- auto-recalc: a byte-identical revise dedupes (same hash, same version,
  empty cascade); cascade carry-forward of chart/display config to
  rebuilt followers (works, but undocumented)
- caching: salt mode bakes no snapshot and reads recompute; salt folds
  source digests into the hash; tracked_expr_from_alias rejects a hash

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Each of these asserts behavior the docs promise and the code does not
deliver; they are expected to fail on CI until the doc-or-code call is
made for each:

1. test_explicit_husk_root_is_dropped_from_the_cone — #154 head-gated
   the scan-driven root set and cone membership, but an explicitly
   passed husk root bypasses the gate and replays; under source drift
   the replay mints a junk sibling of the live head. mcp-server.md says
   roots naming no live entry are silently dropped.

2. test_scratch_follower_rebuilds_with_remap_but_no_alias_revision —
   reactive-recalc.md says an unnamed scratch entry rebuilds with a
   remap entry and zero alias revisions, but the head-gate marks a
   never-aliased catalog_run follower live=False, so it is never
   actionably stale and never recalcs.

3. test_alias_pin_is_not_silently_advanced_by_a_replay — the docs say a
   pin (by alias or hash) is never disturbed by recalc, but when a
   child's OTHER (tracked) parent forces a replay, the recipe's
   pinned_expr_from_alias(name) re-resolves the name to the pinned
   alias's new head. Hash pins are immune; alias pins silently advance.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Each fix is code-side, landing the behavior the docs already promise.

1. Explicit husk root is dropped from the cone (#154 extension).
   `_live_cone` exempted explicitly-named roots from the husk gate, so
   `recalc(project, [husk])` replayed a superseded version and, under
   source drift, minted a junk sibling of the live head. Drop the
   exemption: a husk (`live=False`) is dropped whether dragged in by the
   cone expansion or named explicitly, per mcp-server.md ("roots naming
   no live entry are silently dropped"). The scan's own `live=False`
   verdict is the authority.

2. Scratch followers rebuild again. `scan` gated `live` on "is a current
   alias head", which also excluded never-aliased `catalog_run` scratch
   followers, so an advanced parent never made them actionably stale.
   Redefine the gate as "is a superseded husk" (appears in some alias
   history but is no longer its head); a scratch entry appears in no
   history, so it stays live and rebuilds like any follower — minus the
   alias bookkeeping — matching reactive-recalc.md's scratch semantics.

3. Alias pins survive a replay. `pinned_expr_from_alias("p")` stores the
   alias name, so a replay forced by a *tracked* parent advancing
   re-resolved the name to p's new head and silently advanced the pin.
   Freeze alias-pinned parents to the revision recorded in
   `manifest.parents` for the duration of the replay (new `io.pin_freeze`
   context, set by `_replay_cone` from `_pinned_alias_freeze`), so the
   pin stays on its exact revision as the docs promise. Hash pins were
   already durable and are untouched.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant