test: doc-driven edge-case tests for the reactive system and caching#160
Open
paddymul wants to merge 3 commits into
Open
test: doc-driven edge-case tests for the reactive system and caching#160paddymul wants to merge 3 commits into
paddymul wants to merge 3 commits into
Conversation
Twenty new tests written from docs/reactive-recalc.md, caching.md, architecture.md, expression-lifecycle.md, and mcp-server.md, covering edges the suite did not exercise: - staleness: alias-form pins, deleted alias/source inputs (unknown_axes), alias recreated at a new head, both axes stale at once, cheap-parent source inlining vs expensive-parent shielding (incl. the sources-union invariant), and an in-place edit that preserves mtime and byte length (the faithful re-read claim) - recalc: diamond replay (join child rebuilds once against both advanced heads), stop-and-report through a diamond, explicit roots on transitively-stale or fresh entries, roots=[] as an explicit empty selection, two aliases on one head advancing together, replay after a followed alias was deleted - auto-recalc: a byte-identical revise dedupes (same hash, same version, empty cascade); cascade carry-forward of chart/display config to rebuilt followers (works, but undocumented) - caching: salt mode bakes no snapshot and reads recompute; salt folds source digests into the hash; tracked_expr_from_alias rejects a hash Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Each of these asserts behavior the docs promise and the code does not deliver; they are expected to fail on CI until the doc-or-code call is made for each: 1. test_explicit_husk_root_is_dropped_from_the_cone — #154 head-gated the scan-driven root set and cone membership, but an explicitly passed husk root bypasses the gate and replays; under source drift the replay mints a junk sibling of the live head. mcp-server.md says roots naming no live entry are silently dropped. 2. test_scratch_follower_rebuilds_with_remap_but_no_alias_revision — reactive-recalc.md says an unnamed scratch entry rebuilds with a remap entry and zero alias revisions, but the head-gate marks a never-aliased catalog_run follower live=False, so it is never actionably stale and never recalcs. 3. test_alias_pin_is_not_silently_advanced_by_a_replay — the docs say a pin (by alias or hash) is never disturbed by recalc, but when a child's OTHER (tracked) parent forces a replay, the recipe's pinned_expr_from_alias(name) re-resolves the name to the pinned alias's new head. Hash pins are immune; alias pins silently advance. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Each fix is code-side, landing the behavior the docs already promise. 1. Explicit husk root is dropped from the cone (#154 extension). `_live_cone` exempted explicitly-named roots from the husk gate, so `recalc(project, [husk])` replayed a superseded version and, under source drift, minted a junk sibling of the live head. Drop the exemption: a husk (`live=False`) is dropped whether dragged in by the cone expansion or named explicitly, per mcp-server.md ("roots naming no live entry are silently dropped"). The scan's own `live=False` verdict is the authority. 2. Scratch followers rebuild again. `scan` gated `live` on "is a current alias head", which also excluded never-aliased `catalog_run` scratch followers, so an advanced parent never made them actionably stale. Redefine the gate as "is a superseded husk" (appears in some alias history but is no longer its head); a scratch entry appears in no history, so it stays live and rebuilds like any follower — minus the alias bookkeeping — matching reactive-recalc.md's scratch semantics. 3. Alias pins survive a replay. `pinned_expr_from_alias("p")` stores the alias name, so a replay forced by a *tracked* parent advancing re-resolved the name to p's new head and silently advanced the pin. Freeze alias-pinned parents to the revision recorded in `manifest.parents` for the duration of the replay (new `io.pin_freeze` context, set by `_replay_cone` from `_pinned_alias_freeze`), so the pin stays on its exact revision as the docs promise. Hash pins were already durable and are untouched. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
23 new tests exercising doc-promised edges of the reactive system (staleness axes, recalc cone, auto-recalc) and the caching stack. They were written from
docs/reactive-recalc.md,docs/caching.md,docs/architecture.md,docs/expression-lifecycle.md, anddocs/mcp-server.mdfirst, then reconciled against the API only — the point is to pin what the docs promise, not what the code happens to do.20 pass. 3 fail, each documenting a real doc-vs-code hole (failing tests pushed first per the TDD workflow; whether each fix is code-side or doc-side is a design call):
Failing tests (the findings)
test_recalc.py::test_explicit_husk_root_is_dropped_from_the_cone— recalc: superseded historical entries treated as live — cascade replays them (manufactured junk entries) and orphan-classifier flags them "UNEXPLAINED / file a bug" #154 head-gated the scan-driven root set and cone membership, but an explicitly-passed husk root bypasses the gate:recalc(project, [husk], dry_run=False)replays the superseded version. Under source drift the replay mints a junk sibling entry of the live head — the same junk-replay shape recalc: superseded historical entries treated as live — cascade replays them (manufactured junk entries) and orphan-classifier flags them "UNEXPLAINED / file a bug" #154 fixed on the scan-driven path. mcp-server.md says "Roots naming no live entry are silently dropped"; the scan's ownlive=Falseverdict for the husk says it should be.test_recalc.py::test_scratch_follower_rebuilds_with_remap_but_no_alias_revision— reactive-recalc.md: a rebuilt entry with no alias ("an unnamed scratch entry") "produces a new hash and aremapentry but zero alias revisions". In practice the head-gate marks a never-aliasedcatalog_runfollowerlive=False, so it is never actionably stale (stale=Falseeven with the alias reason recorded) and never recalcs. Either the gate should treat never-aliased scratch entries as live, or the doc's scratch-rebuild semantics are dead and should say so.test_recalc.py::test_alias_pin_is_not_silently_advanced_by_a_replay— the docs promise a pin (by alias or hash) means the child "stays on that exact revision" and is "never disturbed by recalc". True when the pin is the child's only edge; but a child with one tracked parent and one alias-pinned parent gets replayed when the tracked parent advances, and the replay re-runspinned_expr_from_alias("p"), which re-resolves the name to p's new head — the pin silently advances. Hash pins are immune. Fix is either replay-time pin freezing or a doc caveat ("pin by hash if you need replay durability").Passing tests worth a note
test_revise_carry_over.py::test_cascade_carries_chart_to_rebuilt_follower— cascade carry-forward of chart/display config works but is undocumented (caching.md only namescatalog_revise/put_codeas carry-forward callers).test_staleness.py::test_inplace_edit_preserving_mtime_and_size_is_detected— the "forces a faithful re-read" claim holds even when the edit preserves mtime and byte length (the stat-keyed memo's blind spot).test_staleness.py::test_cheap_parent_sources_inline_into_child_and_both_go_directly_stale/test_expensive_parent_shields_child_from_source_axis— the cheap-inline vs expensive-shield source-axis behavior, including the doc's checkable invariant (child sources ⊇ cheap parent sources).test_recalc.py::test_diamond_join_child_rebuilds_once_against_both_new_parents/test_diamond_failure_skips_the_join_child— diamond replay: join child rebuilds once against both advanced heads; stop-and-report skips it when an arm fails.test_recalc.py::test_empty_roots_list_recomputes_nothing—roots=[]is an explicit empty selection, not a falsy fallback to the stale set.test_source_identity.py::test_salt_mode_bakes_no_snapshot_and_reads_still_work/test_salt_rebuild_over_edited_source_forks_hash— first coverage ofsaltmode's documented behavior.test_auto_recalc.py::test_revise_identical_code_keeps_version_and_cascades_nothing— byte-identical revise dedupes: same hash, still version 1, empty cascade.Doc inconsistencies found during the review (no test possible / already covered)
recalcSSE event; architecture.md says it does (andauto-recalc-on-revise.md's currency note confirms the listener shipped). reactive-recalc.md is stale.result_digestviacount_and_result_digest; architecture.md (andtest_cheap_entry_records_no_digest) say worthy-only. expression-lifecycle.md predates ADR: result digest as row-multiset + canonical snapshot ordering #137.test_cas_*reconstruction-pinning tests show digest-pinned reconstruction shipped. Same staleness intest_source_identity.py's module docstring.reasons[].nowis "null if the input is gone" contradicts the same section's "an axis that can't be evaluated … lands in unknown_axes" — in practice a gone input produces no reason at all, andunknown_axesholds axis-qualified refs ("alias:base","source:orders.parquet"), with the bare axis name reserved for a wholly-unevaluable axis (offmode). The doc implies bare names.orphan_stalefrom the scan return shape (mcp-server.md has it) and doesn't document recalc: superseded historical entries treated as live — cascade replays them (manufactured junk entries) and orphan-classifier flags them "UNEXPLAINED / file a bug" #154 head-gating/husk semantics at all ("the scan classifies every live entry" predates theliveflag).🤖 Generated with Claude Code