Skip to content

feat(harness): widen the observation channel — stdout, file mode, file type, and writes outside the state directory - #145

Merged
Victoriakaey merged 9 commits into
mainfrom
feat/phase-b108-b109-observation-channel
Jul 28, 2026
Merged

feat(harness): widen the observation channel — stdout, file mode, file type, and writes outside the state directory#145
Victoriakaey merged 9 commits into
mainfrom
feat/phase-b108-b109-observation-channel

Conversation

@Victoriakaey

Copy link
Copy Markdown
Owner

The differential harness decides, for ~100 rows, whether two implementations of context-floor.sh
agree. It decided that from a record containing exit code, stderr, and {name, content} for the
entries of one directory. Three mutants of the hook were therefore green in all four suites:

  • one printing a permissionDecision on stdout — which, for a PreToolUse hook, overrides the exit
    status the suites were reading;
  • one that chmod 0666s its own guard state, after which any local user can forge or clear another
    user's floor — a bypass the block message is deliberately written to withhold;
  • one that appends to a log in the user's working tree on every gated Edit/Write, naming their
    session ids and file paths, growing forever.

This phase widens what the harness sees. It is the sibling of b90-b103, which fixed what the
harness concludes: a correct verdict over an incomplete observation is still the wrong answer.

What landed

commit task what
d991465 T1 B110 — sections are length-prefixed and the reader walks the record; it never searches for a header. The record moved into a file (observe returns a path) because $( ) strips trailing newlines before a byte count is taken.
8abe368 T2 B108 — a stdout section, and the two branches that judge labelled divergences (diff_case's known-branch, diverge_case's pins) made to read it.
7cecf65 T3 B109 first half — mode= and type= as separate fields, on their own probe, plus mkfake parameters so the meta layer can move one channel at a time.
aa35708 T4 B109 second half — every run_hook call in the contract suite snapshots the sandbox; anything created outside plc-context-floor/ is a finding.
c6ea7d2 T5 B113 — row names drop the platform's stat-formatter family, so the census can be diffed between macOS and Linux at all. Census regenerated.
6c167f4 T6 Four claims corrected that were true when written and stopped being true quietly; the mutation table's own gap closed with three measured mutants.
f9a70cd T7 CHANGELOG [Unreleased], ROADMAP row.

Readings

test-hooks    99 passed, 0 failed
contract      67 → 68 passed, 0 failed, 2 xfail (expected), 0 xpass
differential  91 passed, 0 failed, 0 skipped, 4 known-diverge, 10 deliberate-lead
meta-suite    21 → 28 passed, 0 failed

differential is unchanged from the branch point on purpose — this phase changed observation, not
verdicts. The census (.claude/baseline-differential-rows.txt) moved 2 lines, both deliberate
renames from T5, which is exactly what the branch-point declaration predicted in advance.

Every mutant, measured before and after:

mutant before after
permissionDecision on stdout green, 4 suites red, in the stdout section, every other section byte-identical
chmod 0666 "$MARKER" at the arm site 91 passed, 0 failed ×2 red, naming mode=644 against mode=666
marker replaced by a symlink 91 passed, 0 failed ×2 red, naming type=regular against type=symbolic
append to $CWD/.plc-floor-audit.log 67 passed, 0 failed red, naming the file and its four run_hook call sites
lib: drop mode= / drop type= 27 passed, 2 failed, each on its own meta row
lib: get_section by search / known-branch drops stdout / observe drops stdout meta02e alone · meta04b alone · meta06b + 4 more

Every meta row now has at least one mutation that reddens it. That was not true before T6.

The three findings worth more than the code

  1. Three of the plan's four task-level assertions were wrong, and measurement is what found them.
    T3's "both mutants are green today" held only after the mutant was rebuilt to be mtime-neutral.
    T4's own snippet called no from inside a command-substitution subshell (the counter dies with the
    subshell), and its step-1 probe drove the mutant with PLC_CF_IMPL while the contract suite reads
    PLC_CF_NEW — so its baseline green exercised the clean hook and said nothing. T5's census recipe
    was missing a leading-space strip. The plan was written by the same discipline that caught these;
    carefully written is not measured.
  2. A tool that reports agreement is the one whose agreement needs a second opinion. Three diff
    invocations called the baseline census and the regenerated one identical, while cmp located the
    difference at char 1649 — the two rename lines T5 exists to make. Taken alone, the first reading
    would have blessed a golden without anyone seeing what moved in it.
  3. A mutant is a measuring instrument and needs its own calibration. The first type mutant copied
    the marker with a plain cp, moving its mtime; it reddened three unrelated rows, and two runs of
    the same mutant scored 5 failed and then 3 failed. A mutant that reddens for a reason other than
    the field under test proves nothing about that field. Same-SHA reruns are what made it visible —
    B103's lesson, one level up from the rows it was filed about.

Open, carried out of this phase

The 9-PR batch map is still only in a prior conversation. RESUME.md records that the remaining
backlog was batched into 9 PRs with approval, that this phase is PR 1, and that landing the map in
docs/ROADMAP.md belonged to T7. The map is not in any artifact this session can read, and it has
not been reconstructed from memory — that is precisely the fabricated-causal-chain shape this repo
has shipped once already. Filed in the journal's wrap-up.

Gate

close-gate phase b108 — PASS, all 15 rows.

SKIP: user-story.md — no user-observable surface. Every change lives in the three suites that watch
context-floor.sh; no adopter-facing command, output or configuration key moved.
SKIP: dual-track smoke — same. The deliverable IS the suites.

🤖 Generated with Claude Code

https://claude.ai/code/session_01UrKijNNcAPknxAU78grxWm

Victoriakaey and others added 9 commits July 28, 2026 09:54
…them

A semantic-search tool dropped an ONNX sentence-transformer cache into
skills/project-lifecycle/hooks/local_cache/ because that happened to be the
calling process's cwd. Tens of MB of weights, unrelated to this repo's
deliverable, and untracked-but-not-ignored is one `git add -A` from history.

Unanchored (`**/local_cache/`) on purpose: the directory lands wherever the
caller was standing, not at the repo root. Verified with `git check-ignore -v`
rather than assumed -- the same anchoring mistake cost a PR in phase b27.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UrKijNNcAPknxAU78grxWm
The observation record framed its sections with literal lines (`--stderr--`,
`--fsstate--`) and extracted them with sed ranges, so stderr containing a line
that read `--fsstate--` terminated the extraction early and the remainder was
attributed to the filesystem section (B110).

Sections are now length-prefixed (`<name>:<bytes>:` then exactly that many raw
bytes) and the reader WALKS the record from the top -- read a header, consume
exactly N bytes, repeat. It never searches. That distinction is the whole fix:
a first draft of this change located the header with `grep -n "^$name:[0-9]*:$"`
and the cross-family review measured it returning bytes from inside stderr on a
record whose payload contained `fsstate:4:` -- the fix had rebuilt the bug under
a new spelling.

The record now lives in a file and `observe` returns its path. Command
substitution strips trailing newlines before a byte count can be taken, so
`$(cat "$err")` would have made the prefix describe fewer bytes than the hook
actually wrote -- moving the old `printf x` trailing-newline trap rather than
removing it. Consequences carried through: `diverge_case` pins compose their
lengths from the helper (never hand-counted), a `normalize_record` step
re-derives lengths after sandbox-path folding, and the meta-suite's hand-typed
`CONTRACT_OLD` and its two malformed-record stubs are re-expressed under the
new encoding.

Evidence: differential 91/0/0/4 known + 10 lead -- byte-identical to the branch
point, because this changes framing and not verdicts. contract 67/0/2/0 and
test-hooks 99/0 unchanged. meta-suite 21 -> 24.

meta02e asserts the extracted BYTES, not just the verdict. The two collision
rows added first assert only that the sides DIVERGE, and a search-based reader
still makes them differ -- so both stay green under exactly the regression they
were added to catch. Reverting the reader to a search reddens meta02e alone,
demonstrated.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UrKijNNcAPknxAU78grxWm
…can hide

`context-floor.sh` is a PreToolUse hook, and for PreToolUse stdout is a control
surface: a JSON `hookSpecificOutput.permissionDecision` there overrides the exit
status. The observation record never captured it -- the hook's stdout was
redirected to a file nothing read -- so a mutant that prints
`permissionDecision: allow` before its blocking `exit 2` was reported green by
all four suites. A gate that had stopped gating, reported as no change at all
(B108).

`observe` now emits a `stdout` section, read from the file rather than through
`$(cat ...)` so a trailing newline (which a real JSON decision carries) is
inside the byte count instead of stripped before it. `mkfake` gains a stdout
parameter whose presence is derived from `$#` BEFORE quoting: `printf '%q' ""`
yields `''`, so a post-quoting guard is true for every fake and each one would
emit a literal `''` line -- the absent-vs-empty collapse B85 was filed about,
reproduced in a fixture builder.

Two places had to become stdout-aware or the channel would have been recorded
and then ignored where it matters most:

* `diff_case`'s known-branch compared exit+fsstate only, so a stdout difference
  under a `known, stderr only` label was laundered into a quiet DIVERGE. `known`
  exempts stderr and nothing else. Four corpus rows carry that label.
* `diverge_case`'s pins and `normalize_record` were three-section, leaving the
  ten deliberate-lead rows blind to the new channel. Both pin composers now
  assert `stdout:0:` -- correct today for the blocked and silent paths, and it
  fails loudly if either ever writes to stdout.

meta04b is why those two are not decoration: it drives a stderr-only-labelled
pair differing ONLY in stdout, and reddens under exactly the mutation that
removes stdout from the wider-than-claimed comparison, while meta03 and meta04
stay green. Without it, reverting that comparison moves nothing.

Evidence: differential 91/0/0/4 known + 10 lead -- unchanged from the branch
point. contract 67/0/2/0 and test-hooks 99/0 unchanged. meta-suite 24 -> 26.
The B108 mutant is green before this commit and red after it, differing inside
the stdout section with exit, stderr and fsstate identical.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UrKijNNcAPknxAU78grxWm
@Victoriakaey

Copy link
Copy Markdown
Owner Author

Review round 1 — cross-family (codex), one lens

First real use of scripts/cross-family-code-review.sh (B118, built last phase). The brief asked for
one thing: an observation the widened record still cannot make, ideally as a concrete mutant.

One Important finding, and it was right.

The stray-write check added in aa35708 claimed "the hook writes nothing outside its own state
directory" and snapshotted only the sandbox $d. run_hook's default rundir is . — the harness's
own checkout. codex produced the mutant:

+printf '%s %s\n' "$SID" "$TPATH" >> .plc-floor-audit.log 2>/dev/null || true

Verified, not accepted: 67 passed, 1 failed — caught by the single row that happens to pass an
explicit rundir — while 5.6K of audit log accumulated in the real repository root during the run.
One row out of seventy is a coincidence, not an oracle.

Why my own mutant missed it. I wrote mine as "$CWD/.plc-floor-audit.log", and $CWD is the
event's cwd, which the fixtures set to the sandbox — so my mutant wrote exactly where my check was
looking. The reviewer's used a relative path. The mutant was shaped by the same assumption as the fix.
That is this repo's most-cited finding ("different lens, not more reviewers") arriving again, inside
the fix for the gap it describes.

Fixed in 103d42f: run_hook now snapshots the resolved run directory as well as the sandbox,
deduped when they are the same path, with no plc-context-floor exemption on the run directory
(TMPDIR is $d, so a state directory appearing elsewhere is itself the finding). The mutant is now
caught at line 164, naming the run directory.

Residual, filed rather than smuggled in — B121. A name-set diff detects CREATION. Appending to an
existing file, or writing inside an existing directory, leaves the name set unchanged: the mutant above
is caught at the first call and invisible from the second onward. The real fix is a recursive
snapshot with content digests, ~71 times per run, over a tree containing .git/ — a different cost
that needs an exclusion decision first. Widening it without that decision buys flaky reds, which is
B103's bill.

Readings after the fix: test-hooks 99/0 · contract 68/0/2 XFAIL/0 XPASS · differential 91/0/0/4 known + 10 lead · meta 28/0 · close-gate phase b108 PASS.

Termination. This round produced a code change, so there is unreviewed code after the last review
SHA — B98's point that review-record.md's rule is literally unexecutable. The terminator used is the
D9 form: this round's entire behavioural change is covered by an unattended oracle, namely the
reviewer's own mutant, now a standing measurement.

@Victoriakaey
Victoriakaey merged commit 6f3b863 into main Jul 28, 2026
1 check passed
@Victoriakaey
Victoriakaey deleted the feat/phase-b108-b109-observation-channel branch July 28, 2026 18:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant