feat(harness): widen the observation channel — stdout, file mode, file type, and writes outside the state directory - #145
Conversation
…them A semantic-search tool dropped an ONNX sentence-transformer cache into skills/project-lifecycle/hooks/local_cache/ because that happened to be the calling process's cwd. Tens of MB of weights, unrelated to this repo's deliverable, and untracked-but-not-ignored is one `git add -A` from history. Unanchored (`**/local_cache/`) on purpose: the directory lands wherever the caller was standing, not at the repo root. Verified with `git check-ignore -v` rather than assumed -- the same anchoring mistake cost a PR in phase b27. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UrKijNNcAPknxAU78grxWm
The observation record framed its sections with literal lines (`--stderr--`, `--fsstate--`) and extracted them with sed ranges, so stderr containing a line that read `--fsstate--` terminated the extraction early and the remainder was attributed to the filesystem section (B110). Sections are now length-prefixed (`<name>:<bytes>:` then exactly that many raw bytes) and the reader WALKS the record from the top -- read a header, consume exactly N bytes, repeat. It never searches. That distinction is the whole fix: a first draft of this change located the header with `grep -n "^$name:[0-9]*:$"` and the cross-family review measured it returning bytes from inside stderr on a record whose payload contained `fsstate:4:` -- the fix had rebuilt the bug under a new spelling. The record now lives in a file and `observe` returns its path. Command substitution strips trailing newlines before a byte count can be taken, so `$(cat "$err")` would have made the prefix describe fewer bytes than the hook actually wrote -- moving the old `printf x` trailing-newline trap rather than removing it. Consequences carried through: `diverge_case` pins compose their lengths from the helper (never hand-counted), a `normalize_record` step re-derives lengths after sandbox-path folding, and the meta-suite's hand-typed `CONTRACT_OLD` and its two malformed-record stubs are re-expressed under the new encoding. Evidence: differential 91/0/0/4 known + 10 lead -- byte-identical to the branch point, because this changes framing and not verdicts. contract 67/0/2/0 and test-hooks 99/0 unchanged. meta-suite 21 -> 24. meta02e asserts the extracted BYTES, not just the verdict. The two collision rows added first assert only that the sides DIVERGE, and a search-based reader still makes them differ -- so both stay green under exactly the regression they were added to catch. Reverting the reader to a search reddens meta02e alone, demonstrated. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UrKijNNcAPknxAU78grxWm
…can hide `context-floor.sh` is a PreToolUse hook, and for PreToolUse stdout is a control surface: a JSON `hookSpecificOutput.permissionDecision` there overrides the exit status. The observation record never captured it -- the hook's stdout was redirected to a file nothing read -- so a mutant that prints `permissionDecision: allow` before its blocking `exit 2` was reported green by all four suites. A gate that had stopped gating, reported as no change at all (B108). `observe` now emits a `stdout` section, read from the file rather than through `$(cat ...)` so a trailing newline (which a real JSON decision carries) is inside the byte count instead of stripped before it. `mkfake` gains a stdout parameter whose presence is derived from `$#` BEFORE quoting: `printf '%q' ""` yields `''`, so a post-quoting guard is true for every fake and each one would emit a literal `''` line -- the absent-vs-empty collapse B85 was filed about, reproduced in a fixture builder. Two places had to become stdout-aware or the channel would have been recorded and then ignored where it matters most: * `diff_case`'s known-branch compared exit+fsstate only, so a stdout difference under a `known, stderr only` label was laundered into a quiet DIVERGE. `known` exempts stderr and nothing else. Four corpus rows carry that label. * `diverge_case`'s pins and `normalize_record` were three-section, leaving the ten deliberate-lead rows blind to the new channel. Both pin composers now assert `stdout:0:` -- correct today for the blocked and silent paths, and it fails loudly if either ever writes to stdout. meta04b is why those two are not decoration: it drives a stderr-only-labelled pair differing ONLY in stdout, and reddens under exactly the mutation that removes stdout from the wider-than-claimed comparison, while meta03 and meta04 stay green. Without it, reverting that comparison moves nothing. Evidence: differential 91/0/0/4 known + 10 lead -- unchanged from the branch point. contract 67/0/2/0 and test-hooks 99/0 unchanged. meta-suite 24 -> 26. The B108 mutant is green before this commit and red after it, differing inside the stdout section with exit, stderr and fsstate identical. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UrKijNNcAPknxAU78grxWm
…could hide behind
…, and writes outside its own directory
… ran from the repo
Review round 1 — cross-family (codex), one lensFirst real use of One Important finding, and it was right. The stray-write check added in +printf '%s %s\n' "$SID" "$TPATH" >> .plc-floor-audit.log 2>/dev/null || trueVerified, not accepted: Why my own mutant missed it. I wrote mine as Fixed in Residual, filed rather than smuggled in — B121. A name-set diff detects CREATION. Appending to an Readings after the fix: Termination. This round produced a code change, so there is unreviewed code after the last review |
The differential harness decides, for ~100 rows, whether two implementations of
context-floor.shagree. It decided that from a record containing exit code, stderr, and
{name, content}for theentries of one directory. Three mutants of the hook were therefore green in all four suites:
permissionDecisionon stdout — which, for a PreToolUse hook, overrides the exitstatus the suites were reading;
chmod 0666s its own guard state, after which any local user can forge or clear anotheruser's floor — a bypass the block message is deliberately written to withhold;
session ids and file paths, growing forever.
This phase widens what the harness sees. It is the sibling of
b90-b103, which fixed what theharness concludes: a correct verdict over an incomplete observation is still the wrong answer.
What landed
d991465observereturns a path) because$( )strips trailing newlines before a byte count is taken.8abe368stdoutsection, and the two branches that judge labelled divergences (diff_case's known-branch,diverge_case's pins) made to read it.7cecf65mode=andtype=as separate fields, on their own probe, plusmkfakeparameters so the meta layer can move one channel at a time.aa35708run_hookcall in the contract suite snapshots the sandbox; anything created outsideplc-context-floor/is a finding.c6ea7d2stat-formatter family, so the census can be diffed between macOS and Linux at all. Census regenerated.6c167f4f9a70cd[Unreleased], ROADMAP row.Readings
differentialis unchanged from the branch point on purpose — this phase changed observation, notverdicts. The census (
.claude/baseline-differential-rows.txt) moved 2 lines, both deliberaterenames from T5, which is exactly what the branch-point declaration predicted in advance.
Every mutant, measured before and after:
permissionDecisionon stdoutstdoutsection, every other section byte-identicalchmod 0666 "$MARKER"at the arm site91 passed, 0 failed×2mode=644againstmode=66691 passed, 0 failed×2type=regularagainsttype=symbolic$CWD/.plc-floor-audit.log67 passed, 0 failedrun_hookcall sitesmode=/ droptype=27 passed, 2 failed, each on its own meta rowget_sectionby search / known-branch drops stdout /observedrops stdoutmeta02ealone ·meta04balone ·meta06b+ 4 moreEvery meta row now has at least one mutation that reddens it. That was not true before T6.
The three findings worth more than the code
T3's "both mutants are green today" held only after the mutant was rebuilt to be mtime-neutral.
T4's own snippet called
nofrom inside a command-substitution subshell (the counter dies with thesubshell), and its step-1 probe drove the mutant with
PLC_CF_IMPLwhile the contract suite readsPLC_CF_NEW— so its baseline green exercised the clean hook and said nothing. T5's census recipewas missing a leading-space strip. The plan was written by the same discipline that caught these;
carefully written is not measured.
diffinvocations called the baseline census and the regenerated one identical, while
cmplocated thedifference at char 1649 — the two rename lines T5 exists to make. Taken alone, the first reading
would have blessed a golden without anyone seeing what moved in it.
the marker with a plain
cp, moving its mtime; it reddened three unrelated rows, and two runs ofthe same mutant scored 5 failed and then 3 failed. A mutant that reddens for a reason other than
the field under test proves nothing about that field. Same-SHA reruns are what made it visible —
B103's lesson, one level up from the rows it was filed about.
Open, carried out of this phase
The 9-PR batch map is still only in a prior conversation.
RESUME.mdrecords that the remainingbacklog was batched into 9 PRs with approval, that this phase is PR 1, and that landing the map in
docs/ROADMAP.mdbelonged to T7. The map is not in any artifact this session can read, and it hasnot been reconstructed from memory — that is precisely the fabricated-causal-chain shape this repo
has shipped once already. Filed in the journal's wrap-up.
Gate
close-gate phase b108— PASS, all 15 rows.SKIP: user-story.md — no user-observable surface.Every change lives in the three suites that watchcontext-floor.sh; no adopter-facing command, output or configuration key moved.SKIP: dual-track smoke — same.The deliverable IS the suites.🤖 Generated with Claude Code
https://claude.ai/code/session_01UrKijNNcAPknxAU78grxWm