Skip to content

feat(harness-context): the pi and omp producers over one extension path - #380

Merged
schickling-assistant merged 3 commits into
schickling/hc-03-codexfrom
schickling/hc-04-pi-omp
Aug 29, 2026
Merged

feat(harness-context): the pi and omp producers over one extension path#380
schickling-assistant merged 3 commits into
schickling/hc-03-codexfrom
schickling/hc-04-pi-omp

Conversation

@schickling-assistant

@schickling-assistant schickling-assistant commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Problem

pi and omp seats publish no context. Both harnesses answer the whole fill triple in one call — ctx.getContextUsage() — from an injected extension st2 already ships and already loads for harness-state. Nothing read it.

The same call means different things on the two harnesses, which is the trap: pi's .tokens is the last assistant message's totalTokens (input + output + cacheRead + cacheWrite); omp's is its prompt figure alone. A single producer would publish one harness's arithmetic under the other's tag with no shape change to give it away.

Goal

Two producers over one shared extension path, each pinned to the build its number's meaning was measured on, with compaction carried as one frame that also tells the truth about what it emptied.

Decisions

  • Two producers, one path. The call rides the ctx of every lifecycle event, so neither producer keeps an accumulator or reads a second source. They are separate producers because the identical call carries different meanings — HC-T03's second version-coupled constant. Measured 2026-08-29 in the credential-free labs against pi 0.84.2 and omp 18.0.9, reproduced on 18.0.3.
  • Liberal emit cadence, no producer-side policy. st2's write guard quantizes to 1% of the window, so at most one write per bucket entered however chatty the harness is. message_end is the emit boundary because the wedge case is a single long turn — turn-boundary-only observation was measured at 92% of pre-compaction warnings missed — and each tool call and its result form their own assistant message.
  • One frame per compaction, carrying the withheld reading. Inside pi's own session_compact handler getContextUsage() already reports {tokens: null, percent: null} while the window is still populated, and getEntries() already counts the new entry. So one frame carries the withheld reading, the trigger, and the durable count, and lands as one write. That pairing is load-bearing rather than an optimization: an edge always writes while a withheld percent has no bucket, so an edge sent alone would publish the stale pre-compaction numbers beside it and the truth would wait for the heartbeat.
  • omp's trigger is unknown. Its event names no reason. omp does call its auto-compaction "idle" and "threshold" internally, but those words are not projected onto the event, and inventing one would be a claim no capture supports.
  • costUsd is restated from a producer-side hold. The record replaces a reading's fields wholesale — deliberately, so a withheld number is never fabricated — so a frame from a cost-free event would otherwise erase the published cost. The extension holds the last assistant cost and restates it on every frame, clearing the hold on session replacement.
  • Version pins extend the gates that exist. omp's launch gate admits a minor series; the fixture pins the exact builds a meaning was measured on. Those answer different questions and are allowed to differ — but a test now fails if they drift, because a build the fixture claims to have measured yet the gate refuses to launch is evidence for a version the fleet can never run. pi has no runtime gate, so its fixture pins the flake tarball the extension check already type-checks against.
  • The smoke's channel is a recorder, not true. The frame envelope between the shipped TypeScript assets and pi_channel has its two halves in different languages in different files, and nothing else coupled them: flatten the reading or rename a key and every Rust fixture stays green while the record is never written again — indistinguishable from the pre-producer state. Each smoke now reads its frames back and asserts the wire.

Verification

nix develop -c cargo test --all-targets --all-features --no-fail-fast on this slice (pre-rebase): 1219 passed, 12 failed. Eleven are the sandbox's pre-existing pty/network failures from the branch-base baseline; the twelfth is run::tests::persistent_advisory_warnings_surface_once_not_per_pass, a parallel-execution flake in code this slice does not touch, and the baseline's codex_app_server::tests::a_stop_during_the_websocket_handshake_ends_startup_gracefully passed in this run. Neither recurs in the stack-tip run, whose numbers are quoted on PR 5.

Fixture proofs (src/pi_channel.rs, src/omp_session.rs) each carry both numbers from their capture and assert the one the producer must not publish: the pi fixture asserts 23425 and not 23300; the omp fixture asserts the prompt figure and not that message's totalTokens. The extension smoke (checks.pi-extension-types, a flake check — it does not run under cargo test) drives every handler with three contexts (bare, fully populated, and one whose every telemetry pull throws) and asserts the recorded wire: 23,425 for pi; 22,500 and explicitly not 22,525 for omp; the percent unclamped; an edge whose session store could not be read still arrives countless. Verified non-vacuous — flattening the envelope fails the check with "a context frame must carry a reading object".

axe vrs check --profile strict docs/vrs reports the 107-error pre-existing baseline and nothing more — the DELTA-005 record added here conforms to the canonical delta shape (measured on the stack tip; this slice's only VRS additions are the spec sections and that record).

Complexity

No new module: the producers are added to the shared extension path (hooks/pi-channel.ts, hooks/omp-channel.ts) and to src/pi_channel.rs, which writes through a harness-context Writer beside the harness-state writer it already owned, under the same incarnation. One new MEASURED_CONTEXT_VERSIONS constant and its drift test.

Concerns

  • DELTA-005 is open and needs a maintainer decision. HC-R16 says the record carries the harness-reported session cost; pi and omp report a per-message figure, and the spec's own producer table already said so — the two halves disagreed before any producer existed. Summing to a session total needs exactly the producer-side accumulator HC-R16 refuses one field over (sessionTotalTokens), for the reason that transfers unchanged: a sum whose correctness depends on having seen every message, taken by an extension that may have joined a running session, is worse than an honest smaller number. Requirements are protected, so the delta records it rather than amending HC-R16 unilaterally.
  • A fractional token count off the wire is now tolerated. Both harnesses round today (pi's fallback estimator is Math.ceil(chars / 4), checked in the pinned tarball), so this decodes nothing observed — it exists because a strict integer parse would return None for 1234.75 and the producer would withhold a reading the harness actually had, spending HC-R03's withholding signal on a JSON number shape.
  • The compaction scope narrows silently when the session store cannot be read: the producer still sends the edge and st2 counts it, but the durable count is gone. Written down in the compaction table for that reason.
  • omp does not null its reading at the edge and has no agent_settled; both divergences from pi's path are stated in the spec's omp section.

Friction & bottlenecks

  • Friction: the extension→Rust frame envelope had no coupling test at all — the smoke piped frames into true. A rename on either side was a permanently silent failure. Corrected here; it is the same class of failure as the replication include list that Replicated-path discipline pins names for.
  • Friction: the previous smoke never executed the producer's body in the bare-context case, which is how a TDZ once shipped green through both the type gate and the smoke.
  • Correction to prior work: getContextUsage exists on omp 18.0.3 as well — the 2026-08-25 experiment recorded it as absent there. Both builds were probed on 2026-08-29 and agree, which is what makes the minor-series gate defensible.

Follow-ups

  • DELTA-005: maintainer approval to amend HC-R16 so the cost fact is "the harness-reported cost, at whatever scope that harness reports it, stated per harness in the producer table".

References

  • Parent: the Codex producer PR (PR 3 in this stack). Spec: docs/vrs/08-harness-context/spec.md §pi and §omp; docs/vrs/.delta/DELTA-005-*; decision 0014; HC-R03, HC-R09, HC-R13, HC-R16, HC-T03.
Posted on behalf of @schickling
field value
agent_identity dev3.direct.claude.4f8m9tm7
session dev3.4f8m9tm7
agent_persona generalist
agent_supervisor unavailable
agent_tool Claude Code
agent_tool_version 2.1.250
agent_runtime Claude Code 2.1.250
tooling_profile dotfiles@a1a5f89

@schickling-assistant schickling-assistant changed the title schickling/hc 04 pi omp feat(harness-context): the pi and omp producers over one extension path Aug 29, 2026
@schickling-assistant
schickling-assistant marked this pull request as ready for review August 29, 2026 16:48
schickling-assistant and others added 3 commits August 30, 2026 00:14
One extension path, two harness tags. `ctx.getContextUsage()` answers the whole
fill triple in one call on both harnesses and rides the ctx of every lifecycle
event, so neither producer keeps an accumulator or reads a second source.

The two are separate producers because the identical call means different
things, which is HC-T03's second version-coupled constant: pi's `tokens` is the
last assistant message's `totalTokens` (input + output + cacheRead +
cacheWrite), omp's is its prompt figure alone. Measured 2026-08-29 in the
credential-free labs against pi 0.84.2 and omp 18.0.9, reproduced on 18.0.3. The
fixtures carry both numbers from each capture and assert the one the producer
must NOT publish, so a bump that silently swaps the meaning fails rather than
publishing a differently-meaning number under the same field name.

Emit cadence is deliberately liberal and holds no policy: st2's write guard
quantizes to 1% of the window, so at most one write per bucket entered however
chatty the harness is. The finest boundary is what matters — turn-boundary-only
observation was measured at 92% of pre-compaction warnings missed, because the
wedge case is a single long turn, and `message_end` defeats that since each tool
call and its result form their own assistant message.

Three measurements shaped the compaction path. Inside pi's own `session_compact`
handler `getContextUsage()` ALREADY reports `{tokens: null, percent: null}`
while the window stays populated, and `getEntries()` already counts the new
entry — so one frame carries the withheld reading, the trigger, and the durable
count, and lands as one write. That pairing is load-bearing: an edge always
writes while a withheld percent has no bucket, so an edge sent alone would
publish the stale pre-compaction numbers beside it and the truth would wait for
the heartbeat. omp's event names no reason at all, so its trigger is `unknown`;
it does call its auto-compaction "idle" and "threshold" internally, but those
words are not projected onto the event and inventing one would be a claim no
capture supports.

`costUsd` is the last assistant message's `usage.cost.total`, held in the stash
and restated on every frame: the record replaces a reading's fields wholesale —
deliberately, so a withheld number is never fabricated — so a frame from an
event carrying no cost would otherwise erase it. `sessionTotalTokens` stays null
on both, per HC-R16: summing it needs a producer-side accumulator whose
correctness depends on having seen every message.

The version pins extend the gates that exist rather than adding a mechanism.
omp's launch gate admits a minor series and the fixture pins the exact builds a
meaning was measured on; a new test fails if the fixture ever claims a build the
gate would refuse to launch. pi has no runtime gate, so its fixture pins the
flake tarball the extension check already type-checks and smokes against.

Both smokes now drive each handler with three contexts — bare, fully populated,
and one whose every telemetry pull throws. The bare one alone never executed the
producer's body at all, which is precisely the blind spot that once let a TDZ
ship green. A compile-time pin on pi's own declarations keeps the type gate's
teeth where the runtime guards widen the view.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
agent-identity: dev3.direct.claude.4f8m9tm7
agent-persona: generalist
agent-supervisor: unavailable
agent-tool: Claude Code
agent-tool-version: 2.1.250
agent-runtime: Claude Code 2.1.250
tooling-profile: dotfiles@a1a5f89
The spec's Status section claimed no producer ships, which two of five now do.
HC-R11 is partially satisfied and says so; the three unshipped harnesses still
read `context: null`, which stays the honest state rather than a defect.

Three facts settled while implementing pi's producer, each shaping the code
rather than only its constants, and each recorded in the pi section: the nulls
are already present inside pi's own `session_compact` handler (so the edge and
the withheld reading are one frame and one write, which the write policy makes
load-bearing rather than an optimization); `message_end` is the emit boundary
because the wedge case is a single long turn; and `costUsd` must be restated
from a producer-side hold, because the record replaces a reading's fields
wholesale and a frame from a cost-free event would otherwise erase it.

The omp section gains its two divergences from that path — omp does not null its
reading at the edge, and has no `agent_settled` — and states how the version pin
extends the existing launch gate. The gate admits a minor series and the fixture
pins exact builds; those answer different questions and are allowed to differ,
but a test now fails if they drift, because a build the fixture claims to have
measured but the gate refuses to launch is evidence for a version nothing runs.

The compaction table gains the one degradation path a "harness-durable" row has:
a producer that cannot read the session store still sends the edge, and st2
counts it. The scope narrows silently, which is exactly why it is written down.

DELTA-005 records a real contradiction rather than papering over it. HC-R16 says
the record carries "the harness-reported session cost", but pi and omp report a
per-message figure, and the spec's own producer table already said so — the two
halves disagreed before any producer existed. Summing to a session total needs
the producer-side accumulator HC-R16 refuses one field over, for
`sessionTotalTokens`, for the reason that transfers unchanged: a sum whose
correctness depends on having seen every message, taken by an extension that may
have joined a running session, is worse than an honest smaller number.

The *Harness context discipline* invariant row gains the producer half — a
harness's own numerator, pinned to the build it was measured on, so neither
harness's arithmetic can ship under the other's tag — with the eight new proofs
named. No new row: three producers do not exist yet, and a row claiming
all-harness fixture discipline would be false.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
agent-identity: dev3.direct.claude.4f8m9tm7
agent-persona: generalist
agent-supervisor: unavailable
agent-tool: Claude Code
agent-tool-version: 2.1.250
agent-runtime: Claude Code 2.1.250
tooling-profile: dotfiles@a1a5f89
…ust the load

The frame envelope between the shipped assets and `pi_channel` has its two
halves in different languages in different files, and nothing coupled them.
With `true` as the channel binary the frames went into a pipe nobody read, so
the smoke proved "does not throw" and the Rust fixtures decoded frames written
by hand in the tests. Flatten the reading or rename a key and all of it stays
green while the record is never written again — a failure indistinguishable from
the pre-producer state where every declaration's context reads null, and the
same shape as the replication include list that Replicated-path discipline
already pins names for.

The channel is now a recorder that speaks a protocol-1 hello and appends every
frame, and each smoke reads them back: a context frame is emitted at all, the
reading carries all five keys, the numerator is each harness's own (23,425 for
pi; 22,500 and explicitly not 22,525 for omp), the percent is unclamped, an edge
whose session store could not be read still arrives countless, and pi's withheld
reading rides the same frame as its edge. Verified non-vacuous: flattening the
envelope fails the check with "a context frame must carry a `reading` object".

Frames are selected by predicate rather than position, because which of the
three contexts produced a given frame is not what is under test and asserting on
an index would break whenever the loop gains a case.

Also tolerate a fractional token count off the wire. Both harnesses round today
— pi's fallback estimator is `Math.ceil(chars / 4)`, checked in the pinned
tarball — so this decodes nothing observed. It exists because the failure if one
ever stops rounding is silent in the worst direction: a strict integer parse
returns `None` for `1234.75` and the producer would WITHHOLD a reading the
harness actually had. Withholding is reserved for a harness saying it does not
know (HC-R03), and spending it on a JSON number shape would make the record lie
about which of those two happened. The percent leg already parsed as a float
because pi and omp genuinely emit one there; the operands were stricter for no
measured reason.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
agent-identity: dev3.direct.claude.4f8m9tm7
agent-persona: generalist
agent-supervisor: unavailable
agent-tool: Claude Code
agent-tool-version: 2.1.250
agent-runtime: Claude Code 2.1.250
tooling-profile: dotfiles@a1a5f89
@schickling-assistant
schickling-assistant merged commit f0868c3 into main Aug 29, 2026
2 checks passed
@schickling-assistant
schickling-assistant deleted the schickling/hc-04-pi-omp branch August 30, 2026 09:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant