Skip to content

Commit 1509300

Browse files
schickling-assistantschickling-assistant
committed
test: tighten PTY stream acceptance oracle
Assert exact initial and reconnect geometry, preserved SGR state, ordered exit, and clean side channels. Add deterministic mutations for each failure boundary. agent-session-id: ed878dac-3735-4276-b3e0-ea1b1cd65291 agent-tool: Codex CLI agent-tool-version: 0.145.0 agent-model: unknown agent-runtime-profile: /nix/store/mnx8agbdq3wiyb6vz63lhgscgazkrn98-coding-agent-runtime-profile/share/coding-agents/profile.json agent-skills-manifest: /nix/store/5r69m9k2llmri3na81518zx0a7y0d3cn-agent-skills-corpus/share/agent-skills/manifest.json tooling-profile: dotfiles@0fb7e03
1 parent 62be35b commit 1509300

5 files changed

Lines changed: 159 additions & 67 deletions

File tree

‎cells/pty-attach-machine-stream/README.md‎

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -5,22 +5,24 @@
55
[compoundingtech/pty#141](https://github.com/compoundingtech/pty/pull/141), merged on PTY main at
66
[`d5fabc3917407aeb937a012bd97679c303e18033`](https://github.com/compoundingtech/pty/commit/d5fabc3917407aeb937a012bd97679c303e18033).
77

8-
**Capabilities required:** `pty,jq,node`. No model and no bus. The cell uses the `pty` executable on `PATH`,
8+
**Capabilities required:** `pty,jq,node,script`. No model and no bus. The cell uses the `pty` executable on `PATH`,
99
not a source-tree module or `dist/cli.js` entrypoint.
1010

1111
## What it proves
1212

1313
- **Packaged launcher:** fd 3 crosses the shipped `bin/pty` boundary and carries a parseable v1 stream.
14-
- **Real initial snapshot:** an eval-owned target daemon emits `GEOMETRY` immediately followed by `SCREEN`;
15-
the screen includes colored terminal state produced before attach.
14+
- **Real initial snapshot:** an eval-owned target daemon emits exact `24x80` `GEOMETRY` immediately followed
15+
by `SCREEN`; the screen preserves the red SGR state produced before attach.
1616
- **Real reconnect snapshot:** an installed `pty remote-serve` process exposes the target through a one-shot
1717
local transport proxy. A synthetic `fabric dial` selector drops the first route and withholds the second until
1818
the target has produced another
19-
line. The one continuous fd 3 stream must then contain another adjacent `GEOMETRY`, `SCREEN` pair whose
20-
screen includes that line.
19+
line and attaches a controlled `13x47` client. The one continuous fd 3 stream must then contain exact
20+
`13x47` `GEOMETRY` followed by `SCREEN`, proving both min-wins geometry and a current snapshot.
2121
- **Framing boundary:** terminal content and `EXIT` are decoded from fd 3, while stdout stays empty and stderr
2222
contains only reconnect status. This catches both descriptor loss and protocol/text contamination.
2323
- **Cleanup:** the target and both route servers are removed from the eval-owned PTY root.
24+
- **Oracle mutations:** wrong geometry, stripped SGR, stale reconnect state, data after exit, truncation, and
25+
stdout/stderr contamination all fail the checker before the real composition runs.
2426

2527
The fixture controls only transport selection and the deliberate connection drop; the installed CLI, PTY daemon, remote routing, attach client,
2628
snapshot serialization, reconnect loop, and packaged launcher are all exercised as shipped. This is broader

‎cells/pty-attach-machine-stream/fixture/check-stream.mjs‎

Lines changed: 126 additions & 48 deletions
Original file line numberDiff line numberDiff line change
@@ -1,62 +1,140 @@
11
#!/usr/bin/env node
22
import fs from "node:fs"
3+
import assert from "node:assert/strict"
34

4-
const [path, mode, expectedRaw] = process.argv.slice(2)
5-
if (!path || !mode) process.exit(2)
6-
const data = fs.existsSync(path) ? fs.readFileSync(path) : Buffer.alloc(0)
7-
const packets = []
8-
let offset = 0
9-
while (offset + 5 <= data.length) {
10-
const type = data.readUInt8(offset)
11-
const length = data.readUInt32BE(offset + 1)
12-
if (length > 32 * 1024 * 1024) throw new Error(`oversized frame: ${length}`)
13-
if (offset + 5 + length > data.length) break
14-
packets.push({ type, payload: data.subarray(offset + 5, offset + 5 + length) })
15-
offset += 5 + length
5+
const geometry = (rows, columns) => {
6+
const payload = Buffer.alloc(4)
7+
payload.writeUInt16BE(rows, 0)
8+
payload.writeUInt16BE(columns, 2)
9+
return payload
1610
}
1711

18-
const snapshotIndexes = []
19-
for (let index = 0; index + 1 < packets.length; index++) {
20-
if (packets[index].type === 10 && packets[index + 1].type === 5) snapshotIndexes.push(index)
12+
const frame = (type, payload) => {
13+
const header = Buffer.alloc(5)
14+
header.writeUInt8(type)
15+
header.writeUInt32BE(payload.length, 1)
16+
return Buffer.concat([header, payload])
2117
}
2218

23-
if (mode === "snapshots") {
24-
const expected = Number(expectedRaw)
25-
process.exit(snapshotIndexes.length >= expected ? 0 : 1)
26-
}
27-
if (mode !== "final") process.exit(2)
28-
if (offset !== data.length) throw new Error("truncated trailing frame")
29-
if (packets.length < 5) throw new Error("too few frames")
30-
if (packets[0].type !== 10 || packets[1].type !== 5) {
31-
throw new Error("initial stream does not begin with GEOMETRY, SCREEN")
32-
}
33-
if (snapshotIndexes.length !== 2) {
34-
throw new Error(`expected two snapshots, got ${snapshotIndexes.length}`)
35-
}
36-
for (const index of snapshotIndexes) {
37-
const geometry = packets[index].payload
38-
if (geometry.length !== 4 || geometry.readUInt16BE(0) === 0 || geometry.readUInt16BE(2) === 0) {
39-
throw new Error("invalid geometry payload")
19+
const decode = (data, complete) => {
20+
const packets = []
21+
let offset = 0
22+
while (offset + 5 <= data.length) {
23+
const type = data.readUInt8(offset)
24+
const length = data.readUInt32BE(offset + 1)
25+
if (length > 32 * 1024 * 1024) throw new Error(`oversized frame: ${length}`)
26+
if (offset + 5 + length > data.length) break
27+
packets.push({ type, payload: data.subarray(offset + 5, offset + 5 + length) })
28+
offset += 5 + length
4029
}
30+
if (complete && offset !== data.length) throw new Error("truncated trailing frame")
31+
return packets
4132
}
42-
const initial = packets[snapshotIndexes[0] + 1].payload
43-
const reconnected = packets[snapshotIndexes[1] + 1].payload
44-
if (!initial.includes(Buffer.from("INITIAL_COLOR_61e8")) || !initial.includes(Buffer.from("\x1b"))) {
45-
throw new Error("initial screen lost colored terminal state")
33+
34+
const snapshotIndexes = (packets) => {
35+
const indexes = []
36+
for (let index = 0; index + 1 < packets.length; index++) {
37+
if (packets[index].type === 10 && packets[index + 1].type === 5) indexes.push(index)
38+
}
39+
return indexes
4640
}
47-
if (!reconnected.includes(Buffer.from("AFTER_DROP_61e8"))) {
48-
throw new Error("reconnect screen is not the current terminal state")
41+
42+
const parseGeometries = (raw) => raw.split(",").map((entry) => {
43+
const match = /^(\d+)x(\d+)$/.exec(entry)
44+
if (!match) throw new Error(`invalid expected geometry: ${entry}`)
45+
return { rows: Number(match[1]), columns: Number(match[2]) }
46+
})
47+
48+
const validateSnapshots = (packets, expectedGeometries, count) => {
49+
const indexes = snapshotIndexes(packets)
50+
if (indexes.length < count) throw new Error(`expected ${count} snapshots, got ${indexes.length}`)
51+
for (let snapshot = 0; snapshot < count; snapshot++) {
52+
const payload = packets[indexes[snapshot]].payload
53+
const expected = expectedGeometries[snapshot]
54+
if (!expected || payload.length !== 4 || payload.readUInt16BE(0) !== expected.rows ||
55+
payload.readUInt16BE(2) !== expected.columns) {
56+
throw new Error(`snapshot ${snapshot + 1} geometry does not match ${expected?.rows}x${expected?.columns}`)
57+
}
58+
}
59+
return indexes
4960
}
50-
const exits = packets.filter((packet) => packet.type === 4)
51-
if (exits.length !== 1 || packets.at(-1).type !== 4) throw new Error("stream does not end in one EXIT")
52-
if (!packets.some((packet) => packet.type === 0 && packet.payload.includes(Buffer.from("FINAL_DATA_61e8")))) {
53-
throw new Error("final terminal DATA was not ordered before EXIT")
61+
62+
const validateFinal = (data, expectedGeometries, stdout = Buffer.alloc(0), stderr = Buffer.alloc(0)) => {
63+
const packets = decode(data, true)
64+
if (packets.length < 5) throw new Error("too few frames")
65+
if (packets[0].type !== 10 || packets[1].type !== 5) {
66+
throw new Error("initial stream does not begin with GEOMETRY, SCREEN")
67+
}
68+
const indexes = validateSnapshots(packets, expectedGeometries, expectedGeometries.length)
69+
if (indexes.length !== expectedGeometries.length) {
70+
throw new Error(`expected ${expectedGeometries.length} snapshots, got ${indexes.length}`)
71+
}
72+
const initial = packets[indexes[0] + 1].payload
73+
const reconnected = packets[indexes[1] + 1].payload
74+
const coloredMarker = Buffer.from("\x1b[31mINITIAL_COLOR_61e8")
75+
if (!initial.includes(coloredMarker) || !reconnected.includes(coloredMarker)) {
76+
throw new Error("snapshot lost the red SGR state around the initial marker")
77+
}
78+
if (!reconnected.includes(Buffer.from("AFTER_DROP_61e8"))) {
79+
throw new Error("reconnect screen is not the current terminal state")
80+
}
81+
const exits = packets.filter((packet) => packet.type === 4)
82+
if (exits.length !== 1 || packets.at(-1).type !== 4) throw new Error("stream does not end in one EXIT")
83+
if (!packets.some((packet) => packet.type === 0 && packet.payload.includes(Buffer.from("FINAL_DATA_61e8")))) {
84+
throw new Error("final terminal DATA was not ordered before EXIT")
85+
}
86+
if (packets.some((packet) => ![0, 4, 5, 10].includes(packet.type))) {
87+
throw new Error("unexpected packet type in machine stream")
88+
}
89+
if (stdout.length !== 0) throw new Error("machine attach wrote to stdout")
90+
for (const marker of ["INITIAL_COLOR_61e8", "AFTER_DROP_61e8", "FINAL_DATA_61e8"]) {
91+
if (stderr.includes(Buffer.from(marker))) throw new Error(`terminal marker leaked to stderr: ${marker}`)
92+
}
5493
}
55-
if (packets.some((packet) => ![0, 4, 5, 10].includes(packet.type))) {
56-
throw new Error("unexpected packet type in machine stream")
94+
95+
const selfTest = () => {
96+
const expected = parseGeometries("24x80,13x47")
97+
const colored = Buffer.from("\x1b[31mINITIAL_COLOR_61e8\x1b[0m")
98+
const current = Buffer.concat([colored, Buffer.from("\r\nAFTER_DROP_61e8")])
99+
const packets = [
100+
frame(10, geometry(24, 80)), frame(5, colored),
101+
frame(10, geometry(13, 47)), frame(5, current),
102+
frame(0, Buffer.from("FINAL_DATA_61e8")), frame(4, Buffer.alloc(0)),
103+
]
104+
const valid = Buffer.concat(packets)
105+
validateFinal(valid, expected)
106+
const uncolored = Buffer.from("\x1b[HINITIAL_COLOR_61e8\x1b[0m")
107+
const mutations = [
108+
() => validateFinal(Buffer.concat([frame(10, geometry(1, 1)), ...packets.slice(1)]), expected),
109+
() => validateFinal(Buffer.concat([frame(10, geometry(24, 80)), frame(5, uncolored), ...packets.slice(2)]), expected),
110+
() => validateFinal(Buffer.concat([packets[0], packets[1], packets[2], frame(5, colored), ...packets.slice(4)]), expected),
111+
() => validateFinal(Buffer.concat([...packets.slice(0, 4), packets[5], packets[4]]), expected),
112+
() => validateFinal(valid.subarray(0, valid.length - 1), expected),
113+
() => validateFinal(valid, expected, Buffer.from("unexpected")),
114+
() => validateFinal(valid, expected, Buffer.alloc(0), Buffer.from("FINAL_DATA_61e8")),
115+
]
116+
for (const mutate of mutations) assert.throws(mutate)
117+
console.log("ORACLE-MUTATIONS-GREEN-61e8")
57118
}
58119

59-
console.log("PACKAGED-FD-GREEN-61e8")
60-
console.log("INITIAL-SNAPSHOT-GREEN-61e8")
61-
console.log("RECONNECT-SNAPSHOT-GREEN-61e8")
62-
console.log("FRAMED-TERMINAL-STREAM-GREEN-61e8")
120+
if (process.argv[2] === "--self-test") {
121+
selfTest()
122+
} else {
123+
const [path, mode, expectedRaw, geometryRaw, stdoutPath, stderrPath] = process.argv.slice(2)
124+
if (!path || !mode || !expectedRaw || !geometryRaw) process.exit(2)
125+
const data = fs.existsSync(path) ? fs.readFileSync(path) : Buffer.alloc(0)
126+
const expected = Number(expectedRaw)
127+
const expectedGeometries = parseGeometries(geometryRaw)
128+
if (mode === "snapshots") {
129+
validateSnapshots(decode(data, false), expectedGeometries, expected)
130+
} else if (mode === "final") {
131+
if (!stdoutPath || !stderrPath) process.exit(2)
132+
validateFinal(data, expectedGeometries, fs.readFileSync(stdoutPath), fs.readFileSync(stderrPath))
133+
console.log("PACKAGED-FD-GREEN-61e8")
134+
console.log("INITIAL-SNAPSHOT-GREEN-61e8")
135+
console.log("RECONNECT-SNAPSHOT-GREEN-61e8")
136+
console.log("FRAMED-TERMINAL-STREAM-GREEN-61e8")
137+
} else {
138+
process.exit(2)
139+
}
140+
}

‎cells/pty-attach-machine-stream/fixture/stream.sh‎

Lines changed: 15 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@ stderr="$root/attach.stderr"
1111
attach_pid=""
1212
remote_pid=""
1313
proxy_pid=""
14+
small_pid=""
1415
export PTY_ROOT="$pty_root"
1516

1617
pty_at() {
@@ -22,7 +23,7 @@ cleanup() {
2223
kill "$attach_pid" >/dev/null 2>&1 || true
2324
wait "$attach_pid" >/dev/null 2>&1 || true
2425
fi
25-
for pid in "$proxy_pid" "$remote_pid"; do
26+
for pid in "$small_pid" "$proxy_pid" "$remote_pid"; do
2627
if test -n "$pid"; then
2728
kill "$pid" >/dev/null 2>&1 || true
2829
wait "$pid" >/dev/null 2>&1 || true
@@ -77,40 +78,45 @@ env -u PTY_SESSION PTY_ROOT="$PTY_ROOT" \
7778
3>"$stream" >"$stdout" 2>"$stderr" &
7879
attach_pid="$!"
7980

80-
wait_for "initial machine snapshot" node "$root/check-stream.mjs" "$stream" snapshots 1
81+
wait_for "initial machine snapshot" node "$root/check-stream.mjs" "$stream" snapshots 1 24x80,13x47
8182
touch "$root/drop-first"
8283
wait_for "second fabric dial" test -f "$root/fabric-state/second-dial"
8384

8485
pty_at send ms.target --seq AFTER_DROP_61e8 --seq key:return
8586
wait_for "post-drop target output" peek_has AFTER_DROP_61e8
87+
env -u PTY_SESSION PTY_ROOT="$PTY_ROOT" \
88+
script -qefc 'stty rows 13 cols 47; exec pty attach --no-restart ms.target' /dev/null \
89+
>"$root/small.stdout" 2>"$root/small.stderr" &
90+
small_pid="$!"
91+
wait_for "smaller attached client" grep -Fq INITIAL_COLOR_61e8 "$root/small.stdout"
8692
touch "$root/fabric-state/release-second"
8793

88-
wait_for "reconnect machine snapshot" node "$root/check-stream.mjs" "$stream" snapshots 2
94+
wait_for "reconnect machine snapshot" node "$root/check-stream.mjs" "$stream" snapshots 2 24x80,13x47
8995
pty_at send ms.target --seq EXIT_61e8 --seq key:return
9096

9197
wait_for "attach process exit" sh -c "! kill -0 '$attach_pid' 2>/dev/null"
92-
if ! wait "$attach_pid"; then
98+
if wait "$attach_pid"; then
99+
attach_status=0
100+
else
93101
attach_status="$?"
94102
printf 'machine attach exited %s\n' "$attach_status" >&2
95103
sed -n '1,120p' "$stderr" >&2
96104
exit "$attach_status"
97105
fi
98106
attach_pid=""
99107

100-
node "$root/check-stream.mjs" "$stream" final > "$root/stream-proof"
108+
node "$root/check-stream.mjs" "$stream" final 2 24x80,13x47 "$stdout" "$stderr" > "$root/stream-proof"
101109
grep -Fqx PACKAGED-FD-GREEN-61e8 "$root/stream-proof"
102110
grep -Fqx INITIAL-SNAPSHOT-GREEN-61e8 "$root/stream-proof"
103111
grep -Fqx RECONNECT-SNAPSHOT-GREEN-61e8 "$root/stream-proof"
104112
grep -Fqx FRAMED-TERMINAL-STREAM-GREEN-61e8 "$root/stream-proof"
105-
test ! -s "$stdout"
106-
! grep -Fq INITIAL_COLOR_61e8 "$stderr"
107-
! grep -Fq AFTER_DROP_61e8 "$stderr"
108113
cat "$root/stream-proof"
109114

110-
for pid in "$proxy_pid" "$remote_pid"; do
115+
for pid in "$small_pid" "$proxy_pid" "$remote_pid"; do
111116
kill "$pid" >/dev/null 2>&1 || true
112117
wait "$pid" >/dev/null 2>&1 || true
113118
done
119+
small_pid=""
114120
proxy_pid=""
115121
remote_pid=""
116122
pty_at kill ms.target >/dev/null 2>&1 || true

‎cells/pty-attach-machine-stream/pty-attach-machine-stream.kdl‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,9 @@ eval {
77
run "stream" {
88
command "bash ./stream.sh"
99
}
10+
run "oracle-mutations" {
11+
command "node ./check-stream.mjs --self-test"
12+
}
1013

1114
judges {
1215
judge "LAUNCHER - the shipped pty executable preserves the caller-owned inherited descriptor" {

‎docs/vrs/spec.md‎

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -47,11 +47,14 @@ tracked cell
4747
- **R01, R04, R05, R07, R11:** The model-free
4848
`pty-attach-machine-stream` cell composes the installed PTY launcher, target
4949
daemon, remote route, forced transport replacement, and caller-owned framed
50-
descriptor. Its held-out judges require ordered `GEOMETRY`, `SCREEN`, live
51-
`DATA`, and terminal `EXIT` frames without stdout or stderr contamination,
52-
then require removal of every eval-owned process and PTY session. The fixture
53-
controls route selection and transport failure but does not import PTY source
54-
modules or bypass the packaged launcher.
50+
descriptor. Its held-out judges require exact initial and min-wins reconnect
51+
geometry, preserved SGR color state, current `SCREEN`, live `DATA`, and one
52+
terminal `EXIT` without stdout or stderr contamination, then require removal
53+
of every eval-owned process and PTY session. A model-free mutation matrix
54+
rejects wrong geometry, stripped color, stale reconnect state, misordered or
55+
truncated frames, and side-channel terminal bytes. The fixture controls route
56+
selection and transport failure but does not import PTY source modules or
57+
bypass the packaged launcher.
5558

5659
## Current execution
5760

0 commit comments

Comments
 (0)