Skip to content

Commit 80e42a9

Browse files
phodalQoder-AI
andcommitted
fix(harness): assert redacted tool path per host separator
The experiment runner test failed only on windows-latest, expecting `<trial-root>/README.md` while the run emitted `<trial-root>\README.md`. The fault was in the test, not the runner. Its fake executor builds `input.path` with native `join(context.worktree, "README.md")`, and `redactEvidenceValue` deliberately rewrites just the trial-root prefix and leaves separators alone. Normalizing separators inside the runner would be wrong: tool payloads are opaque host data, so a blanket backslash rewrite would corrupt code content, regex literals, and Windows command lines. Replace the stringified-blob `toContain` with a structural assertion on the `tool-call-started` event's `input.path`, normalizing separators at that boundary. This matches the existing precedent in compare.test.ts and asserts on parsed event shape rather than on text that happens to contain it. Also close a latent gap in the neighbouring leak guard: `JSON.stringify` escapes backslashes, so `not.toContain(root)` could never match a leaked native Windows root and passed vacuously there. It now also checks the JSON-escaped form, which is equivalent to the raw check on POSIX. Validated with the full harness package suite (172 tests, 20 files) and `tsc --noEmit`. Since a local macOS pass is not Windows evidence, the runner's redaction logic was also replayed against `path.win32` drive and UNC roots plus posix; the win32 form reproduces the exact CI symptom and normalizes correctly. The windows-latest job remains the authoritative receipt. Co-authored-by: QoderAI (Qwen 3.8 Max) <qoder_ai@qoder.com>
1 parent 684e727 commit 80e42a9

1 file changed

Lines changed: 9 additions & 2 deletions

File tree

packages/harness/test/experiment-runner.test.ts

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -174,8 +174,15 @@ describe("checkpoint experiment runner", () => {
174174
expect.objectContaining({ experimentId: "exp_parallel_test", laneId: "default", runId: "exp_parallel_test:default:1" }),
175175
expect.objectContaining({ experimentId: "exp_parallel_test", laneId: "minimal", runId: "exp_parallel_test:minimal:1" }),
176176
]));
177-
expect(JSON.stringify(events)).not.toContain(root);
178-
expect(JSON.stringify(events)).toContain("<trial-root>/README.md");
177+
const serializedEvents = JSON.stringify(events);
178+
expect(serializedEvents).not.toContain(root);
179+
expect(serializedEvents).not.toContain(root.replaceAll("\\", "\\\\"));
180+
const callEvent = events.find((event) => event.type === "lane-event"
181+
&& event.event.type === "tool-call-started");
182+
// Redaction rewrites the trial root only; tool inputs keep host-native separators.
183+
expect(callEvent?.type === "lane-event" && callEvent.event.type === "tool-call-started"
184+
? (callEvent.event.input as { path: string }).path.replaceAll("\\", "/")
185+
: null).toBe("<trial-root>/README.md");
179186
const resultEvent = events.find((event) => event.type === "lane-event"
180187
&& event.event.type === "tool-call-result");
181188
expect(resultEvent?.type === "lane-event" && resultEvent.event.type === "tool-call-result"

0 commit comments

Comments
 (0)