Capture private OMP eval chat transcripts

[ericjuta]

Changes

• Capture ordered prompts, assistant text, tool calls and tool results through the stock OMP stream API. Exclude reasoning and raw provider frames.
• Write private, bounded, credential-redacted .transcripts-v1.jsonl companions with planned trial identity, mode 0600, fsync and exact report-byte binding.
• Retain partial timeout/interruption transcripts, cancel stalled persistence, and preserve existing journal and grading contracts.
• Expose opt-in library capture and document automatic CLI capture. Public aggregate and attempt schemas remain unchanged.

Commits

1. c8298d7 — bounded capture, redaction, callback lifetime and regressions.
2. 89199b8 — private persistence, CLI integration, exports and documentation.

Please merge with a merge commit to retain both commits.

Verification

• 370 tests passed across 33 files.
• Formatting, lint, typecheck, imports and package builds passed.
• bun run verify:artifacts, bun run check:public-boundary, bun run check:target-conformance, and bun run smoke:install passed on the clean committed tree.
• Stock OMP 18.1.17 synthetic local smoke: three trials, including one expected MCP failure; ordered transcript capture, redaction, private permissions, report hash binding and session cleanup verified. No live credentials or real model calls.
• Independent correctness/security review findings were repaired and re-reviewed. Three built-package reproducers failed before repair and passed afterward: assignment-prefixed private-key redaction, post-redaction label limits, and cancellation of a stalled writer before scope shutdown.

No measured evaluation publication or deployment claim.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated
plugins-evals	Skipped		Sep 11, 2026 8:20pm UTC