fix(daemon): decode raw stdout Buffer chunks safely in the shared JSON-line stream parser

[vecsop]

Fixes #

Why

agent-protocol/core/json-line-stream.ts's createJsonLineStream().feed() accumulates incoming chunks with buffer += chunk. Two of its real callers hand it raw, non-setEncoding'd stdout Buffer chunks: agent-protocol/pi-rpc/session.ts (used by the pi adapter) never calls setEncoding on stdout and, before this PR, additionally ran each chunk through its own chunk.toString('utf8') ahead of feed; agent-protocol/acp/session.ts (used by every ACP-backed adapter) calls setEncoding('utf8') on child.stderr but never on child.stdout — the pre-existing (chunk: string) parameter annotation on that handler was misleading, the runtime value is a raw Buffer.

Decoding a Buffer chunk independently of its neighbors — whether via chunk.toString('utf8') or the implicit coercion buffer += chunk performs — has no memory of a previous chunk's trailing bytes. A multi-byte UTF-8 character (any CJK character, most emoji) whose bytes straddle a chunk boundary gets corrupted into U+FFFD on each side of the split independently. Assistant text and tool-call arguments streamed over these two paths can contain arbitrary UTF-8, so this is a live, reproducible corruption bug for any non-ASCII content that happens to land on a chunk boundary, not a contrived edge case.

What users will see

• ACP-backed adapters (e.g. AMR/Vela) and the pi adapter no longer occasionally corrupt CJK/emoji characters in streamed assistant text into �.
• No other user-visible change — the parser's line-buffering, multiline-JSON aggregation, and public API are otherwise unchanged.

Surface area

• None — internal correctness fix to an existing stream parser; no new UI, API, or config surface.

Files touched: the shared parser (json-line-stream.ts) plus the two call sites that needed a type/decode-ownership correction (pi-rpc/session.ts, acp/session.ts), and a new regression test file.

json-event-stream.ts / copilot-stream.ts / claude-stream.ts have the identical-looking feed(chunk: string) { buffer += chunk; ... } shape but are not touched here: their only real callers (server.ts, connectionTest.ts) already call setEncoding('utf8') before attaching a 'data' listener, and Node's StringDecoder (which setEncoding uses internally) is itself stream-safe for multi-byte sequences split across the underlying Buffer reads — by the time feed() sees a chunk on those paths, it's already a complete, valid string. Verified this with the same split-byte harness used for the two paths that do reproduce, before deciding what to touch, rather than assuming all four parsers were equally affected because they look the same. Happy to harden those three the same way in a follow-up if maintainers would rather not depend on every caller remembering setEncoding — kept this PR's diff limited to paths with a demonstrated failure.

Bug fix verification

• Test path: apps/daemon/tests/agent-protocol/json-line-stream-chunk-boundary.test.ts — two tests reproduce each real call site's exact shape (raw Buffer chunks split mid multi-byte-character): one modeled on pi-rpc/session.ts's stream (a CJK character split), one on acp/session.ts's (an emoji split). A third test pins the companion invariant: a caller that did call setEncoding('utf8') hands over an already-decoded string, which must pass through unchanged rather than be re-run through TextDecoder.decode (which expects raw bytes).
• Did the test go red on main and green on this branch? Yes. Confirmed by reverting feed's decode change and pi-rpc/session.ts's parser.feed(chunk) call locally and re-running: 2 of the 3 new tests fail with U+FFFD in the reassembled JSON on the pre-fix code, all 3 pass after.

Validation

• pnpm --filter @open-design/daemon typecheck — clean, 0 errors.
• pnpm guard — clean.
• pnpm exec vitest run -c vitest.config.ts tests/agent-protocol/json-line-stream-chunk-boundary.test.ts tests/acp.test.ts tests/pi-rpc.test.ts — 151/153 passed. The 2 failures (pi-rpc.test.ts symlink-upload tests) are EPERM: operation not permitted, symlink in my local sandbox (no Windows Developer Mode / symlink privilege) and reproduce identically on a clean main checkout with none of this PR's changes applied — pre-existing environment limitation, not a regression from this change.

[lefarcen]

Hey @vecsop — thanks for tightening the shared decoder path here; moving UTF-8 boundary handling into the shared parser is the right direction.

Could you fill in the PR body with Why, What users will see, Surface area, and Validation? I also added needs-validation because this touches a live daemon runtime path, so we'll want a QA pass before merge.

[mrcfps]

@vecsop Thanks for pinning the UTF-8 split at the shared parser instead of hoping every stdout listener remembers setEncoding.

I walked the changed ranges in createJsonLineStream, the pi-rpc and ACP session data handlers, and the new chunk-boundary tests. feed() now keeps one TextDecoder with { stream: true } for raw Buffer chunks, leaves already-decoded strings alone, and flush() ends the decoder so a truncated sequence becomes U+FFFD instead of disappearing. Removing pi-rpc's per-chunk toString('utf8') is necessary for that decoder to ever see the split bytes. The regression cases cover a CJK 3-byte split, an emoji 4-byte split, and a string caller that already went through setEncoding.

Nice, focused bugfix — appreciate the careful scoping away from the stream parsers that were already safe.

_{🔁 Powered by Looper · runner=reviewer · agent=omp · An autonomous AI dev team for your GitHub repos.}

[lefarcen]

Queued for QA validation — this change is on a live daemon runtime path, and now that the PR has review approval plus green CI, the next step is a manual QA pass before merge.