v0.8.3 — JSONL record integrity, UTF-8 report, encode-before-spawn, \Z role ids - #7
Conversation
…fore-spawn; \Z role ids Four robustness fixes surfaced by the council review (v0.8.3). All four were reproduced before fixing. - _iter_json_objects split on "\n" instead of str.splitlines(): splitlines also breaks on U+2028/U+2029/U+0085, which are legal *unescaped* inside a JSON string. codex can emit an agent_message containing one literally, and splitting there tore the record into invalid fragments — silently dropping a completed reply and reporting the role as failed. - _force_utf8_streams() pins stdout/stderr to UTF-8 at startup. Under a strict C locale with UTF-8 mode and C-coercion both disabled, the report header's em dash raised UnicodeEncodeError and lost BOTH the report and the CODEX_COUNCIL_DONE sentinel; ASCII input alone still triggered it. - _run_codex_subprocess encodes the prompt BEFORE create_subprocess_exec and reaps the child on any post-spawn BaseException, not just CancelledError. A lone surrogate (e.g. escaped "\uD800" in roles.json) previously raised after the child existed, leaking a codex process blocked forever on stdin. - ROLE_ID_PATTERN uses \Z instead of $, so "architect\n" (which $ accepts, matching before a trailing newline) can no longer inject a newline into state filenames and report/progress lines. Adds regression tests for all four (unit + a strict-C-locale CLI test); 268 passing, ruff clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
There was a problem hiding this comment.
Code Review
This pull request introduces several robustness improvements to the Codex Council plugin, including updating the role ID regex to use \Z to prevent trailing newlines, splitting JSONL streams strictly on \n to preserve valid Unicode line separators, encoding prompts before spawning subprocesses to avoid process leaks, catching BaseException during subprocess communication for reliable cleanup, and forcing stdout/stderr streams to UTF-8 to prevent encoding errors in strict C locales. The review feedback suggests broadening the exception handling in _force_utf8_streams to catch Exception instead of just (ValueError, OSError) to ensure compatibility with custom or mock stream implementations, along with a corresponding test update.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| except (ValueError, OSError): | ||
| pass |
There was a problem hiding this comment.
Since _force_utf8_streams is a best-effort initialization step, it should be extremely robust against any custom or mock stream implementations. Standard streams or third-party wrappers might raise other exceptions like TypeError (if the signature of reconfigure is different or doesn't accept these keyword arguments) or AttributeError. Catching Exception instead of just (ValueError, OSError) ensures that the CLI tool never crashes at startup due to stream reconfiguration failures.
| except (ValueError, OSError): | |
| pass | |
| except Exception: | |
| pass |
| class Boom: | ||
| def reconfigure(self, **kw): | ||
| raise ValueError("nope") |
There was a problem hiding this comment.
To verify that _force_utf8_streams is robust against other exceptions (such as TypeError which can occur if a custom stream's reconfigure method has an incompatible signature), we should update this test to raise a TypeError instead of ValueError.
| class Boom: | |
| def reconfigure(self, **kw): | |
| raise ValueError("nope") | |
| class Boom: | |
| def reconfigure(self, **kw): | |
| raise TypeError("nope") |
v0.8.3 — JSONL record integrity, UTF-8 report, encode-before-spawn, \Z role ids
Four robustness fixes from the council review. Each was reproduced before fixing; regression tests added for all four (268 passing, ruff clean).
Fixes (
codex_council.py)U+2028/U+2029/U+0085no longer dropped —_iter_json_objectsusedstr.splitlines(), which breaks on those three chars even though they're legal unescaped inside a JSON string. codex can emit anagent_messagecontaining one literally, so the record was torn into invalid fragments and the completed reply silently discarded → role reported FAILED. Now splits strictly on"\n"._force_utf8_streams()pins stdout/stderr to UTF-8 at startup. UnderLC_ALL=C PYTHONUTF8=0 PYTHONCOERCECLOCALE=0, stdout defaulted to ASCII and the report header's em dash raisedUnicodeEncodeError, losing both the report and theCODEX_COUNCIL_DONEsentinel (even with ASCII role replies)._run_codex_subprocessnow encodes the prompt beforecreate_subprocess_execand reaps the child on any post-spawnBaseException, not justCancelledError. A lone surrogate (e.g. escaped\uD800in roles.json) previously raised after the child existed, leaving codex blocked forever on stdin.ROLE_ID_PATTERNuses\Zinstead of$(which matches before a final\n), so"architect\n"no longer reaches state filenames / report / progress lines.Tests
_force_utf8_streamsbehavior, trailing/embedded-newline role ids.LC_ALL=C PYTHONUTF8=0 PYTHONCOERCECLOCALE=0asserting exit 0, report + em dash present, and the genuine sentinel.Follows v0.8.2 (#6).
🤖 Generated with Claude Code