Summary
The backend writes run_state.json and the report's meta.json with raw, unescaped control characters embedded in LLM-generated text fields (agent action strings, report markdown). The resulting files fail standard parsing (json.load raises), which breaks any downstream tooling that consumes run artifacts. We had to fall back to regex extraction to build our run manifests.
Reproduction
- Complete (or partially complete) any simulation run whose agents emit multi-line or control-character-containing text — in practice, nearly every run.
python -c "import json; json.load(open('run_state.json'))" → json.decoder.JSONDecodeError (invalid control character).- Same for the generated report's
meta.json.
Observed
- Unescaped control characters (raw newlines/tabs and similar) inside JSON string values in
action and markdown-carrying fields.
- Standard-library and strict parsers reject the files; only lenient/regex extraction works. This affected effectively all of our 18 runs' manifest tooling.
Expected
All emitted .json artifacts should be valid JSON parseable by json.load with default (strict) settings.
Suggested fix
Serialize these artifacts with proper JSON encoding rather than string templating/concatenation — i.e. build a dict and json.dump(obj, f, ensure_ascii=False) so the encoder escapes control characters. If any field is currently interpolated into a JSON template as a raw string, route it through json.dumps instead. A round-trip test (json.load on every emitted artifact) in CI would prevent regressions.
Context / disclosure: We are an independent research team auditing multi-agent simulation pipelines. We found this while running a systematic 18-run study on MiroFish-Offline across four LLM families. We plan to publish a reproducible study referencing this issue in roughly 4 weeks, and we're happy to share drafts with maintainers beforehand. Thank you for open-sourcing this project — it made this kind of research possible in the first place.
Summary
The backend writes
run_state.jsonand the report'smeta.jsonwith raw, unescaped control characters embedded in LLM-generated text fields (agentactionstrings, report markdown). The resulting files fail standard parsing (json.loadraises), which breaks any downstream tooling that consumes run artifacts. We had to fall back to regex extraction to build our run manifests.Reproduction
python -c "import json; json.load(open('run_state.json'))"→json.decoder.JSONDecodeError(invalid control character).meta.json.Observed
actionand markdown-carrying fields.Expected
All emitted
.jsonartifacts should be valid JSON parseable byjson.loadwith default (strict) settings.Suggested fix
Serialize these artifacts with proper JSON encoding rather than string templating/concatenation — i.e. build a dict and
json.dump(obj, f, ensure_ascii=False)so the encoder escapes control characters. If any field is currently interpolated into a JSON template as a raw string, route it throughjson.dumpsinstead. A round-trip test (json.loadon every emitted artifact) in CI would prevent regressions.Context / disclosure: We are an independent research team auditing multi-agent simulation pipelines. We found this while running a systematic 18-run study on MiroFish-Offline across four LLM families. We plan to publish a reproducible study referencing this issue in roughly 4 weeks, and we're happy to share drafts with maintainers beforehand. Thank you for open-sourcing this project — it made this kind of research possible in the first place.