Skip to content

fix: make_json_serializable must not reinterpret JSON-looking strings#2524

Open
chuenchen309 wants to merge 1 commit into
huggingface:mainfrom
chuenchen309:fix/make-json-serializable-preserves-string-type
Open

fix: make_json_serializable must not reinterpret JSON-looking strings#2524
chuenchen309 wants to merge 1 commit into
huggingface:mainfrom
chuenchen309:fix/make-json-serializable-preserves-string-type

Conversation

@chuenchen309

Copy link
Copy Markdown

What

make_json_serializable() opportunistically ran json.loads() on any string starting with {/[ and ending with }/], silently replacing the original string with the parsed dict/list. This changes the Python type of a value based on its content, which breaks the "make this object JSON-serializable" contract — a string is already JSON-serializable and should never need reinterpreting.

>>> make_json_serializable({"content": '{"key": "value"}', "path": "out.json"})
{"content": {"key": "value"}, "path": "out.json"}   # before this fix - "content" silently became a dict

This is reachable through ToolCall.dict() (memory.py), which feeds both the documented RunResult.steps API and, more importantly, ActionStep.to_messages()'s "Calling tools:\n" + str([tc.dict() for tc in self.tool_calls]) — the actual conversation history replayed back to the model. Any tool whose argument is a plain string that happens to look like JSON (e.g. a file-writing tool given literal JSON text to write) has that argument silently mutated from str to dict in what the model sees of its own prior tool calls.

Fix

Removed the JSON-sniffing block so strings pass through unchanged.

Testing

Added test_make_json_serializable_preserves_json_looking_strings to tests/test_utils.py. Confirmed via git stash that it fails against the pre-fix code (string values get silently turned into dict/list) and passes after.

  • pytest tests/test_utils.py tests/test_memory.py — 45 + 12 passed, no regressions.
  • ruff check and ruff format --check both clean on the changed files.

AI disclosure

Developed with AI assistance (Claude Code), which located the bug via a targeted search for functions with no test coverage and traced how ToolCall.dict()'s output reaches both the public RunResult.steps API and the model-facing conversation history. I reviewed the diff, independently reproduced the type-mutation bug and confirmed the fix by executing both against the actual function, and ran the tests/linters myself before opening this PR.

make_json_serializable() opportunistically ran json.loads() on any string
starting with "{"/"[" and ending with "}"/"]", silently replacing the
original string with the parsed dict/list. This changes the Python type
of the value based on its content, which breaks the "make this object
JSON-serializable" contract - a string is already JSON-serializable and
should never need reinterpreting.

This is reachable through ToolCall.dict() (memory.py), which feeds both
the documented RunResult.steps API and, more importantly,
ActionStep.to_messages()'s "Calling tools:\n" + str([tc.dict() for tc in
self.tool_calls]) - the actual conversation history replayed back to the
model. Any tool whose argument is a plain string that happens to look
like JSON (e.g. a file-writing tool given literal JSON text to write) has
that argument silently mutated from str to dict in what the model sees
of its own prior tool calls.

Removed the JSON-sniffing block so strings pass through unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant