fix(proxy): normalize leaked GLM/qwen tool-call arg dialect from responses (#85) - #86
Merged
Conversation
…onses (#85) Some open-weight backends (z-ai/glm-5.2, qwen family) intermittently fail to decode their own tool-call argument encoding, leaving raw markup inside the structured `arguments` a well-formed native tool_calls entry returns: GLM: <arg_key>NAME</arg_key> <arg_value>VALUE</arg_value> qwen: <parameter=NAME>VALUE</parameter> The intended payload is intact inside the wrapper, so it's a parse gap not lost data. Normalize both the value-level leak (dialect inside one value of an otherwise-valid JSON object, the #85 symptom) and the whole-string leak, applied to each successful response before it is returned or logged, so no downstream consumer (client or log) sees the markup. Fully defensive: any parse failure leaves the value untouched. Repair count recorded as tool_call_dialect_repaired on the response log so the leak rate stays measurable. Durable server-side fix for the class geo-agent#276 defends against client-side.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #85.
Problem
z-ai/glm-5.2(and, per the same leak class, the qwen family) intermittently fail to decode their own tool-call argument encoding, leaving raw markup inside the structuredargumentsthat a well-formed nativetool_callsentry hands back. The verbatim symptom from the issue:The intended payload (
{"by_res": {…}}) is intact inside the wrapper — a serialization/parse gap, not lost data. The dialect reached the client (breaking tool execution) and the logs.Fix
A proxy-side normalization pass (
_normalize_response_tool_calls) applied to every successful response before it is returned or logged, so no downstream consumer — client or log — ever sees the markup. This is the durable server-side fix for the leak class that geo-agent#276 was defending against client-side.It handles two shapes:
argumentsis valid JSON but one value is wrapped in the dialect. The wrapper is stripped and the inner payload is re-parsed so structured values come back structured.argumentsstring is raw dialect; rebuilt into a JSON object by pairing each key tag with the value tag following it.Both the GLM (
<arg_key>/<arg_value>) and qwen/hermes (<parameter=NAME>…</parameter>) forms are covered, including unterminated (truncated) tags.Fully defensive: any parse failure leaves the value untouched — normalization must never corrupt a response or break serving.
Observability
The number of repaired arguments is recorded as
tool_call_dialect_repairedon the response log entry (and a🧹stdout breadcrumb), so the leak rate stays measurable even though the markup itself no longer reaches the logs.Tests
8 new tests in
test_logging.pycover value-level leak, unterminated tag, whole-string dialect, the qwen<parameter=>form, clean-passthrough (no wasted re-serialize), the in-place response pass + repair count, defensive handling of malformed shapes, and an end-to-endproxy_chatrun asserting the client receives structured data and the log records the count. Full suite: 31 passed.