Schema-blind per-parameter json.loads() fallback (3 branches: Qwen/Llama, GLM, MiniMax) silently drops array/object tool-call params to raw strings on minor JSON syntax errors

[bobacus]

Problem

When the native tool-call parser fails on a model completion with slightly malformed JSON inside a parameter value (a real but minor model generation defect — e.g. a bracket mismatch), omlx/api/tool_calling.py's XML fallback parsers recover the tool call itself but silently keep the malformed parameter as a raw string instead of the array/object it's supposed to be. Downstream clients that validate tool-call arguments against the declared JSON schema then reject the call:

Validation failed for tool "edit":
  - edits.0: must be object
Received arguments: {"edits": "[{...}]", ...}

i.e. the array-typed edits parameter arrives JSON-encoded as a string instead of a native array.

Diagnosis

Root cause — one pattern duplicated across 3 branches

The same defective pattern — parse each parameter/arg value independently with a bare try: json.loads(val) except: val, no knowledge of the tool's declared schema — is copy-pasted across three separate parser branches:

# omlx/api/tool_calling.py, _parse_xml_tool_calls(), Qwen/Llama branch, ~line 183
for pm in re.finditer(
    r"<parameter=(\w+)>\s*(.*?)\s*</parameter>", params_text, re.DOTALL
):
    key = pm.group(1)
    val = pm.group(2).strip()
    try:
        arguments[key] = json.loads(val)
    except (json.JSONDecodeError, ValueError):
        arguments[key] = val   # <-- kept as a raw string, no repair, no schema

The identical pattern also appears in:

• _parse_xml_tool_calls()'s GLM branch (<arg_key>/<arg_value>, ~line 213)
• _parse_namespaced_tool_calls() (MiniMax <invoke name=...><parameter name=...>, ~line 268)

A fourth, related-but-different spot, _parse_bracket_tool_calls() ([Calling tool: name(args)] format, ~line 400), wraps a parse failure in {"raw": args_str} instead of mistyping a specific field — still not schema-aware, but a different failure mode; flagging it, not claiming it's the same bug.

Contrast with parse_tool_calls() (the primary path all of these fall back from), which does thread tools through to mlx_lm's native parser so it can correctly type-coerce declared array/object parameters. These three fallbacks take no tools parameter at all, so a value that's almost valid JSON (just malformed) degrades silently into a string instead of being repaired or rejected with a clear error.

Gemma 4 has its own separate, more sophisticated repair chain for its call:name{key:val} syntax (_gemma4_args_to_json_robust/ _gemma4_transcode_to_json) that doesn't have this defect — which is presumably why it doesn't show this corruption in my data (see below); it goes through a different code path entirely.

Minimal repro (no model or server needed — pure function call against the currently-installed package):

from omlx.api.tool_calling import _parse_xml_tool_calls
import json

malformed = '''<tool_call>
<function=edit>
<parameter=path>
config.py
</parameter>
<parameter=edits>
[{"range": ["3#8C", "3#8C"], "lines": ["    value: 100,"}]}
</parameter>
</function>
</tool_call>'''

_, calls = _parse_xml_tool_calls(malformed)
parsed = json.loads(calls[0].function.arguments)
print(type(parsed["edits"]))   # <class 'str'>, should be <class 'list'>
print(parsed["edits"])         # '[{"range": ["3#8C", "3#8C"], "lines": ["    value: 100,"}]}'

Confirmed the identical result calling the GLM branch and _parse_namespaced_tool_calls() directly with equivalent malformed input — all three branches reproduce the same corruption via the same root cause. The malformed input itself (a real }/] mismatch) is exactly the shape observed live from actual model output, not a contrived edge case.

Real-world evidence

~/.omlx/logs/server.log* already logs this failure mode at WARNING level. Across 4 days of logs I found 23 total occurrences, splitting cleanly by tool-call format:

• 19/19 Native tool parser failed (...), recovered via XML fallback entries use the <function=name><parameter=key>... format (this issue), all from Qwen3.5/3.6-family models I was running (prism-ml/Ternary-Bonsai-27B-mlx-2bit, mlx-community/Qwen3.6-35B-A3B-OptiQ-4bit). Every one shows this exact corruption — not hypothetical.
• 4/4 ... and XML fallback could not recover. Dropping match entries use Gemma 4's call:name{key:val} format — a different symptom (dropped entirely, not type-corrupted) via Gemma 4's separate repair chain; different bug, out of scope here.

Real agentic sessions doing file edits (longer content, more escaping, multi-line strings) give the model's baseline rate of small JSON mistakes enough chances to fire that this becomes a real reliability problem, while short/simple test prompts rarely trigger it.

Environment

• oMLX 0.5.3 (also present in 0.5.1; unchanged across that range)
• Models observed triggering this: prism-ml/Ternary-Bonsai-27B-mlx-2bit, mlx-community/Qwen3.6-35B-A3B-OptiQ-4bit
• Client: Pi (@earendil-works/pi-coding-agent) via the OpenAI-compatible /v1/chat/completions endpoint — but this is a server-side bug, reproducible with no client at all (see repro above)

Suggested fix

Since the pattern is duplicated three times, a shared helper seems better than three separate patches:

1. Thread tools through _parse_xml_tool_calls() and _parse_namespaced_tool_calls() (mirroring what parse_tool_calls()
   already does for the native parser) so they can type-coerce per the declared schema.
2. Attempt a real repair pass (e.g. a JSON-repair library) on parameter values that fail strict json.loads(), instead of falling back to the verbatim string.
3. At minimum, don't silently succeed with a wrong type — if a value looks like it was meant to be an array/object but fails to parse, surface a clear parse error instead of a validation failure two layers away.

Related

• [bug] JSONDecodeError: Invalid control character when agent calls tool without sanitising tab characters #1672 is the closest existing issue — same function, same log signature — fixed same-day by @jundot in 64a1b8c. That fix only
  touches the generic {"name": ..., "arguments": ...} top-level JSON branch, not the three branches this issue is about.
• Tool call output as raw XML instead of internal tool invocation with qwen3.6-27b #2221 and Qwen3.6-35B-A3B-8bit + Hermes ，bug #854 are separate Qwen3.6 tool-calling issues (streaming state-machine detection, and a multi-turn replay crash respectively) — related in spirit, different bugs.

Happy to help test a fix — have real session logs and standalone repros against all three affected branches if useful.

[jundot]

Thanks for the great writeup, the standalone repro made this easy to confirm. You were right about the root cause, the fallback parsers never see the tool schema, so a broken array param just stayed as a plain string. Fixed in 093e7df. The fallbacks now get the schema and try a small bracket repair on array/object params, so your example comes back as a real array. If you can run your agent workload against the next build and let me know how it goes, that would help a lot.

[bobacus]

I will check now. I should probably acknowledge Claude for the great write-up (and helping to diagnose this issue)

[bobacus]

Yes, this fix addresses the problem. Tool calling reliability is now much better. Thanks!