mcp-schema-normalize - a gateway-side normalizer for the documented json-schema-to-grammar limitations

[rsclafani]

I kept hitting Unable to generate parser for this template / Error resolving ref … anyOf not in {…} when routing MCP tool calls through llama-server, and traced every failure back to the limitations already documented in grammars/README.md.

Rather than ask the converter to change, I wrote a normalizer that sits in front of it and rewrites tool schemas into the subset it accepts.
Sharing it here in case others on local backends hit the same wall.

PyPI: pip install mcp-schema-normalize · Repo: https://github.com/rsclafani/mcp-schema-normalize · MIT

The gap

MCP mandates JSON Schema 2020-12 (SEP-1613). json-schema-to-grammar.cpp compiles a deliberately narrower subset. Schemas that pass fine against hosted Anthropic/OpenAI APIs (which forward them verbatim) die at the grammar step on a local backend. The same tool list that works on a cloud model will error (400) on Qwen3-Coder or Nemotron-Nano behind llama-server.

These aren't bugs waiting on a fix, they're documented, accepted limitations. The library is a workaround for the documented gap, not a stopgap for something pending. It maps directly onto the converter's own limitation list:

Failure mode	What the library does
anyOf / oneOf beside properties/type/required (#7703)	Distributes siblings into each union branch → self-contained objects
Nested $refs into anyOf nodes (#8073)	Inlines non-cyclic refs; leaves cycles in place (the converter handles those natively)
{"not": {}} sentinels from zod-to-json-schema (#17574)	Drops empty-not; preserves real not schemas
Schemas expanding past the size threshold (#21228)	Coarsens inlines that would blow the budget
Silent fallback to unconstrained generation on grammar-build failure (#19051)	Pre-flight size budget + telemetry to make the silent fallback visible
Dangling $ref (a zod-to-json-schema singleton-union artifact)	Replaces with permissive {} so the request completes — see the caveat below

The honest caveat

The dangling-$ref case is upstream-schema corruption, not a llama.cpp issue: zod-to-json-schema collapses singleton z.union(...) to its sole variant but still emits $ref strings pointing at the pre-collapse anyOf envelope. The library makes the request go through by swapping the unresolvable ref for {} (match-anything)... which means that field loses its type spec. Every such event increments a refs_unresolved counter and logs a WARN line. If you can't watch telemetry for it, set STRICT_UNRESOLVED_REFS = True and it fails loud (400) instead of degrading quietly.

The README's "When NOT to use this" section is explicit about this tradeoff with the intent to scare off a misuse than earn a reputation for silently loosening validation.

Where it runs

Pure-Python core, zero runtime deps, operates on plain JSON Schema so it works with direct llama-server, Ollama, or any OpenAI-compatible client. First-class LiteLLM proxy hook ships behind a [litellm] extra. vLLM/TabbyAPI adapters are a thin wrapper around normalize_tools(); PRs welcome.

Feedback I'm after

• If you've hit these on a different MCP server, I'd welcome the broken schema as a test case. The more examples I see, the better the normalizer gets.
• @ochafik, you maintain the grammars README that documents these limitations. Would a one-line pointer to a gateway-side normalizer be welcome in the "JSON schemas → GBNF" section, or do you consider that out of scope for the docs? Happy to open a PR if useful, but wanted to check before I do.