Commit 2c6b3b7
committed
chore(release): v0.9.0 - SGLang-driven serving hardening + OOM crash guard
Highlights since v0.8.1:
- response_format: clean schema-shaped JSON on both streaming and
non-streaming, Gemma4 AND Qwen3.6 (first-JSON-value truncation +
Eager schema grammar). Fixes silent free-text on non-streaming.
- Tool-call grammar enforcement: tool_choice=required/named now
structurally fills required params on quantized Gemma4 + Qwen3.6
(kills the empty-parameter infinite-loop defect). Plus a Qwen
vocab-padding fix that makes its grammar actually enforce.
- Robustness: prompt-size reject cap now guards ALL MLX chat paths
(OpenAI + Anthropic, stream + non-stream) so an oversized prompt
returns a clean error instead of crashing the whole server via an
uncaught Metal OOM. Safer default (16384).
- GET /v1/loads JSON observability; multi-entry LPM prefix cache for
branching agentic sessions; MLX native decode overlap.
- App: memory calculator with model-aware tuning + budget auto-optimize.
Verified on Gemma-4-26b-nvfp4 + Qwen3.6-35B-mxfp4 (e2e suite 10/10, 8/8).1 parent 300b2b6 commit 2c6b3b7
4 files changed
Lines changed: 4 additions & 4 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | | - | |
| 3 | + | |
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
4 | | - | |
| 4 | + | |
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
4 | | - | |
| 4 | + | |
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
| |||
0 commit comments