Summary
LM Studio 0.4.14 added MTP speculative decoding for the GGUF/llama.cpp path. The MLX engine has no equivalent: when a checkpoint ships built-in multi-token-prediction (nextn) heads, the MLX engine loads the model fine but does not use those heads, and gives no indication that it is skipping them.
On Apple Silicon this inverts the expected ranking. MLX has the better base decode, but ends up the slower option overall, purely because MTP exists only on the llama.cpp side.
Environment
- MacBook Pro, M5 Max (40-core GPU), 128 GB unified memory, ~614 GB/s
- LM Studio: current release as of 2026-08-20
- Model:
Youssofal/Qwen3.8-27B-MTPLX-Optimized-Speed (MLX, 19.85 GB, ships MTP heads)
- Sampling: temp 0.6 / top_p 0.95 / top_k 20, full GPU
Observed
These are from normal chat use, not a controlled harness. I'm reporting them as a signal about the gap, not as a benchmark:
| runtime |
MTP heads used |
decode |
| MTPLX (MLX runtime that uses the MTP heads, draft depth 2) |
yes |
60-80 tok/s |
| LM Studio, MLX engine, same checkpoint |
no |
~30 tok/s |
LM Studio / Unsloth, GGUF, --spec-type draft-mtp --spec-draft-n-max 2 --parallel 1 |
yes |
35-40 tok/s peak, 24-28 typical |
Happy to re-measure any row under a configuration you specify, on a fixed prompt set — just say what you want measured and I'll run it.
Request
- Use the MTP /
nextn heads in the MLX engine when a checkpoint provides them, with a configurable draft depth. Depth 2 was optimal on this machine; 1-4 covers the useful range.
- Independently of (1), and much cheaper: surface a load-time notice when a checkpoint contains MTP heads the engine cannot use. Today this is entirely silent, so there is no way for a user to tell that the MLX path is leaving speculation unused.
Related
#205 (MTP for GLM) and #323 (MTP model types fail to load) are both about architectures that don't load at all. This one is different: the model loads correctly, and the MTP heads inside it are ignored.
cc @neilmehta24 (assignee on #323)
Summary
LM Studio 0.4.14 added MTP speculative decoding for the GGUF/llama.cpp path. The MLX engine has no equivalent: when a checkpoint ships built-in multi-token-prediction (
nextn) heads, the MLX engine loads the model fine but does not use those heads, and gives no indication that it is skipping them.On Apple Silicon this inverts the expected ranking. MLX has the better base decode, but ends up the slower option overall, purely because MTP exists only on the llama.cpp side.
Environment
Youssofal/Qwen3.8-27B-MTPLX-Optimized-Speed(MLX, 19.85 GB, ships MTP heads)Observed
These are from normal chat use, not a controlled harness. I'm reporting them as a signal about the gap, not as a benchmark:
--spec-type draft-mtp --spec-draft-n-max 2 --parallel 1Happy to re-measure any row under a configuration you specify, on a fixed prompt set — just say what you want measured and I'll run it.
Request
nextnheads in the MLX engine when a checkpoint provides them, with a configurable draft depth. Depth 2 was optimal on this machine; 1-4 covers the useful range.Related
#205 (MTP for GLM) and #323 (MTP model types fail to load) are both about architectures that don't load at all. This one is different: the model loads correctly, and the MTP heads inside it are ignored.
cc @neilmehta24 (assignee on #323)