Batched vision models cannot use a draft model: is_draft_model_compatible() returns False unconditionally

[cristyanul]

Summary

BatchedVisionModelKit.is_draft_model_compatible() returns False unconditionally, so no draft model can be selected for any multimodal MLX model. The Draft Model dropdown is permanently empty for VLMs.

https://github.com/lmstudio-ai/mlx-engine/blob/main/mlx_engine/model_kit/batched_vision/model_kit.py#L373-L384

def is_draft_model_compatible(self, path: str | Path) -> bool:
    return False

def load_draft_model(self, path: str | Path) -> None:
    raise ValueError(
        "Speculative decoding is not currently supported for batched vision models"
    )

Model-kit dispatch selects this kit whenever config.json contains a vision_config, so it applies to every VLM, including text-only requests.

is_speculative_decoding_supported() rejects it a second time:

def is_speculative_decoding_supported(model_kit: object) -> bool:
    return type(model_kit) is ModelKit

https://github.com/lmstudio-ai/mlx-engine/blob/main/mlx_engine/utils/speculative_decoding.py#L15

Related issues:

• SpeculativeDecodingNotSupportedError: Speculative decoding is not supported for batched MLX models. #269 / Use ModelKit if max_seq_nums is explicitly set to 1 #272 — same gate for BatchedModelKit (text), fixed via the max_seq_nums == 1 escape hatch. Vision kits have no equivalent.
• Bug: Batched VLM loader treats mtp.safetensors as base weights for Qwen3.5-4B-OptiQ-4bit #343 — mtp.* sidecar strict-load failure. A loading bug, explicitly not acceleration.
• New MTP models not supported #323 — general MTP support.

Environment

• LM Studio: 0.4.20+1
• Selected MLX runtime: mlx-llm-mac-arm64-apple-metal-advsimd@1.11.0 (also checked 1.10.1; identical)
• Vendored mlx_vlm: 0.6.3
• macOS: 26.5, arm64

Reproduction

1. Load any MLX model whose config.json contains a vision_config.
2. Open Speculative Decoding → Draft Model.

Actual Behavior

The dropdown shows "No compatible draft models found for your current model selection" for every candidate, including a native MTP drafter split from the same base.

is_draft_model_compatible() returns before any comparison is performed, so drafter suitability is never evaluated.

Setting the drafter directly via llm.prediction.speculativeDecoding.draftModel reaches load_draft_model(), which raises:

ValueError: Speculative decoding is not currently supported for batched vision models

Expected Behavior

A matching drafter should be selectable and used, as it is on the text path.

The same base/drafter pair works through the vendored mlx_vlm directly:

generate_step(input_ids, model, pixel_values=None, mask=None,
              draft_model=drafter, draft_kind="mtp")

so this is a dispatch limitation rather than a model or library one.

Notes

Enabling the flag alone is not sufficient. MTP verification needs hidden_states[-1] and shared_kv_states from the target model (mlx_vlm/speculative/utils.py::run_speculative_rounds), while GenerationBatch._step() consumes .logits only — the drafter would load and go unused.

I have a local patch that routes single text-only requests through mlx_vlm's generate_step when a drafter is loaded, leaving image and concurrent requests on the stock batched path. Measured 1.77x (18.9 vs 10.7 tok/s) with byte-identical output on a 27B 6-bit VLM. It forgoes prompt-cache reuse while drafting, which is why I'm filing this rather than a PR — happy to share it if useful.