Contributing guidelines and PR review process

[Thump604]

Hi @waybarrios — I've been contributing to vllm-mlx over the past few weeks and have 8 open PRs (#165, #169, #171, #173, #174, #175, #177, #180), all with green CI. I noticed other contributors also have unreviewed PRs (#183, #185, #160).

A few questions:

1. Review cadence — Is there a regular review schedule, or is it best to ping on individual PRs?
2. PR size preference — Would you prefer smaller, more focused PRs or is the current granularity fine?
3. Contributing guidelines — Any preferences on code style, testing, or PR format that would help get things merged faster?
4. Prioritization — Are there specific areas where contributions are most welcome?

Happy to adjust my approach based on your workflow. Thanks for maintaining vllm-mlx!

[Thump604]

Open PRs — review order and test evidence

Hi @waybarrios — all PR descriptions have been updated to be short and scannable. Here's the full picture.

Review order

Standalone fixes (no dependencies, merge in any order):

• fix: normalize messages in BatchedEngine before chat template #190 — Normalize messages in all engine paths (3 files, CI green)
• fix: honor tool_choice=none by stripping tools and suppressing parsing #173 — Honor tool_choice="none" — strip tools from template, skip parsing
• fix: parse tool calls in streaming reasoning branch #177 — Fix streaming when reasoning tags and tool calls coexist
• fix: chunked prefill compat with mlx-lm >= 0.31.0 #169 — Chunked prefill compatibility with mlx-lm >= 0.31.0

Core features (review in order):

1. fix: MLLM continuous batching for hybrid models #165 — MLLM continuous batching for hybrid VLM+MTP models
2. feat: MLLM+MTP per-request routing for text and vision #171 — MTP per-request routing: text→TextModel with MTP, media→MLLM (depends on fix: MLLM continuous batching for hybrid models #165)
3. feat: configurable prefill step size for SimpleEngine #174 — Configurable --prefill-step-size

SpecPrefill:
4. #180 — SpecPrefill in SimpleEngine (depends on #171)
5. #192 — BatchedEngine parity: MTP routing + SpecPrefill + prefix caching (depends on #171, #180)

Additional features:

• True token-by-token streaming for MLLM path #187 — True MLLM streaming (queue-based)
• Add repetition detector for degenerate generation loops #188 — Repetition detector
• Token-level JSON schema constrained decoding #189 — JSON schema constrained decoding (outlines-core)

Closed: #175 (system KV cache — superseded by native prefix caching in #192)

Test results

Tested on Mac Studio M2 Ultra 128GB, macOS 26.3.1, Qwen3.5-122B-A10B-VLM-MTP (5-bit, 79GB).

Stability — 30+ hours of inference, zero crashes:

• 7h RULER benchmark (250 requests at 16K–32K context)
• 8h+ LongBench evaluation (440 samples across baseline and SpecPrefill)
• Multiple SpecPrefill benchmark runs: ablation across 8K–128K (5 trials each), 8 adversarial test types, perplexity analysis, cross-architecture validation (Nemotron-H 120B, GPT-OSS 120B)
• 5 concurrent requests x 1024 tokens: all complete in 8s
• 60 requests across 3 concurrent sessions: all complete
• Daily production serving since March 12 (coding agents via Claude Code / OpenCode)
• Comparison: 41 SIGSEGV crashes in 9 days on MLX 0.31.1 → zero crashes since MLX thread-local-streams (PR Make each thread have its own default stream ml-explore/mlx#3281)

Feature verification:

Test	Result
MTP routing (text-only)	Pass — server log confirms TextModel path, tok/s matches
MTP routing (media)	Pass — routes to MLLM
Message normalization (developer role)	Pass — mapped to system
Consecutive same-role merge	Pass — merged with \n\n separator
tool_choice="none"	Pass — tools stripped, tool_calls: null
response_format: json_schema	Pass — JSON as content, not parsed as tool call
Concurrent requests (3–5)	Pass — no corruption
Prefix cache (repeated system prompt)	Pass — 5.9s cold → 0.8s cached
Rollback (continuous_batching=false)	Pass — reverts to SimpleEngine cleanly

SpecPrefill TTFT (Qwen3.5-122B, 2B draft, 20% keep, 5 trials):

Context	Baseline	SpecPrefill	Speedup
8K	45.0s	12.1s	3.71x
16K	92.0s	22.4s	4.11x
32K	190.4s	45.0s	4.23x
64K	417.6s	92.8s	4.50x
128K	1155.8s	212.3s	5.45x

Quality (LongBench, 11 tasks, N=20, greedy): 9/11 tasks within baseline noise. Code completion (RepoBench-P) is the known risk at 20% keep — documented in the paper.

All PRs have green CI. Happy to rebase, squash, or restructure any of these.