[Roadmap] Diffusion LLMs (2026 S1)

[ClawSeven]

Checklist

• If this is not a feature request but a general question, please start a discussion at https://github.com/sgl-project/sglang/discussions. Otherwise, it will be closed.
• Please use English. Otherwise, it will be closed.

Motivation

Earlier this year, LLaDA released the first diffusion LLM (dLLM), immediately capturing significant attention from both the academic and industrial communities. But there were no production-ready dLLM serving engine.

We plan to implement the most performant, production-ready dLLM framework in SGLang, make dLLM robust !

Features

• Initial diffusion LLM framework @ClawSeven @btw616 [Feature] Initial block diffusion language model support #12588
  • Support LLaDA2.0-flash / LLaDA2.0-mini
  • Support tensor parallel / expert parallel
  • Support block diffusion and kv cache
• Doc and CI @ClawSeven @Monstertail
  [DLLM] Add documentation for diffusion LLMs  #14358
  [DLLM] Add CI for diffusion LLMs #14723
• Support cuda graph @btw616 [DLLM] Add initial cuda graph support #14203
• Support self-defined attention mask
• Support parallel decoding @btw616 @Monstertail [DLLM] Add threshold based parallel decoding support #14412
• Support temperature, topp, topk
• dLLM code refactoring @ClawSeven
  • Support initial dynamic batching [DLLM] Implement initial dynamic batching for diffusion LLM #14883
  • Batching optimization & dLLM scheduling refactor [DLLM] Basic dLLM scheduling strategy and implementation #17484
  • Requests early exit (for decoding optimization)
• Support radix cache (for prefill optimization) @btw616 [DLLM] Add initial radix cache support #18724
• Support overlap scheduling
• Support dLLM editing @edwardzjl @btw616 [DLLM] Add JointThreshold algorithm for joint M2T and T2T decoding #18171
• Metrics for dLLM @zhanghaotong
• Support non-block diffusion LLMs
• Var-length Prefill optimization, rely on feat(dllm): add Block Extend Attention for Diffusion LLM flashinfer-ai/flashinfer#2722
• piecewise CUDA Graph (for prefill optimization)

For RL

• Support step maps @RuixiangMa @ClawSeven [DLLM] support step map #17297
• Support abort request

VL-dLLM

• Initial multi-modal LLM implementation @btw616

More supported models

• LLaDA2.0
• SDAR @chengshuang18 Add SDAR model support #18318
• Fast-dLLM v2 @Monstertail (WIP) [DLLM]Fast-dLLM-v2 support with HierarchyBlock algorithm for parallel decoding #17577

More Hardwares

• Nvidia
• AMD:
  • triton backend: [AMD] Add DLLM support for AMD GPUs with LLaDA2 testing #15560
  • aiter backend
• Ascend
  • ascend backend: [NPU] [DLLM]DLLM LLaDA2.x graph mode support with NPU speedup modifications #18485
  • triton backend
• Intel

More Parallelism

• Tensor parallelism
• Expert parallelism
• Data Parallelism (with DPA)
• Context Parallelism
• Pipeline parallelism

Kernel Optimization for dLLM

• Moe
  The dLLM makes the fused‑MoE kernel a bottleneck, so we need to optimize the fused implementation for dLLM scenarios.
  • FP8 optimization for small batch sizes: Add SwapAB Optimization for triton fused_moe_kernel on SM90. #15712
• Attention
  Optimize block‑wise causal attention for dLLM prefill.
• Decoding Algorithm
  • Low-confidence @YJMSTR [DLLM] Add fused Triton post-process kernel with on-device fallback #18810

More disaggregation

PD is not suitable for dLLM, but AFD might be a viable option.

More Tests

• Small unit tests for specific functions
• Nightly unit tests for E2E accuracy and throughput testing

Better streaming output

• Support diffusion-style streaming output (like Mercury)

RFC

#12766

Related resources

• [Feature] Add nightly test for diffusion LLMs(like LLaDA-2-flash 100B) #15256
• [Feature] support Support Nvidia Fast-dLLM v2 model #15011

[Monstertail]

Checklist

• If this is not a feature request but a general question, please start a discussion at https://github.com/sgl-project/sglang/discussions. Otherwise, it will be closed.[ ] Please use English. Otherwise, it will be closed.

Motivation

Earlier this year, LLaDA released the first Diffusion Language Model (dLLM), immediately capturing significant attention from both the academic and industrial communities. But there were no production-ready diffusion LLM serving engine.

We plan to implement the most performant, production-ready DLM framework in SGLang, make dLLMs robust !

Features

• • Initial diffusion LLM framework [Feature] Initial block diffusion language model support #12588

    • Support LLaDA2.0-flash / LLaDA2.0-mini* [x] Support tensor parallel / expert parallel* [x] Support block diffusion and kv cache[ ] Doc and CI[ ] Support cuda graph[ ] Support Nvidia Fast-dLLM v2 model[ ] Support self-defined attention mask[ ] Support parallel decoding[ ] Support step maps[ ] Support temperature, topp, topk[ ] Support dynamic batching[ ] Support radix cache[ ] Distinguish between prefill and decode patterns[ ] Support overlap scheduling[ ] Support non-block diffusion language model

RFC

#12766

Related resources

No response

Great to have this.
BTW, on algorithm side, the current plan is to support the models of fast-dllm v2, then support custom_mask and parallel_decoding.

[Monstertail]

[Algorithm side features]
Currently, we refactor the algorithm interface with a config file as an advanced setting for the algorithms.

(keep updating) We will provide some classic block diffusion algorithms.

• Fast-dllm v2[ongoing]
• BD3-LM: https://m-arriola.com/bd3lms/ (to decide)
• Uniform State discrete Diffusion Models(USDM) (to decide)

[zhaochenyang20]

🐂🐂🐂🐂🐂🐂

[cklxx]

I’m new to the project and very interested in sglang’s diffusion LLM support. I would love to contribute—are there any features or tasks where my help would be useful?

[asirgogogo]

Would you consider supporting the Ilada 1.0（8B instruct..） or Dream?

[tom-jerr]

I’m very interested in sglang’s diffusion LLM support. I would love to contribute—are there any features or tasks where my help would be useful?

[Monstertail]

Would you consider supporting the Ilada 1.0（8B instruct..） or Dream?

We will support the block diffusion first. As for non-block ones, we will consider them in the future.

[asirgogogo]

Would you consider supporting the Ilada 1.0（8B instruct..） or Dream?

We will support the block diffusion first. As for non-block ones, we will consider them in the future.

thanks

[Monstertail]

I’m very interested in sglang’s diffusion LLM support. I would love to contribute—are there any features or tasks where my help would be useful?

Thanks for your interest. Currently, we are trying to support some basic yet important features in a small team for efficient communication. When we lay a solid foundation, we will be very happy to include more contributors to join:)

[baonudesifeizhai]

just support fast dllm v2 for initial
#15013