[Roadmap] [25H2] Custom Kernel Library for SGLang on Ascend NPU

[iforgetmyname]

Planning of custom kernels on 2025 H2

DeepEP-compatible EP Library

Our goal of this DeepEP-compatible library is to provide a set of consistent APIs with DeepSeek's DeepEP, relies upon our custom kernels.

Low-latency Kernels

Please be noticed that we have a restriction of BS <= 512 for low-latency kernels. Which means these kernels only can be used in decode instances, not prefill ones.

• [Aug 15th.] Remove the restriction of HiddenSize = 7168, thus supporting more MoE models.

Normal Kernels

• [Aug 30th.] Support prefill with SeqLen <= 2048

Quantization Kernels

• [TBD] Simple AWQ dequant kernels
• [TBD] Simple GPTQ dequant kernels

[huangmengwei-cmss]

大佬好，有针对该项目的微信交流群不。。。

[ping1jing2]

大佬好，有针对该项目的微信交流群不。。。

目前没有，如果发现了bug建议提issue并且assign给我，如果有其他问题不好描述可以给我发邮件联系呢zl19940307@163.com

[SoonyangZhang]

建议丰富下文档，这样公司内外，能够形成合力，在算子适配上，可以缩短时间间隔。关于华为昇腾的算子开发文档，要么找不到，要么讲的不清楚。这样，对新手很不友好。开源了，就是很好的学习素材。

[ping1jing2]

建议丰富下文档，这样公司内外，能够形成合力，在算子适配上，可以缩短时间间隔。关于华为昇腾的算子开发文档，要么找不到，要么讲的不清楚。这样，对新手很不友好。开源了，就是很好的学习素材。

好的，感谢建议

[sleepy-dev-bin]

Is there any plan to support block-sparse attention mechanisms? With this operator, we can support speculative reasoning features such as Eagle. Currently, there is an implementation example of a CUDA operator: https://github.com/mit-han-lab/Block-Sparse-Attention

[RuixuanZhang06]

Maybe we can support block-sparse attention later using Triton—once Triton on Ascend is mature, probably around H1 2026.