Weekly Meeting 2026‐02‐03

Issues

To be discussed:

Assigned

To be solved:

MHA @YuchongLi

• [Feature Request] Multi-Head-Attention - Benchmark

CI @YuchongLi

• [Enhancement] Add Docker Images in CI
• [CI][Enhancement] Miss the tileOps install and build process
• [CI][Enhancement] Refactor CI functionality to enhance CI stability and expand test coverage

NSA @JienengYu

• [Feature][NSA] Implement a NSA compression forward kernel.

Hadamard Transform @Renkeyiiiiii

• [New Op] Implement Hadamard transform

MLA decode with kvcache @ZhejinXu

• [Feature Request] MultiHeadLatentAttentionDecodeWithKVCache Kernels
• [Feature Request] MultiHeadLatentAttentionDecodeWithKVCache Function
• [Feature Request] MultiHeadLatentAttentionDecodeWithKVCache Function
• [Feature Request] MultiHeadLatentAttentionDecodeWithKVCache Layer

Benchmark @Zheng QiHang

• [Potential] The benchmark check's default constraint is too relaxed.

MultiHeadAttention Decode @AngGao

• [Feature][Enhancement] Add paged attention for MHA_decode and GQA_decode

Solved this week:

Gemm benchmark @YuchongLi

• [Feature] [GEMM] The GEMM performance test has not been implemented in the benchmark.

DSA @JienengYu

• [Feature][NSA] Implement the nsa_kernel_topk.

DeepSeek Sparse Attention Decode @YuxianDu

• [Feature] [New Op] implement topk op for DSA
• [feature Request] Implement Indexer Kernel for DSA Attention in TileOps
• [Feature] [New Op] Implement TileLang Quant Kernel for Indexer Op using Torch Method

manifold-constrained hyper-connection @AngGao

• [Feature][New Op] manifold Hyper Connection (mHC)

PRs:

Merged:

• [Feat][mHC] Implement mHC kernels @AngGao
• [Feat][NSA] Implement the NSA topk kernel. @JienengYu
• Add topk op for DSA @YuxianDu
• [Feature] Implement tilelang quant kernel for dsa indexer @YuxianDu
• [Feat] Add Gemm ops benchmark baseline profile @YuchongLi

Actions Items

to-do

1. Create documentation for user-facing dependency installation
2. Convert Python tests to Pytest.
3. Distribute Docker images (one for internal development, one for external reproduction).