Use FI tinygemm for faster BF16 GEMMs#39921

Open

askliar wants to merge 7 commits intovllm-project:mainfrom

askliar:askliar/flashinfer-tinygemm-bf16

Commits on Apr 15, 2026

Use FlashInfer tinygemm_bf16 for small-M BF16 GEMVs (decode optimization)

Andrii Skliar
and
claude
committed
Fix tinygemm dispatch for multi-dimensional input tensors

Andrii Skliar
and
claude
committed
Fix torch.compile compatibility for tinygemm dispatch

Andrii Skliar
and
claude
committed
Fix torch.compile (AOT fullgraph) compatibility for tinygemm

Andrii Skliar
and
claude
committed
Refactor tinygemm to use vLLM custom op pattern for torch.compile

Andrii Skliar
and
claude
committed

Commits on Apr 16, 2026

Merge branch 'main' into askliar/flashinfer-tinygemm-bf16
askliar
authored

Commits on Apr 21, 2026

Merge branch 'main' into askliar/flashinfer-tinygemm-bf16
askliar
authored