Skip to content

feat(attn): add paged-decode tuple 16,128,64,false,true,false - #526

Merged
jikunshang merged 1 commit into
vllm-project:mainfrom
jikunshang:kunshang/cp-524
Aug 14, 2026
Merged

feat(attn): add paged-decode tuple 16,128,64,false,true,false#526
jikunshang merged 1 commit into
vllm-project:mainfrom
jikunshang:kunshang/cp-524

Conversation

@jikunshang

Copy link
Copy Markdown
Member

Cherry-pick from #524
Muse-Glimmer-30B uses local attention with qgroup=16, head=128, page=64. The causal=false/local=false/sink=false sibling is already compiled for GLM/Seed. Without the local=true specialization the first paged-decode warmup raises:

RuntimeError: Paged decode kernel tuple not compiled for this
configuration. 16,128,64,false,true,false

Format: qgroup,headsize,pagesize,causal,local,sink.

Essential Elements of an Effective PR Description Checklist

  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS ABOVE HAVE BEEN CONSIDERED.

Purpose

Test Plan

Test Result

(Optional) Documentation Update

BEFORE SUBMITTING, PLEASE READ https://docs.vllm.ai/en/latest/contributing (anything written below this line will be removed by GitHub Actions)

Muse-Glimmer-30B uses local attention with qgroup=16, head=128,
page=64. The causal=false/local=false/sink=false sibling is already
compiled for GLM/Seed. Without the local=true specialization the first
paged-decode warmup raises:

  RuntimeError: Paged decode kernel tuple not compiled for this
  configuration. 16,128,64,false,true,false

Format: qgroup,headsize,pagesize,causal,local,sink.
Signed-off-by: SergiioB <sergiobarrientose@outlook.com>
Copilot AI lite review requested due to automatic review settings August 14, 2026 02:39

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a missing paged-decode kernel tuple to the default Xe2 paged-decode configuration so models requiring qgroup=16, head_size=128, page_size=64, causal=false, local=true, sink=false (e.g., Muse-Glimmer-30B) don’t fail warmup with a “tuple not compiled” runtime error.

Changes:

  • Add the 16,128,64,false,true,false tuple to paged_decode_default.conf.
  • Document the tuple addition inline with a short model note.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@jikunshang
jikunshang enabled auto-merge (squash) August 14, 2026 02:53
@jikunshang
jikunshang merged commit 27214a3 into vllm-project:main Aug 14, 2026
10 checks passed
cursor Bot pushed a commit to krisclarkdev/vllm-xpu-kernels that referenced this pull request Aug 17, 2026
Resolve paged_decode_default.conf conflict by keeping both the
pagesize=128 GLM/Seed entry from this PR and the Muse-Glimmer
local-attn tuple from vllm-project#526.

Signed-off-by: Kristopher Clark <krisclarkdev@krisbox.org>
Co-authored-by: Cursor <cursoragent@cursor.com>
Signed-off-by: Kristopher Clark <krisclarkdev@krisbox.org>
jikunshang added a commit that referenced this pull request Sep 2, 2026
Muse-Glimmer-30B uses local attention with qgroup=16, head=128,
page=64. The causal=false/local=false/sink=false sibling is already
compiled for GLM/Seed. Without the local=true specialization the first
paged-decode warmup raises:

  RuntimeError: Paged decode kernel tuple not compiled for this
  configuration. 16,128,64,false,true,false

Format: qgroup,headsize,pagesize,causal,local,sink.

Signed-off-by: SergiioB <sergiobarrientose@outlook.com>
Co-authored-by: SergiioB <sergiobarrientose@outlook.com>
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants