Skip to content

Extend CK fmha_batch_prefill kernel coverage to head_dim=256 #3328

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: develop
Choose a base branch
from
Open

Extend CK fmha_batch_prefill kernel coverage to head_dim=256 #3328

wants to merge 1 commit into from
+1 −0

Conversation

@vivienfanghuagood
Copy link

@vivienfanghuagood vivienfanghuagood commented Nov 28, 2025

Proposed changes

I add CK to generate fmha_batch_prefill kernels for hdim_q=hdim_v=256 in group mode (paged KV). It's required because in Qwen3-Next, the head_dim is 256 which is not supported in AITER Attention Backend(the default option in AMD GPUs)

Checklist

Please put an x into the boxes that apply. You can also fill these out after creating the PR. If you're not sure, please don't hesitate to ask.

  • I have added tests relevant to the introduced functionality, and the unit tests are passing locally
  • I have added the test to REGRESSION_TESTS list defined at the top of CMakeLists.txt in tests/CMakeLists.txt, IF the test takes more than 30 seconds to run.
  • I have added inline documentation which enables the maintainers with understanding the motivation
  • I have removed the stale documentation which is no longer relevant after this pull request
  • (If this change is user-facing) I have added release notes which provide the end users with a brief summary of the improvement from this pull request
  • I have run clang-format on all changed files
  • Any dependent changes have been merged

Discussion

If this is a relatively large or complex change, feel free to start a discussion by explaining why you chose the solution you did and what alternatives you considered

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@illsilin illsilin Awaiting requested review from illsilin illsilin is a code owner

@carlushuang carlushuang Awaiting requested review from carlushuang carlushuang is a code owner

@qianfengz qianfengz Awaiting requested review from qianfengz qianfengz is a code owner

@aosewski aosewski Awaiting requested review from aosewski aosewski is a code owner

@poyenc poyenc Awaiting requested review from poyenc poyenc is a code owner

@geyyer geyyer Awaiting requested review from geyyer geyyer is a code owner

@bartekxk bartekxk Awaiting requested review from bartekxk bartekxk is a code owner

@andriy-ca andriy-ca Awaiting requested review from andriy-ca andriy-ca is a code owner

@afagaj afagaj Awaiting requested review from afagaj afagaj is a code owner

@asleepzzz asleepzzz Awaiting requested review from asleepzzz asleepzzz is a code owner

@tenpercent tenpercent Awaiting requested review from tenpercent tenpercent is a code owner

@ThomasNing ThomasNing Awaiting requested review from ThomasNing ThomasNing is a code owner

@coderfeli coderfeli Awaiting requested review from coderfeli coderfeli is a code owner

@cgmillette cgmillette Awaiting requested review from cgmillette cgmillette is a code owner

@shumway shumway Awaiting requested review from shumway shumway is a code owner

@vidyasagar-amd vidyasagar-amd Awaiting requested review from vidyasagar-amd vidyasagar-amd is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@vivienfanghuagood