Skip to content

[v1] refactor NPU kernel matching by model type#10643

Open
xvxuopop wants to merge 1 commit into
hiyouga:mainfrom
xvxuopop:update-npu-kernels
Open

[v1] refactor NPU kernel matching by model type#10643
xvxuopop wants to merge 1 commit into
hiyouga:mainfrom
xvxuopop:update-npu-kernels

Conversation

@xvxuopop

@xvxuopop xvxuopop commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Refactors v1 NPU kernels to use explicit model_type-based matching for RoPE, RMSNorm, SwiGLU, and Fused MoE. It also separates Transformers v4/v5 MoE mappings.

Before submitting

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the NPU kernel implementations (including MoE, SwiGLU, RMSNorm, and RoPE) to select and apply patches based on model types rather than class names or architectures, and introduces corresponding unit tests. Feedback on the changes highlights a critical issue in the SwiGLU kernel where MLP submodules typically lack a config attribute, causing the patch check to fail. It is recommended to pass the top-level model.config to _get_patch_forward and update the unit tests accordingly.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread tests_v1/plugins/model_plugins/test_npu_kernels.py Outdated
@xvxuopop xvxuopop force-pushed the update-npu-kernels branch from 3c13c20 to 8956429 Compare July 14, 2026 08:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant