Skip to content

Add MMQ v2 and generic model adapters #21

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
lesj0610 wants to merge 28 commits into
base: main
Choose a base branch
from
Open

Add MMQ v2 and generic model adapters #21

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
964c1fa
Add GGUF MMQ v2 kernels
lesj0610 Jun 10, 2026
5d36fca
Resolve GGUF config sources in plugin
lesj0610 Jun 10, 2026
6b2eb3c
Add GGUF model adapters and tuple shard loading
lesj0610 Jun 10, 2026
76ebfe8
Support GGUF MTP config and weight loading
lesj0610 Jun 10, 2026
7725a04
Address GGUF integration review findings
lesj0610 Jun 10, 2026
cb425d3
Fix GGUF speculator probe API contract
lesj0610 Jun 10, 2026
94aa0ca
Normalize Qwen3.5 GGUF config metadata
lesj0610 Jun 10, 2026
3afd5f5
Fix GGUF weight loader v2 shardless calls
lesj0610 Jun 10, 2026
2ec35c0
Load GGUF multimodal sidecar weights
lesj0610 Jun 10, 2026
77befb5
Combine split Qwen GGUF patch embeddings
lesj0610 Jun 10, 2026
92f5274
Fix Gemma4 GGUF adapter mappings
lesj0610 Jun 10, 2026
f1b64c5
Build GGUF configs from metadata
lesj0610 Jun 10, 2026
f4cc33f
Build tokenizers from GGUF metadata
lesj0610 Jun 10, 2026
b1471e3
Build GGUF MM processor metadata locally
lesj0610 Jun 11, 2026
db3bb31
Address GGUF metadata and MoE review feedback
lesj0610 Jun 11, 2026
06f48ff
Materialize Gemma4 GGUF tokenizer special tokens
lesj0610 Jun 11, 2026
7893372
Use plugin namespace for GGUF custom ops
lesj0610 Jun 11, 2026
d72a801
Fix dense Qwen GGUF loading and shard memory
lesj0610 Jun 11, 2026
93cfb4f
Fix Qwen dense GGUF text-only weight layout
lesj0610 Jun 11, 2026
311584b
Fix GGUF Gemma4 text-only mappings
lesj0610 Jun 12, 2026
05db965
Fix dense Qwen GGUF MTP layer mapping
lesj0610 Jun 12, 2026
2b5a13c
Fix Qwen multimodal GGUF embedding mapping
lesj0610 Jun 12, 2026
17fa3f8
Restore GGUF tokenizer token types
lesj0610 Jun 12, 2026
91e43bf
Match GGUF quant override signature
lesj0610 Jun 12, 2026
9585809
Prioritize sidecar GGUF config and complete Qwen metadata
lesj0610 Jun 12, 2026
f032a63
Remove native GGUF config builder fallback
lesj0610 Jun 12, 2026
4ec509b
Fix Qwen3.5 GGUF adapter weight transforms
lesj0610 Jun 13, 2026
4611077
Respect local config special tokens for GGUF tokenizers
lesj0610 Jun 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,8 @@ extend-exclude = [
"vllm_gguf_plugin/csrc/gguf/dequantize.cuh",
"vllm_gguf_plugin/csrc/gguf/vecdotq.cuh",
"vllm_gguf_plugin/csrc/gguf/mmq.cuh",
"vllm_gguf_plugin/csrc/gguf/mma_v2.cuh",
"vllm_gguf_plugin/csrc/gguf/mmq_v2.cuh",
"vllm_gguf_plugin/csrc/gguf/mmvq.cuh",
]
ignore-hidden = false
Expand Down
Loading