Experimental vibecoded Gguf map tool - #118
Draft
amd-satre wants to merge 3 commits into
Draft
Conversation
Adding an architecture starts by diffing two lists of ~1000 tensor names to find which map to which, what is left over on each side, and where shapes disagree. This does that mechanically and prints a WeightsMapper skeleton. Matching is a global assignment, not per-tensor greedy: locally, Qwen3.5's attn_output and ssm_out are both plausible matches for linear_attn.out_proj and have identical shapes, so whichever is considered first wins and the other is left wrong. Resolving highest-confidence pairs first leaves the loser its uncontested second choice. Layer occupancy is the signal that separates them -- the two occupy disjoint layer index sets. Shape incompatibility disqualifies a pairing outright rather than merely penalising it, so an unmappable tensor is reported as unmatched instead of being carried by name similarity into a bogus "transform needed". --target vllm reconstructs the names load_weights accepts from the module tree, since vLLM fuses linears and stacks experts but its loader still consumes the unfused per-expert names. Reads GGUF headers over an HTTP range request and builds the target on meta tensors: no GPU, no weight download. Verified against the OLMoE adapter already in-tree -- the derived rules equal build_olmoe_mapper() exactly, which the tests pin. Also 100% coverage with no unmatched tensors on Qwen3.6-27B, Qwen3-0.6B, DeepSeek-V2-Lite and DeepSeek-V3. Signed-off-by: Shreyas Atre <satre@amd.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A QKVParallelLinear or MergedColumnParallelLinear concatenates its bias the same way it concatenates its weight, but only the weight was being expanded back into per-part names. Any architecture with attention_bias set therefore had attn_q/k/v.bias reported unmatched with no target to map to. Found by running the tool on GLM-4.5-Air, which goes from 20/29 to 23/29 templates mapped. The remaining 6 are its nextn (MTP) block, correctly unmatched: vLLM does not instantiate that module without a speculative config, so those targets genuinely do not exist. Signed-off-by: Shreyas Atre <satre@amd.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Tested against vllm-gguf-plugin#113 (Muse Glimmer), which exercises three things the tool could not do. A vision-language model ships its projector as a separate mmproj file, so --gguf now takes more than one. Neither transformers 5.14 nor vLLM knows muse_glimmer, so neither existing target could be built at all. --target safetensors reads the checkpoint's own headers over an HTTP range request instead, which needs no support for the architecture in either library -- the usual situation when GGUF support lands first. Not every integer in a name is a layer index. A projector numbers its stages mm.0/mm.1/mm.2, and collapsing those into one template hid two of the three behind a single shape and left their targets uncovered. Members that no two of which share a shape are an enumeration, not a repetition. The template shape is now the most common among members rather than the first seen, so one odd layer cannot set it. The emitted skeleton was also silently wrong for any model with two towers. ffn_up is mlp.up_proj under blk and mlp.fc1 under v.blk; both were written to the same orig_to_new_substr key and the second overwrote the first, dropping five text-stack rules with no indication. A leaf whose target depends on its prefix is now emitted as an orig_to_new_regex rule scoped to that prefix, which is what WeightsMapper's regex-then-substr-then-prefix order needs. Verified by applying the derived mapper and vllm-project#113's hand-written one to all 1540 real GGUF tensors: 1539 rename identically. The one difference is v.patch_embd.weight, which the tool leaves unmatched because its shape does not match the target -- it needs a split, not a rename, which is what vllm-project#113 implements. The 104 attn_q/k_norm tensors vllm-project#113 drops are likewise reported unmatched. No regression on Qwen3.6-27B, Qwen3-0.6B, DeepSeek-V2-Lite, DeepSeek-V3, OLMoE or GLM-4.5-Air. Signed-off-by: Shreyas Atre <satre@amd.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
amd-satre
force-pushed
the
gguf-map-tool
branch
from
August 21, 2026 22:03
df45f67 to
8968f6c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Edit 08/21/2026: \_" " _/ haven't verified with my eyes yet, save yourselves the trouble to review.