Commit 9d6826a
committed
[Bugfix] Fix HYV3 shared_mlp prefix for compressed-tensors ignore matching
HYV3MoEFused built its shared expert (shared_mlp) as an HYV3FeedForward
with `prefix=f"{prefix}"`, omitting the `.shared_mlp` segment. The Linear
layers inside it (gate_up_proj, down_proj) register their parameters under
`...mlp.shared_mlp.<proj>.*` (the attribute name is appended by nn.Module
regardless of the prefix argument), but CompressedTensorsConfig ignores
Linears by matching the *prefix* against its `ignore` list. With the
missing segment the ignore check saw `...mlp.down_proj` instead of
`...mlp.shared_mlp.down_proj`, so a `re:.*shared_mlp.*` ignore entry
failed to match.
Consequence: the shared expert was built as a quantized Linear expecting
weight_packed/scale/shape, while real W4A16 hy_v3 checkpoints (produced
by llm-compressor) correctly keep shared_mlp in BF16 as `.weight`. This
raised `KeyError: '...mlp.shared_mlp.down_proj.weight'` at load time.
Fix: pass `prefix=f"{prefix}.shared_mlp"` so the ignore check matches the
real parameter registration path and the shared expert is built
unquantized (UnquantizedLinearMethod), aligning with the BF16 checkpoint.
Parameter registration names are unchanged.
Co-authored-by: xu_xiaoyi@hotmail.com
Signed-off-by: Xiaoyi Xu <xu_xiaoyi@hotmail.com>1 parent 31be872 commit 9d6826a
1 file changed
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
171 | 171 | | |
172 | 172 | | |
173 | 173 | | |
174 | | - | |
| 174 | + | |
175 | 175 | | |
176 | 176 | | |
177 | 177 | | |
| |||
0 commit comments