Misc. bug: convert_hf_to_gguf failed

[JSXGQ]

Name and Version

convert_hf_to_gguf failed

Operating systems

No response

Which llama.cpp modules do you know to be affected?

No response

Command line

Problem description & steps to reproduce

model source: https://huggingface.co/deepseek-ai/DeepSeek-R1/tree/main
execute: python3 convert_hf_to_gguf.py /data/DeepSeek-R1 --outtype f16 --verbose --outfile deepseek_r1_f16.gguf
alert: ValueError: Can not map tensor 'model.layers.0.mlp.down_proj.weight_scale_inv'
detail: The details have been uploaded as pictures

First Bad Commit

No response

Relevant log output

[w77ww]

Same problem, Fedora 40.
https://huggingface.co/deepseek-ai/DeepSeek-V3/blob/main/README_WEIGHTS.md#dequantization-method describes the purpose of weight_scale_inv tensors to rescale blocks of 8b weights.

Unsloth has GGUFs for R1, but I don't know if they approach quantization the same way. I'll unpack theirs and see if there's any correspondence but my internet is pretty slow.

I don't know the codebase yet, but three ideas present themselves.

1. If GGUF already understands block rescaling, add alias "model.layers.{div}.mlp.down_proj.weight_scale_inv" in the correct place in gguf-py/gguf/tensor_mapping.py.
2. (Hard), Dig into GGUF and add functionality.
3. (Bad), Rescale to BF16 during conversion, doubling the size of the model.

[JSXGQ]

Same problem, Fedora 40. https://huggingface.co/deepseek-ai/DeepSeek-V3/blob/main/README_WEIGHTS.md#dequantization-method describes the purpose of weight_scale_inv tensors to rescale blocks of 8b weights.

Unsloth has GGUFs for R1, but I don't know if they approach quantization the same way. I'll unpack theirs and see if there's any correspondence but my internet is pretty slow.

I don't know the codebase yet, but three ideas present themselves.

1. If GGUF already understands block rescaling, add alias "model.layers.{div}.mlp.down_proj.weight_scale_inv" in the correct place in gguf-py/gguf/tensor_mapping.py.
2. (Hard), Dig into GGUF and add functionality.
3. (Bad), Rescale to BF16 during conversion, doubling the size of the model.

If it is the first or second way, then it is a bug or feature that needs to be verified and fixed by llama officials.
If it is the third kind of problem, I don't know how to solve it, haha
Thanks for your answer