Skip to content

cannot find gemma3.attention.head_count in metadata #3215

@daviscai

Description

@daviscai

version: 0.9.2-alpha.1
model: https://huggingface.co/unsloth/embeddinggemma-300m-GGUF
model file: embeddinggemma-300M-Q8_0.gguf

my code:
`
use candle_core::{IndexOp, Tensor, quantized::gguf_file};
use candle_transformers::models::quantized_gemma3::ModelWeights;

let mut file = std::fs::File::open(&model_file)?;
let model_content = gguf_file::Content::read(&mut file).map_err(|e| e.with_path(model_path))?;
let model = ModelWeights::from_gguf(model_content, &mut file, &device)?;
`
error:
cannot find gemma3.attention.head_count in metadata

check ModelWeights::from_gguf source code:

impl ModelWeights {
    pub fn from_gguf<R: std::io::Seek + std::io::Read>(
        ct: gguf_file::Content,
        reader: &mut R,
        device: &Device,
    ) -> Result<Self> {
        let md_get = |s: &str| match ct.metadata.get(s) {
            None => candle::bail!("**cannot find {s} in metadata**"),
            Some(v) => Ok(v),
        };

but, I use candle_transformers::models::quantized_qwen3 is fine,

quantized_qwen3::ModelWeights::from_gguf source code:

impl ModelWeights {
    pub fn from_gguf<R: Read + Seek>(
        ct: gguf_file::Content,
        reader: &mut R,
        device: &Device,
    ) -> Result<Self> {
        let mut gg = Gguf::new(ct, reader, device.clone());
        let md_get = |s: &str| match gg.metadata().get(s) {
            None => candle::bail!("cannot find {s} in metadata"),
            Some(v) => Ok(v),
        };

Should quantized_gemma3::ModelWeights::from_gguf be changed to add this line let mut gg = Gguf::new(ct, reader, device.clone()); ?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions