gemma-3-270m

[vishpat]

Has anyone gotten the google/gemma-3-270m model working with the candle transformers? I tried implementing the model https://github.com/vishpat/gemma-slurm/tree/c33fb9034f284558f222dc8df31e65a1067f463f, but I get a repeating value for the given test prompt.

[vishpat]

Got it working https://github.com/vishpat/gemma-slurm/tree/0.1

[vishpat]

Had to add the following to the model config.json

"sliding_window_pattern": 6,

[baijal1982]

I am still getting below error . I am exactly running your code
🚀 Initializing Language Model Question-Answer System
Loading model (this may take a moment on first run)...
Loading language model...
Loading config from: huggingface/models--google--gemma-3-270m/snapshots/9b0cfec892e2bc2afd938c98eabe4e4a7b1e0ca1/config.json
❌ Failed to initialize system: missing field sliding_window_pattern at line 54 column 1
💡 This might be due to network issues or model availability

[baijal1982]

thanks @vishpat . this is working now . The main fix here was you need to manually add sliding_window_pattern properly in config.json file

[vishpat]

The sliding_window_pattern parameter value needs to be 6 and not 128.

Even then I don't get sane answers from the model most of the times, even with temperature value of 0

[alvarobartt]

Hey @vishpat not sure if useful, but we implemented Gemma3 in candle on https://github.com/huggingface/text-embeddings-inference for EmbeddingGemma, but the architecture is the same, at https://github.com/huggingface/text-embeddings-inference/blob/main/backends/candle/src/models/gemma3.rs in case you feel like using as a reference implementation for handling the sliding + full attention pattern, handled in lines https://github.com/huggingface/text-embeddings-inference/blob/ff3969a9e55405dda42b6dd167bd0c5c6900c2b0/backends/candle/src/models/gemma3.rs#L640-L648

Hope that's useful! 🤗

[vishpat]

This is freaking awesome. Thanks for sharing.