Add New Gemma 4 models to the Foundry Local model list

[Jacob Abel (OneDeuxTriSeiGo)]

The new Gemma 4 models are out and they seem like they'd be a lot of fun to experiment with from a device local perspective.

Would it be possible to get the E2B, E4B, 26B A4B, and maybe even the 31B model (or a quantized variant) added to the supported model list?

https://blog.google/innovation-and-ai/technology/developers-tools/gemma-4/

[Jacob Abel (OneDeuxTriSeiGo)]

Also I know in past Gemma versions there have been licensing concerns but Gemma 4 is now under a straight Apache 2.0 license so those issues should be resolved

[Dias Khan (Diaskhan)]

Hi fellows, any updates ?

[babuvt-labs]

Any update on integrating Gemma with Foundry Local?

[Homero Kzam (homerokzam)]

The project is really interesting and has a lot of potential for local AI workflows. 👏
One major limitation today is the model availability and update speed, which feels quite restrictive. The current model catalog evolves slowly and limits experimentation, especially compared to the broader ecosystem.

Any updates ?

[ecyrbe]

You can use olive to experiment if it works. not tested yet, but it's on my todo list to check if this works :

olive auto-opt \
  --model_name_or_path google/gemma-4-E4B-it \
  --trust_remote_code \
  --output_path ./gemma4_e4b_cpu \
  --device cpu \
  --provider CPUExecutionProvider \
  --use_ort_genai \
  --precision int4 \
  --max_seq_len 2048 \
  --log_level 1

and for amd npu :

olive auto-opt \
  --model_name_or_path google/gemma-4-E4B-it \
  --trust_remote_code \
  --output_path ./gemma4_e4b_vitis \
  --device npu \
  --provider VitisAIExecutionProvider \
  --use_ort_genai \
  --precision int4 \
  --max_seq_len 2048 \
  --context_length 128000 \
  --log_level 1

[behrooz bozorgchamy (behroozbc)]

Samuel Kemp (@samuel100), Any updates?

[behrooz bozorgchamy (behroozbc)]

Google introduced a new gemma 4 12 billion parameters I think it's a very good choice to be available.

[Christopher Cashell (cashell)]

The fact that Gemma 4's smaller variants can be run without GPU/NPU makes them particularly interesting for a lot of us, too. Right now, the only Foundry Local non-Speech models that are available for CPU are the qwen3 models. It'd be great to have a few more options available.