[WebGPU] JSEP crashes when running Gemma 3 (1b-it)

[KevinL10]

System Info

version: 3.8.0
browser: Chrome 142.0.7444.176
os: macOS 15.6.1

Environment/Platform

• Website/web-app
• Browser extension
• Server-side (e.g., Node.js, Deno, Bun)
• Desktop app (e.g., Electron)
• Other (e.g., VSCode extension)

Description

Loading onnx-community/gemma-3-1b-it-ONNX-GQA with either device: "webgpu" or device: "wasm" causes a JSEP crash (see #1239).

The raised error in the reproduction case below is just the string 3436070408, but in some cases it seems like the model load throws a stack trace that looks like:

worker.ts:78 Error details: Aborted(). Build with -sASSERTIONS for more info. RuntimeError: Aborted(). Build with -sASSERTIONS for more info.
    at L (https://cdn.jsdelivr.net/npm/@huggingface/transformers@3.8.0/dist/ort-wasm-simd-threaded.jsep.mjs:22:101)
    at tb (https://cdn.jsdelivr.net/npm/@huggingface/transformers@3.8.0/dist/ort-wasm-simd-threaded.jsep.mjs:60:98)
    at https://cdn.jsdelivr.net/npm/@huggingface/transformers@3.8.0/dist/ort-wasm-simd-threaded.jsep.wasm:wasm-function[1255]:0x1365fa
    at https://cdn.jsdelivr.net/npm/@huggingface/transformers@3.8.0/dist/ort-wasm-simd-threaded.jsep.wasm:wasm-function[303]:0x404b9
    at https://cdn.jsdelivr.net/npm/@huggingface/transformers@3.8.0/dist/ort-wasm-simd-threaded.jsep.wasm:wasm-function[545]:0x75f9b
    at https://cdn.jsdelivr.net/npm/@huggingface/transformers@3.8.0/dist/ort-wasm-simd-threaded.jsep.wasm:wasm-function[82]:0xd24b
    at https://cdn.jsdelivr.net/npm/@huggingface/transformers@3.8.0/dist/ort-wasm-simd-threaded.jsep.wasm:wasm-function[153]:0x1bdf9
    at https://cdn.jsdelivr.net/npm/@huggingface/transformers@3.8.0/dist/ort-wasm-simd-threaded.jsep.wasm:wasm-function[690]:0x9d8cc
    at https://cdn.jsdelivr.net/npm/@huggingface/transformers@3.8.0/dist/ort-wasm-simd-threaded.jsep.wasm:wasm-function[10708]:0xed51eb
    at https://cdn.jsdelivr.net/npm/@huggingface/transformers@3.8.0/dist/ort-wasm-simd-threaded.jsep.wasm:wasm-function[1110]:0x114f19 RuntimeError: Aborted(). Build with -sASSERTIONS for more info.
    at L (ort-wasm-simd-threaded.jsep.mjs:22:101)
    at tb (ort-wasm-simd-threaded.jsep.mjs:60:98)
    at ort-wasm-simd-threaded.jsep.wasm:0x1365fa
    at ort-wasm-simd-threaded.jsep.wasm:0x404b9
    at ort-wasm-simd-threaded.jsep.wasm:0x75f9b
    at ort-wasm-simd-threaded.jsep.wasm:0xd24b
    at ort-wasm-simd-threaded.jsep.wasm:0x1bdf9
    at ort-wasm-simd-threaded.jsep.wasm:0x9d8cc
    at ort-wasm-simd-threaded.jsep.wasm:0xed51eb

The same code snippet works fine with Node (tested on v24.7.0).

@xenova maybe we could re-upload the onnx models with the latest conversion scripts?

Reproduction

The example from the docs causes a crash on webgpu/wasm:

import { pipeline } from "@huggingface/transformers";

const generator = await pipeline(
  "text-generation",
  "onnx-community/gemma-3-1b-it-ONNX-GQA",
  { dtype: "q4" },
);

[xenova]

Thanks for the heads up! I have updated the model, which should fix the issue for fp16 and q4f16 variants. Unfortunately, the q4 and fp32 versions may have some issues when running in-browser currently because the embedding layer is extremely large for this model (causing buffer size issues).

Example usage (jsfiddle):

import { pipeline, TextStreamer } from "https://cdn.jsdelivr.net/npm/@huggingface/transformers@3.8.0";

// Create a text generation pipeline
const generator = await pipeline(
  "text-generation",
  "onnx-community/gemma-3-1b-it-ONNX-GQA",
  { dtype: "q4f16", device: "webgpu" },
);

// Define the list of messages
const messages = [
  { role: "system", content: "You are a helpful assistant." },
  { role: "user", content: "What is the capital of France?" },
];

// Generate a response
const output = await generator(messages, {
  max_new_tokens: 512,
  do_sample: false,
  streamer: new TextStreamer(generator.tokenizer, { skip_prompt: true })
});
console.log(output[0].generated_text.at(-1).content);

[KevinL10]

Thanks, works for me now. FWIW I've seen this happen in a lot of other onnx models as well (e.g. onnx-community/Qwen3-1.7B-ONNX), probably because of the buffer size issue.

I've also seen a similar case for onnx-community/gemma-3-270m-it-ONNX where sampling from the f16 or q4f16 model returns all NaNs (and the generated output is an empty string). I imagine the issue is something other than buffer size for that.

[KevinL10]

Aside, it would be nice to publicly track quantization support for the common ONNX models somewhere, since it seems unclear (and sometimes flaky) whether a model can run in-browser. I made a quick example with some common models: https://transformers-js-support.vercel.app/

[xenova]

I've also seen a similar case for onnx-community/gemma-3-270m-it-ONNX where sampling from the f16 or q4f16 model returns all NaNs (and the generated output is an empty string).

Oh right, I've made a same similar fix here -- thanks.

[xenova]

Aside, it would be nice to publicly track quantization support for the common ONNX models somewhere, since it seems unclear (and sometimes flaky) whether a model can run in-browser. I made a quick example with some common models: https://transformers-js-support.vercel.app/

Thanks! Very helpful :) And yes, we're aiming to improve this... especially trying to choose good defaults and suggesting to users which are "good/working" models. cc @nico-martin for viz.

[sroussey]

Aside, it would be nice to publicly track quantization support for the common ONNX models somewhere, since it seems unclear (and sometimes flaky) whether a model can run in-browser. I made a quick example with some common models: https://transformers-js-support.vercel.app/

Might want to update, as there are some fixes like onnx-community/gemma-3-270m-it-ONNX with q4f16 support.