Quantized ONNX model fails on WebGPU due to invalid int32 DequantizeLinear (bias zero_point)

[urim-thummim]

Description

Hello, thank you for releasing this model.

While running the provided ONNX model with onnxruntime-web using the WebGPU execution provider, inference fails with the following error:

[WebGPU] Kernel "[DequantizeLinear] inner.encoder.conv1.bias_DequantizeLinear" failed.
Error: In the case of dequantizing int32 there is no zero point.

This prevents the model from running on WebGPU, even though the same model may run on WASM / CPU execution providers.

⸻

Root Cause (Likely)

The failing node is:

inner.encoder.conv1.bias_DequantizeLinear

This node appears to dequantize a bias tensor stored as int32, which is standard for quantized Conv layers.

However, the exported DequantizeLinear node seems to include a zero_point input for an int32 tensor.
According to the ONNX specification:
• DequantizeLinear must not use a zero_point for int32 inputs
(or the zero point must be implicitly assumed to be 0)

WebGPU strictly enforces this rule and errors out, whereas other execution providers may overlook it.

⸻

Expected Behavior

The model should be exported such that int32 bias dequantization does not include a zero_point input, or otherwise follows the ONNX spec for DequantizeLinear(int32).

With a corrected export:
• The model should run correctly on WebGPU
• No runtime kernel error should occur

⸻

Request

Would it be possible to:
1. Re-export / re-publish the ONNX model with corrected DequantizeLinear nodes for int32 bias, and
2. Upload the updated .onnx file(s) to this repository?

This change would greatly improve compatibility with:
• onnxruntime-web
• WebGPU execution provider
• browser-based inference use cases

⸻

Additional Notes
• This issue is not WebGPU-specific logic in the model, but rather strict spec validation in WebGPU EP.
• Many users targeting browser inference (especially WebGPU) are likely to hit this.
• I’m happy to provide a minimal reproducible example or pinpoint the exact node if needed.

Thank you very much for your work and for considering this request.

[marcus-daily]

@urim-thummim Thanks for the report. I've just released v3.1 which has some changes to how quantization/export is performed. Could you test the new version please and see if the same issue exists?

https://www.daily.co/blog/improved-accuracy-in-smart-turn-v3-1/

If so, a minimal code example would be great, as would a PR if you know the root cause.

[urim-thummim]

The release of version 3.1 and the addition of more datasets is good news. However, when we tested it directly in the same environment, it appears to show significantly lower accuracy performance compared to the existing 3.0 version. (onnx-runtime-web, wasm EP)
I would really appreciate it if you could check whether the ONNX export was done correctly.

Additionally, the issue I raised seems to occur identically in 3.1 GPU-only models. However, since I've decided to run it in the wasm environment, it's ok for my situation for now.

/// <reference lib="webworker" />

import { WhisperFeatureExtractor, env as hfEnv } from "@huggingface/transformers";
import * as ort from "onnxruntime-web";
import type { ModelInfo, TurnDetectionResult } from "../../types/turn-detection.js";
import { fetchModelBinary } from "../../utils/modelLoader.js";
import { extractFeatureArray, resampleToTarget } from "./audioWorkerUtils.js";

// preprocessor_config.json of whisper tiny
const TARGET_SAMPLE_RATE = 16000;
const HOP_LENGTH = 160;
const N_FFT = 350;
const MEL_BINS = 80;
const CONTEXT_SECONDS = 8;
const CONTEXT_SAMPLES = TARGET_SAMPLE_RATE * CONTEXT_SECONDS;
const NB_MAX_FRAMES = Math.floor(CONTEXT_SAMPLES / HOP_LENGTH);

hfEnv.allowLocalModels = true;
hfEnv.useBrowserCache = false;
ort.env.wasm.wasmPaths = {
  wasm: new URL("../../assets/ort/ort-wasm-simd-threaded.jsep.wasm", import.meta.url).href,
  mjs: new URL("../../assets/ort/ort-wasm-simd-threaded.jsep.mjs", import.meta.url).href,
};
ort.env.wasm.proxy = false;
ort.env.wasm.numThreads = 1;

class SmartTurnWorkerProcessor {
  private session: ort.InferenceSession | null = null;
  private inputName: string | null = null;
  private outputName: string | null = null;
  private featureExtractor: WhisperFeatureExtractor | null = null;
  private audioBuffer = new Float32Array(0);

  async initialize(modelInfo?: ModelInfo) {
    if (!modelInfo?.onnxFile) throw new Error("Smart Turn requires an onnxFile path.");
    const buffer = await fetchModelBinary(modelInfo.onnxFile);

    this.session = await ort.InferenceSession.create(buffer, {
      executionProviders: ["wasm"],
      graphOptimizationLevel: "all",
      executionMode: "sequential",
    });
    this.inputName = this.session.inputNames[0] ?? null;
    this.outputName = this.session.outputNames[0] ?? null;

    this.featureExtractor = new WhisperFeatureExtractor({
      chunk_length: CONTEXT_SECONDS,
      feature_size: MEL_BINS,
      hop_length: HOP_LENGTH,
      n_fft: N_FFT,
      n_samples: CONTEXT_SAMPLES,
      nb_max_frames: NB_MAX_FRAMES,
      return_attention_mask: false,
      sampling_rate: TARGET_SAMPLE_RATE,
      do_normalize: true,
    });
  }

  async processChunk(data: ArrayBuffer, sampleRate: number, timestamp: number): Promise<TurnDetectionResult | null> {
    if (!this.session || !this.inputName || !this.outputName || !this.featureExtractor) return null;

    const pcm = new Float32Array(data);
    const prepared = resampleToTarget(pcm, sampleRate, TARGET_SAMPLE_RATE);
    this.enqueueSamples(prepared);

    const features = await this.generateInputFeatures();
    if (!features) return null;

    const tensor = new ort.Tensor("float32", features.data, features.dims);
    const results = await this.session.run({ [this.inputName]: tensor });
    const output = results[this.outputName];
    if (!output) return null;

    const probability = 1 - this.extractCompleteProbability(output);
    return { confidence: probability, timestamp };
  }

  private extractCompleteProbability(output: ort.Tensor | undefined) {
    const data = output?.data as Float32Array | number[] | undefined;
    if (!data || (Array.isArray(data) && data.length === 0)) return 0;
    return Array.isArray(data) ? (typeof data[0] === "number" ? data[0] : 0) : data[0] ?? 0;
  }

  private async generateInputFeatures() {
    const audio = this.getContextAudioWindow();
    if (!this.featureExtractor || !audio) return null;

    const processed = await this.featureExtractor(audio, {
      sampling_rate: TARGET_SAMPLE_RATE,
      return_tensors: "np",
      padding: "max_length",
      max_length: CONTEXT_SAMPLES,
      truncation: true,
      do_normalize: true,
    });

    const tensor = processed.input_features;
    const rawData = extractFeatureArray(tensor);
    return { data: this.normalizeFeatureArray(rawData), dims: [1, MEL_BINS, NB_MAX_FRAMES] };
  }

  private normalizeFeatureArray(raw: Float32Array) {
    const expectedLength = MEL_BINS * NB_MAX_FRAMES;
    const data = raw.length === expectedLength ? raw : new Float32Array(expectedLength);
    if (raw.length > expectedLength) data.set(raw.subarray(raw.length - expectedLength));
    else if (raw.length > 0) data.set(raw, expectedLength - raw.length);
    for (let i = 0; i < data.length; i += 1) if (!Number.isFinite(data[i])) data[i] = 0;
    return data;
  }

  private getContextAudioWindow() {
    const available = this.audioBuffer.length;
    if (available >= CONTEXT_SAMPLES) return this.audioBuffer.slice(available - CONTEXT_SAMPLES);
    const window = new Float32Array(CONTEXT_SAMPLES);
    if (available > 0) window.set(this.audioBuffer, CONTEXT_SAMPLES - available);
    return window;
  }

  private enqueueSamples(samples: Float32Array) {
    if (samples.length === 0) return;
    const current = this.audioBuffer;
    const totalLength = Math.min(current.length + samples.length, CONTEXT_SAMPLES);
    const newBuffer = new Float32Array(totalLength);

    const samplesToCopy = Math.min(samples.length, totalLength);
    newBuffer.set(samples.subarray(samples.length - samplesToCopy), totalLength - samplesToCopy);

    const existingSpace = totalLength - samplesToCopy;
    const existingToCopy = Math.min(current.length, existingSpace);
    if (existingToCopy > 0) {
      const offset = existingSpace - existingToCopy;
      newBuffer.set(current.subarray(current.length - existingToCopy), offset);
    }

    this.audioBuffer = newBuffer;
  }
}

export {};

This code is the result of requesting the LLM to organize and output the actual code, leaving only the core flow to indirectly grasp the inference environment.

[marcus-daily]

Thanks @urim-thummim. Just checking how you're measuring the accuracy? We ran our own benchmarks on the exported ONNX model and see a big accuracy improvement compared to v3.0. If you're able to send me some audio samples of cases which fail, we might be able to train the model to handle these better in future.

[ldenoue]

@urim-thummim do you have a repo somewhere with the full example? I'm also toying with running SmartTurn in a web worker.

[urim-thummim]

@urim-thummim do you have a repo somewhere with the full example? I'm also toying with running SmartTurn in a web worker.

I'm sorry, but due to copyright issues, I cannot disclose the entire code.

[urim-thummim]

Thanks @urim-thummim. Just checking how you're measuring the accuracy? We ran our own benchmarks on the exported ONNX model and see a big accuracy improvement compared to v3.0. If you're able to send me some audio samples of cases which fail, we might be able to train the model to handle these better in future.

We confirmed that the Java-based ONNX Runtime functions correctly in both CPU and NPU environments. and when it is used with VAD, we also confirmed that version 3.1 demonstrates significantly higher accuracy. 👍

The previously encountered issue appears to occur only in the web ONNX Runtime, particularly in webGPU environments.

            "onnxruntime-web": "^1.23.2",