[WebGPU] com.microsoft.QMoE produces invalid results for certain attribute combinations

[xenova]

Describe the issue

The com.microsoft.QMoE operation fails with large correctness issue for the following attributes:

cc @guschmue

To reproduce

import onnxruntime as ort
from huggingface_hub import hf_hub_download

path = hf_hub_download(repo_id="Xenova/onnxruntime-webgpu-test-data", filename="fail_673_standalone.onnx", repo_type="dataset")
cpu_session = ort.InferenceSession(path, providers=['CPUExecutionProvider'])
cpu_result = cpu_session.run(None, {})

webgpu_session = ort.InferenceSession(path, providers=['WebGpuExecutionProvider'])
webgpu_result = webgpu_session.run(None, {})

diff = cpu_result[0] - webgpu_result[0]
print('Diff stats: ', f'min={diff.min()}, max={diff.max()}, mean={diff.mean()}, std={diff.std()}')

produces this error:

Diff stats:  min=-7.402378082275391, max=8.158369064331055, mean=-0.0036832974292337894, std=0.3670535087585449

Urgency

medium-high -- blocks certain models which use this operation.

ONNX Runtime Installation

Built from Source

ONNX Runtime Version or Commit ID

main

Execution Provider

'webgpu' (WebGPU)

[guschmue]

I need to take a look. We tested webgpu mostly against transformers implementation of gpu-oss-20b and numbers align there.

[guschmue]

cool, fix is already there.

[xenova]

I need to take a look. We tested webgpu mostly against transformers implementation of gpu-oss-20b and numbers align there.

Indeed, this is because swiglu_limit is set to 7 for gpt-oss: https://huggingface.co/openai/gpt-oss-20b/blob/main/config.json#L71

So, when it is not set (like https://huggingface.co/LiquidAI/LFM2-8B-A1B), zero is used (incorrect)