[WebGPU] Failed to find kernel for Cast(13) for WebGpuExecutionProvider

[xenova]

Describe the issue

Error: Exception during initialization: /Users/cloudtest/vss/_work/1/s/onnxruntime/core/optimizer/transformer_memcpy.cc:467 bool onnxruntime::TransformerMemcpyImpl::ProcessInitializers(const KernelRegistryManager &, const InitializedTensorSet &, const logging::Logger &) status.IsOK() was false. Failed to find kernel for Cast(13) (node:'/transformer/Cast_2_output_cast0' ep:'WebGpuExecutionProvider'). Op with name (/transformer/Cast_2_output_cast0) domain () and type (Cast) kernel is not supported in WebGpuExecutionProvider. Encountered following errors: ( Version mismatch. node_version: 13 kernel start version: 6 kernel_end_version: 8
 Version mismatch. node_version: 13 kernel start version: 9 kernel_end_version: 12
Kernel found kernel in the supported version range (node_version: 13). However the types are incompatible. This op has been implemented only for the following types (tensor(float16),tensor(float),tensor(int32),tensor(uint32),tensor(bool),), but the node in the model has the following type (tensor(int64))
 Version mismatch. node_version: 13 kernel start version: 19 kernel_end_version: 20
 Version mismatch. node_version: 13 kernel start version: 21 kernel_end_version: 22
 Version mismatch. node_version: 13 kernel start version: 23 kernel_end_version: 2147483647

To reproduce

1. Get RFDetr model

wget https://huggingface.co/onnx-community/rfdetr_nano-ONNX/resolve/main/onnx/model_fp16.onnx -O model_fp16.onnx

2. Load inference session

python -c "import onnxruntime as ort; ort.InferenceSession('model_fp16.onnx', providers=['WebGpuExecutionProvider'])"

Urgency

low-medium. Users can use fp32 in meantime

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

main

Execution Provider

'webgpu' (WebGPU)

[fs-eire]

this can be reproduced not only in 1.24.1, but also in 1.22 and 1.23

still doing further investigating

[fs-eire]

While it may take a while to fix the root cause, I would suggest to optimize the model to workaround.

the model contains a pattern that 2 Cast operators are in sequence, while one uses int64 -> float and the other one uses float -> float16. The first one is not supported by webgpu (because webgpu does not deal with int64 by default), and the second on is supported. 2 Cast nodes are assigned to different EPs but the graph optimizer does not know this detail and try to fuse the 2 nodes into one, so this causes the problem.

If we are possible to edit the graph to make the Cast nodes into one (int64->f16), the error should not occur

[xenova]

If we are possible to edit the graph to make the Cast nodes into one (int64->f16), the error should not occur

Theoretically yes, I could update the models, but it is worth pointing out that they were produced with torch.onnx.export, which means there will certainly be many others that could have this issue too. So, solving it for just this model series may be shortsighted.