[Web] Facing this error in WebGPU: Model warmup failed: Error: input 'detection' is missing in 'feeds'.

[KabirSinghMehrok]

Describe the issue

I am trying to run yolov8 nano model in browser using onnx. The model is working fine when executionProvider is cpu, but it fails to work when I set execution environment as webgpu. The following error occurs when trying to initalize the model by parsing dummy values

Model warmup failed: Error: input 'detection' is missing in 'feeds'.

To reproduce

Here is the code initializing the model

export const initModel = async (gpuAvailable=true) => {
  // ort.env.wasm.wasmPaths = `./`;
  ort.env.wasm.wasmPaths = `${window.location.origin}/`;


  let executionProvider = 'cpu';
  if (useGPU && gpuAvailable) {
    const gpuAvailable = await isWebGPUAvailable();
    if (gpuAvailable) {
      executionProvider = 'webgpu';
      console.log('Using WebGPU execution provider');
    } else {
      console.log('WebGPU requested but not available, falling back to CPU');
    }
  }

  console.log('start initializing the model');
  // TODO: check if this array buffer causes potential issue
  const modelPath = await fetch(modelPathOnServer).then((response) => response.arrayBuffer());
  const nmsPath = await fetch(nmsPathOnServer).then((response) => response.arrayBuffer());

  let yolov8 = null, nms=null
  try {
    // create session
    const [a, b] = await Promise.all([
      ort.InferenceSession.create(modelPath, { executionProviders: [ executionProvider ] }),
      ort.InferenceSession.create(nmsPath, { executionProviders: [ executionProvider ] }),
    ]);
    yolov8=a;
    nms=b;
  } catch (error) {
    console.error('Error caught while creating session', error);
  }


  console.log('nms is', nms, 'model is', yolov8);
  
  // warmup main model
  const tensor = new ort.Tensor(
    "float32",
    new Float32Array(modelInputShape.reduce((a, b) => a * b)),
    modelInputShape
  );

  try {
    const {output0}  = await yolov8.run({ images: tensor });

    console.log('output0 and other values', output0, topk, iouThreshold, scoreThreshold);
    await nms.run({
      input: output0,
      topk: topk,
      iou_threshold: iouThreshold,
      score_threshold: scoreThreshold,
    });
    output0.dispose();
    console.log('Model warmup successful');
  } catch (error) {
    console.error('Model warmup failed:', error);
    // If WebGPU fails, fall back to CPU
    if (executionProvider === 'webgpu') {
      console.log('Falling back to CPU execution provider');
      return initModel(false); // Recursively call with useGPU set to false
    }
  }

  session = {net: yolov8, nms:nms };
  console.log('session created', {net: yolov8, nms:nms});
  console.log('session has a run', session, session.net.run);
  console.log('end initializing the model');

  return session
};

The following code snippet (part of the code above) to warmup nms was taken from another repository. This is where the error is thrown

await nms.run({
      input: output0,
      topk: topk,
      iou_threshold: iouThreshold,
      score_threshold: scoreThreshold,
    });

Please let me know regarding any other clarification I should add. Working on my last 2 braincells :P

Urgency

This is an urgent issue. I have been unable to find reason behind this error, nor is this highlighted well in any documentation available online

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.20.1

Execution Provider

'webgpu' (WebGPU)

[skottmckay]

An onnx model has a required set of inputs. View your model in Netron to see what they're called. Is there an input named 'detection'? If so, you have to provide something with that name when running the model.