[Bug] InvalidArgument Error After Optimizing Model with ONNX Runtime

[Thrsu]

Describe the issue

After optimizing an ONNX model using ONNX Runtime, I encounter an error when running the optimized model. The error message indicates that the Gelu node expects an input with at least 1 dimension, but the input received has 0 dimensions. This issue does not occur with the original model and it occurs regardless of the optimization level (opt_level=0, opt_level=1, opt_level=99).

Error Message:

Traceback (most recent call last):
  File "/share_container/optfuzz/ONNX/reproduce/Gelu_123.py", line 14, in <module>
    optimized_result = optimized_session.run(output_names, input_data)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/software/onnxruntime/build/Linux/Release/onnxruntime/capi/onnxruntime_inference_collection.py", line 266, in run
    return self._sess.run(output_names, input_feed, run_options)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Non-zero status code returned while running Gelu node. Name:'Gelu' Status Message: Input 0 is expected to have 1 or more dimensions, got 0

To reproduce

1. Download the model
2. Run the below script:

import onnx
import onnxruntime as ort
import numpy as np
from onnxruntime.transformers import optimizer

model_path = "Gelu.onnx"
optimized_model_path = f"./opt.onnx"
optimized_model = optimizer.optimize_model(model_path, opt_level=99, use_gpu=True)
optimized_model.save_model_to_file(optimized_model_path)
this_provider_list = ort.get_available_providers()
optimized_session = ort.InferenceSession(optimized_model_path, providers=this_provider_list)
input_data = {'v8_0': np.random.rand(1, 1).astype(np.float32)}
output_names = [output.name for output in optimized_session.get_outputs()]
optimized_result = optimized_session.run(output_names, input_data)

Urgency

No response

Platform

Linux

OS Version

Ubuntu 20.04

ONNX Runtime Installation

Built from Source

ONNX Runtime Version or Commit ID

5c1b7cc

ONNX Runtime API

Python

Architecture

X64

Execution Provider

CUDA

Execution Provider Library Version

No response

[tianleiwu]

If your model need the Gelu fusion, you can remove the Squeeze before Div node as in the following graph. Given a tensor with shape [1], applying squeeze() on axis 0 will remove this dimension, resulting in a tensor with an empty shape []. This signifies a scalar tensor, containing a single element without any dimensions. However, Gelu operator currently does not work on scalar.

.

For the Gelu fusion bug, we will take a look to add some checking later to avoid such case. Thanks for reporting.

[Thrsu]

Thank you for your detailed analysis and explanation！

[github-actions]

This issue has been automatically marked as stale due to inactivity and will be closed in 30 days if no further activity occurs. If further support is needed, please provide an update and/or more details.