ONNXRuntime Optimization Causes Output Discrepancy in BiasDropout Operator

[Thrsu]

Describe the issue

The outputs of an ONNX model optimized with ONNXRuntime show discrepancies compared to the original model, particularly for outputs C and D, which are related to the BiasDropout operator.

Actual Behavior:

Traceback (most recent call last):
      ...
      np.testing.assert_allclose(r1, r2, atol=1e-3, rtol=1e-3)
  File "/root/miniconda3/lib/python3.12/site-packages/numpy/testing/_private/utils.py", line 1504, in assert_allclose
    assert_array_compare(compare, actual, desired, err_msg=str(err_msg),
  File "/root/miniconda3/lib/python3.12/contextlib.py", line 81, in inner
    return func(*args, **kwds)
           ^^^^^^^^^^^^^^^^^^^
  File "/root/miniconda3/lib/python3.12/site-packages/numpy/testing/_private/utils.py", line 797, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Not equal to tolerance rtol=0.001, atol=0.001

Mismatched elements: 1011 / 3072 (32.9%)
Max absolute difference: 9.99115
Max relative difference: 10.000002
 x: array([[[0.589064, 6.314322, 0.172512, ..., 0.03122 , 0.27859 ,
         0.55074 ]]], dtype=float32)
 y: array([[[0.589064, 0.574029, 0.172512, ..., 0.343425, 0.27859 ,
         6.058143]]], dtype=float32)

To reproduce

1. Download the model
2. Run the below script:

import onnx
import onnxruntime as ort
import numpy as np
from onnxruntime.transformers import optimizer

model_path = "inconsis4.onnx"
optimized_model_path = f"./opt.onnx"
sess_options = ort.SessionOptions()
sess_options.graph_optimization_level = ort.GraphOptimizationLevel.ORT_DISABLE_ALL
this_provider_list = ort.get_available_providers()

original_session = ort.InferenceSession(model_path, sess_options, providers=this_provider_list)
input_data = {"v8_0": np.random.rand(48, 1, 39, 27).astype(np.float64)}
output_names = [output.name for output in original_session.get_outputs()]
original_result = original_session.run(output_names, input_data)

optimized_model = optimizer.optimize_model(model_path, opt_level=1, use_gpu=True)
optimized_model.save_model_to_file(optimized_model_path)
optimized_session = ort.InferenceSession(optimized_model_path, providers=this_provider_list)
optimized_model = onnx.load(optimized_model_path)
optimized_result = optimized_session.run(output_names, input_data)
for r1, r2 in zip(original_result, optimized_result):
    np.testing.assert_allclose(r1, r2, atol=1e-3, rtol=1e-3)

Urgency

No response

Platform

Linux

OS Version

Ubuntu 20.04

ONNX Runtime Installation

Built from Source

ONNX Runtime Version or Commit ID

5c1b7cc

ONNX Runtime API

Python

Architecture

X64

Execution Provider

CUDA

Execution Provider Library Version

No response