[Performance] CreateSession takes very long time to load .onnx file when working with FileFuzzer tool

[chenyihong0504]

Describe the issue

We hit this perf issue when doing Fuzz testing using FileFuzzer tool for Speech Runtime, check below for what we got. Normally it is expected that scenario 1 is faster than scenario 2, but it is too slow for CreateSession() function which is blocking us to do the fuzz testing.

Scenario 1: E2E decoding scenario using Command Prompt Windows => Took about 4 secs in total, CreateSession() for big encoder.quantqint8.onnx file (87.8MB) took less than 2 secs, total time for 4 .onnx files is around 2 secs.
Scenario 2: E2E decoding scenario through FileFuzzer tool => Took about 67 secs in total, CreateSession() for big encoder.quantqint8.onnx file (87.8MB) took about 51 secs, total time for 4 .onnx files is about 56 secs.

Notes: we verified with the latest version [v1.20.1], this issue also repro.

To reproduce

Here is a local setup to repro the issue using en-US model:
\MixedRealityFS\PRIVATE\Speech\speechshare\USR\yihche\4ONNXRuntimeComponent\ReproFuzzBug11045

1. Copy ReproFuzzBug11045 to local machine e.g. at E:\FuzzTesting
2. Install FileFuzzer 3.0 with MSI files at Software\FileFuzzer 3.0\amd64
   2.1 Install FCL.msi first
   2.2 Then install FileFuzzer.msi
   Default installation path is at C:\Program Files\Microsoft\FileFuzzer
3. Modify FileFuzzer_ConfigFile.xml for , and to matching your local setup path.
   Here are the examples:
   E:\FuzzTesting\ReproFuzzBug11045\audio\en-US
   E:\FuzzTesting\ReproFuzzBug11045\output-Fuzzer
   E:\FuzzTesting\ReproFuzzBug11045\ReproBug11045_SlowIssue.bat
4. Open FileFuzzer UI as administrator from C:\Program Files\Microsoft\FileFuzzer\FileFuzzerConfig.exe
   4.1 Click "Load Settings" and select FileFuzzer_ConfigFile.xml
   4.2 Click "Start" button.
5. Check testing results in path

Urgency

It is a blocking issue for us to do Fuzz testing, and we have to reduce seed and iteration numbers as temporary workaround.

Platform

Windows

OS Version

Windows 11

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

v1.15.1

ONNX Runtime API

C++

Architecture

X64

Execution Provider

Default CPU

Execution Provider Library Version

No response

Model File

No response

Is this a quantized model?

Yes

[github-actions]

This issue has been automatically marked as stale due to inactivity and will be closed in 30 days if no further activity occurs. If further support is needed, please provide an update and/or more details.