DML EP shape mismatch failure in InferAndVerifyOutputSizes if torch export dynamo=True

[LigaLiao]

2025-12-11 23:11:51.7008023 [E:onnxruntime:, inference_session.cc:2545 onnxruntime::InferenceSession::Initialize::<lambda_73d8de3ce9bc7d47058d99ebffb3c8e5>::operator ()] Exception during initialization: E:_work\1\s\onnxruntime\core\providers\dml\DmlExecutionProvider\src\MLOperatorAuthorImpl.cpp(2853)\onnxruntime_pybind11_state.pyd!00007FF9C3BBA59D: (caller: 00007FF9C3BDAA58) Exception(1) tid(c4e4) 80070057 ?*************** EP Error ***************
EP Error 'utf-8' codec can't decode byte 0xb2 in position 286: invalid start byte when using ['DmlExecutionProvider']
Falling back to ['CPUExecutionProvider'] and retrying.

1.clone
https://github.com/ladaapp/lada

2.export
lada/scripts/convert_restoration_to_onnx2.py
Use this https://github.com/nusu-github/deform_conv2d_onnx_exporter Do not use pip install deform_conv2d_onnx_exporter

cmd: lada>python scripts/convert_restoration_to_onnx2.py

3.test
lada/model_weights/test2.py

cmd: lada\model_weights>python test2.py

run collect_env:

PyTorch version: 2.9.1+cpu
Is debug build: False
CUDA used to build PyTorch: Could not collect
ROCM used to build PyTorch: N/A

OS: Microsoft Windows 11 专业版 (10.0.26100 64 位)
GCC version: Could not collect
Clang version: Could not collect
CMake version: version 3.30.8
Libc version: N/A

Python version: 3.12.12 | packaged by Anaconda, Inc. | (main, Oct 21 2025, 20:05:38) [MSC v.1929 64 bit (AMD64)] (64-bit runtime)
Python platform: Windows-11-10.0.26100-SP0
Is CUDA available: False
CUDA runtime version: Could not collect
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 3080
Nvidia driver version: 581.80
cuDNN version: Could not collect
Is XPU available: False
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Name: AMD Ryzen 9 9950X 16-Core Processor
Manufacturer: AuthenticAMD
Family: 107
Architecture: 9
ProcessorType: 3
DeviceID: CPU0
CurrentClockSpeed: 4300
MaxClockSpeed: 4300
L2CacheSize: 16384
L2CacheSpeed: None
Revision: 17408

Versions of relevant libraries:
[pip3] deform-conv2d-onnx-exporter==1.2.0
[pip3] numpy==2.3.4
[pip3] onnx==1.20.0
[pip3] onnx-ir==0.1.12
[pip3] onnxruntime-directml==1.23.0
[pip3] onnxscript==0.5.6
[pip3] onnxsim==0.4.36
[pip3] onnxslim==0.1.78
[pip3] torch==2.9.1
[pip3] torchvision==0.24.1
[conda] numpy 2.3.4 pypi_0 pypi
[conda] torch 2.9.1 pypi_0 pypi
[conda] torchvision 0.24.1 pypi_0 pypi

report:onnx_export_2025-12-17_18-11-38-222918_success.md

onnx:https://mega.nz/file/0RlREAgY#6vBIxOvaZ-MRuRvGbz_9gi-R0tKMWjBuZlRUICTQMUs

[fdwr]

onnxruntime\core\providers\dml\DmlExecutionProvider\src\MLOperatorAuthorImpl.cpp(2853)

The error line is here, suggesting either a shape mismatch or unsupported dynamic dimensions, which need to be set via freeDimensionOverrides / AddFreeDimensionOverride. Here is
InferAndVerifyOutputSizes:

    void InferAndVerifyOutputSizes(...)
    {
        ...
                for (uint32_t output_dim = 0; output_dim < outputShapes.GetShape(outputIndex).size(); ++output_dim)
                {
                    if (shape.dim(output_dim).has_dim_value())
                    {
                        int64_t expected_size = shape.dim(output_dim).dim_value();
                        int64_t actual_size = outputShapes.GetShape(outputIndex)[output_dim];
                        assert(expected_size == actual_size);
>>>>                    ML_CHECK_BOOL(expected_size == actual_size);
                    }
                }

The reported error message though makes no sense 🤨: "EP Error 'utf-8' codec can't decode byte 0xb2 in position 286: invalid start byte". Btw, mega.nz is blocked by the company from here.

[LigaLiao]

onnxruntime\core\providers\dml\DmlExecutionProvider\src\MLOperatorAuthorImpl.cpp(2853)

The error line is here, suggesting either a shape mismatch or unsupported dynamic dimensions, which need to be set via freeDimensionOverrides / AddFreeDimensionOverride:

            for (uint32_t output_dim = 0; output_dim < outputShapes.GetShape(outputIndex).size(); ++output_dim)
            {
                if (shape.dim(output_dim).has_dim_value())
                {
                    int64_t expected_size = shape.dim(output_dim).dim_value();
                    int64_t actual_size = outputShapes.GetShape(outputIndex)[output_dim];
                    assert(expected_size == actual_size);

               ML_CHECK_BOOL(expected_size == actual_size);

                }
            }

The reported error message though makes no sense 🤨: "EP Error 'utf-8' codec can't decode byte 0xb2 in position 286: invalid start byte". Btw, mega.nz is blocked by the company from here.

It shouldn't be that simple; I commented out "dynamic_shapes=dynamic_shapes,", the shape is fixed as (1, 5, 3, 256, 256), but the DML backend still throws an error. When falling back to the CPU backend, no errors are reported (although I'm not sure if the result is correct).

onnx: https://drive.google.com/file/d/1bacS_W2u7toiDMTPX7Qrn1F91OBRbTI4/view?usp=sharing

[LigaLiao]

After verification, When falling back to the CPU backend, its output is correct

[LigaLiao]

This should be a bug in DML

[fdwr]

✅ Reproduced locally with fixed dimensions.

CPU:

DML:

[LigaLiao]

✅ Reproduced locally with fixed dimensions.✅ 已在本地以固定尺寸复现。

CPU:

## DML:

I hope this can help you with debugging: #26944

[microsoft-github-policy-service]

Applying stale label due to no activity in 30 days. Please react to the issue by assigning it or labeling it as 'contributions welcome', 'documentation', 'feature request', 'regression', or 'no stale'.