DML Execution Provider WebNN Conformance Test Failures on Qualcomm

This issue tracks subtest failures that have been observed in the WebNN conformance suite for WinML's DMLExecutionProvider. Report attached for reference. Ideally, the suite would pass cleanly, so the subtests here should be investigated and fixed where possible.

To reproduce, on a compatible AMD device, run the wpt suite using the [webnn-test](https://github.com/[webgfx/webnn-test](https://github.com/webgfx/webnn-test)) tool. I used Chrome Canary for my repro. 

[Test Results Link](https://github.com/user-attachments/files/25112853/20260203222352-with-subtest-failures.html)

The following arguments are passed on the browser command line:
`--webnn-ort-ep-device=DmlExecutionProvider,0x4d4f4351,0x36334330 --webnn-ort-logging-level=VERBOSE`

## Summary of Failures by Category

| Category | Issue Type | Affected Tests |
|----------|-----------|----------------|
| Data Type Support | `dequantizeLinear` doesn't support `int32` | dequantizeLinear, qdq_subgraph |
| Data Type Support | `gatherElements`/`gatherND` don't support `uint32` indices | gatherElements, gatherND |
| Data Type Support | `int64` not supported for activation ops | relu, prelu |
| Session Creation | 5D tensor broadcasting failures | matmul, triangular |
| Session Creation | int4/uint4 odd-size tensor failures | dequantizeLinear, quantizeLinear |
| Session Creation | `batchNormalization` axis=0 for 1D tensors | batch_normalization |
| Precision/ULP | Float16/Float32 precision exceeds tolerance | batch_normalization_constant, dequantizeLinear, layer_normalization, quantizeLinear, subgraph |
| Context Lost | `Context is lost` errors | quantizeLinear, qdq_subgraph |
| Fusion Issues | Conv2d + activation fusion incorrect results | subgraph |
| API Gap | `graph.devices` is empty | graph_devices |

---

## Detailed Failing Tests

### 1. `batch_normalization` (2 failing / 25 total)

| Subtest | Error |
|---------|-------|
| `[optional] batchNormalization float32 1D tensor options.axis=0` | Failed to create session |
| `[optional] batchNormalization float16 1D tensor options.axis=0` | Failed to create session |

### 2. `batch_normalization_constant` (1 failing / 3 total)

| Subtest | Error |
|---------|-------|
| `[optional] batchNormalization float32 2D constant tensors default options` | ULP distance 12 (expected ≤6) |

### 3. `dequantizeLinear` (12 failing / 33 total)

| Subtest | Error |
|---------|-------|
| `[required] dequantizeLinear int8 4D tensor broadcasting float32 4D scale and int8 4D zeroPoint` | ULP distance 33982206 (expected ≤1) |
| `[optional] dequantizeLinear uint4 1D tensor of odd size with float32 1D scale` | Failed to create session |
| `[optional] dequantizeLinear uint4 4D tensor with broadcasting float32 4D scale and uint4 4D zeroPoint` | Failed to create session |
| `[optional] dequantizeLinear int4 1D tensor of odd size with float32 1D scale` | Failed to create session |
| `[optional] dequantizeLinear int32 1D tensor with float32 1D scale` | Unsupported data type int32 |
| `[required] dequantizeLinear with float32 3D scale as an intermediate node` | Unsupported data type int32 |
| `[optional] dequantizeLinear int8 4D tensor broadcasting float16 4D scale and int8 4D zeroPoint` | ULP distance 4149 (expected ≤1) |
| `[optional] dequantizeLinear uint4 1D tensor of odd size with float16 1D scale` | Failed to create session |
| `[optional] dequantizeLinear uint4 4D tensor with broadcasting float16 4D scale and uint4 4D zeroPoint` | Failed to create session |
| `[optional] dequantizeLinear int4 1D tensor of odd size with float16 1D scale` | Failed to create session |
| `[optional] dequantizeLinear int32 1D tensor with float16 1D scale` | Unsupported data type int32 |
| `[optional] dequantizeLinear with float16 3D scale as an intermediate node` | Unsupported data type int32 |

### 4. `gatherElements` (2 failing / 12 total)

| Subtest | Error |
|---------|-------|
| `[optional] gatherElements float32 2D input and uint32 indices options.axis=1` | Unsupported data type uint32 for indices |
| `[optional] gatherElements float16 2D input and uint32 indices options.axis=1` | Unsupported data type uint32 for indices |

### 5. `gatherND` (2 failing / 17 total)

| Subtest | Error |
|---------|-------|
| `[optional] gatherND float32 4D input and 1D uint32 indices` | Unsupported data type uint32 for indices |
| `[optional] gatherND float16 4D input and 1D uint32 indices` | Unsupported data type uint32 for indices |

### 6. `graph_devices` (1 failing / 1 total)

| Subtest | Error |
|---------|-------|
| `test graph.devices` | graph.devices is empty |

### 7. `layer_normalization` (1 failing / 26 total)

| Subtest | Error |
|---------|-------|
| `[required] layerNormalization float32 4D tensor all options` | ULP distance 19 (expected ≤14) |

### 8. `matmul` (6 failing / 23 total)

| Subtest | Error |
|---------|-------|
| `[required] matmul float32 5D and 5D tensors, broadcast the two leftmost dimensions of inputB` | Failed to create session |
| `[required] matmul float32 5D and 5D tensors, broadcast the leftmost dimensions of inputB` | Failed to create session |
| `[required] matmul float32 5D and 2D tensors` | Failed to create session |
| `[required] matmul float16 5D and 5D tensors, broadcast the two leftmost dimensions of inputB` | Failed to create session |
| `[required] matmul float16 5D and 5D tensors, broadcast the leftmost dimensions of inputB` | Failed to create session |
| `[required] matmul float16 5D and 2D tensors` | Failed to create session |

### 9. `prelu` (1 failing / 28 total)

| Subtest | Error |
|---------|-------|
| `[optional] prelu int64 2D constant tensors` | Failed to create session |

### 10. `quantizeLinear` (6 failing / 29 total)

| Subtest | Error |
|---------|-------|
| `[required] quantizeLinear float32 4D tensor broadcasting scale and zeroPoint` | ULP distance 16 (expected ≤1) |
| `[optional] quantizeLinear float32 tensor with int4 zeroPoint which has even size` | Context is lost |
| `[optional] quantizeLinear float32 2D tensor with int4 zeroPoint which has even size` | Failed to create session |
| `[optional] quantizeLinear float32 tensor with uint4 zeroPoint which has odd size` | Failed to create session |
| `[optional] quantizeLinear float16 4D tensor broadcasting scale and zeroPoint` | ULP distance 16 (expected ≤1) |
| `[optional] quantizeLinear float16 2D tensor with int4 zeroPoint which has even size` | Failed to create session |
| `[optional] quantizeLinear float16 tensor with uint4 zeroPoint which has odd size` | Failed to create session |

### 11. `qdq_subgraph` (9 failing / 42 total)

| Subtest | Error |
|---------|-------|
| `[optional] quantized conv2d` | Unsupported data type int32 for dequantizeLinear |
| `[optional] dequantizeLinear -> conv2d -> clamp -> quantizeLinear` | Unsupported data type int32 for dequantizeLinear |
| `[optional] dequantizeLinear -> conv2d -> relu -> quantizeLinear` | Unsupported data type int32 for dequantizeLinear |
| `[optional] quantized convTranspose2d` | Unsupported data type int32 for dequantizeLinear |
| `[optional] quantized conv2d with padding` | Unsupported data type int32 for dequantizeLinear |
| `[optional] quantized gemm with bias` | Unsupported data type int32 for dequantizeLinear |
| `[required] quantized leaky relu` | Context is lost |
| `[required] quantized averagePool2d` | Failed to create session |
| `[required] quantized softmax` | Context is lost |

### 12. `relu` (1 failing / 18 total)

| Subtest | Error |
|---------|-------|
| `[optional] relu int64 4D tensor` | Failed to create session |

### 13. `subgraph` (4 failing / 49 total)

| Subtest | Error |
|---------|-------|
| `[required] conv2d default + hardSigmoid` | ULP distance 7712460 (expected ≤20) |
| `[required] conv2d default + softplus` | ULP distance 1640755 (expected ≤36) |
| `[required] conv2d default + softsign` | ULP distance 11090707 (expected ≤21) |
| `[required] batchNormalization options.axis=0 + gelu` | ULP distance 38 (expected ≤24) |

### 14. `triangular` (2 failing / 33 total)

| Subtest | Error |
|---------|-------|
| `[required] triangular float32 5D tensor default options` | Failed to create session |
| `[required] triangular float16 5D tensor default options` | Failed to create session |

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DML Execution Provider WebNN Conformance Test Failures on Qualcomm #27261

Summary of Failures by Category

Detailed Failing Tests

1. `batch_normalization` (2 failing / 25 total)

2. `batch_normalization_constant` (1 failing / 3 total)

3. `dequantizeLinear` (12 failing / 33 total)

4. `gatherElements` (2 failing / 12 total)

5. `gatherND` (2 failing / 17 total)

6. `graph_devices` (1 failing / 1 total)

7. `layer_normalization` (1 failing / 26 total)

8. `matmul` (6 failing / 23 total)

9. `prelu` (1 failing / 28 total)

10. `quantizeLinear` (6 failing / 29 total)

11. `qdq_subgraph` (9 failing / 42 total)

12. `relu` (1 failing / 18 total)

13. `subgraph` (4 failing / 49 total)

14. `triangular` (2 failing / 33 total)

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Category	Issue Type	Affected Tests
Data Type Support	`dequantizeLinear` doesn't support `int32`	dequantizeLinear, qdq_subgraph
Data Type Support	`gatherElements`/`gatherND` don't support `uint32` indices	gatherElements, gatherND
Data Type Support	`int64` not supported for activation ops	relu, prelu
Session Creation	5D tensor broadcasting failures	matmul, triangular
Session Creation	int4/uint4 odd-size tensor failures	dequantizeLinear, quantizeLinear
Session Creation	`batchNormalization` axis=0 for 1D tensors	batch_normalization
Precision/ULP	Float16/Float32 precision exceeds tolerance	batch_normalization_constant, dequantizeLinear, layer_normalization, quantizeLinear, subgraph
Context Lost	`Context is lost` errors	quantizeLinear, qdq_subgraph
Fusion Issues	Conv2d + activation fusion incorrect results	subgraph
API Gap	`graph.devices` is empty	graph_devices

Subtest	Error
`[optional] batchNormalization float32 1D tensor options.axis=0`	Failed to create session
`[optional] batchNormalization float16 1D tensor options.axis=0`	Failed to create session

Subtest	Error
`[required] dequantizeLinear int8 4D tensor broadcasting float32 4D scale and int8 4D zeroPoint`	ULP distance 33982206 (expected ≤1)
`[optional] dequantizeLinear uint4 1D tensor of odd size with float32 1D scale`	Failed to create session
`[optional] dequantizeLinear uint4 4D tensor with broadcasting float32 4D scale and uint4 4D zeroPoint`	Failed to create session
`[optional] dequantizeLinear int4 1D tensor of odd size with float32 1D scale`	Failed to create session
`[optional] dequantizeLinear int32 1D tensor with float32 1D scale`	Unsupported data type int32
`[required] dequantizeLinear with float32 3D scale as an intermediate node`	Unsupported data type int32
`[optional] dequantizeLinear int8 4D tensor broadcasting float16 4D scale and int8 4D zeroPoint`	ULP distance 4149 (expected ≤1)
`[optional] dequantizeLinear uint4 1D tensor of odd size with float16 1D scale`	Failed to create session
`[optional] dequantizeLinear uint4 4D tensor with broadcasting float16 4D scale and uint4 4D zeroPoint`	Failed to create session
`[optional] dequantizeLinear int4 1D tensor of odd size with float16 1D scale`	Failed to create session
`[optional] dequantizeLinear int32 1D tensor with float16 1D scale`	Unsupported data type int32
`[optional] dequantizeLinear with float16 3D scale as an intermediate node`	Unsupported data type int32

Subtest	Error
`[optional] gatherElements float32 2D input and uint32 indices options.axis=1`	Unsupported data type uint32 for indices
`[optional] gatherElements float16 2D input and uint32 indices options.axis=1`	Unsupported data type uint32 for indices

Subtest	Error
`[optional] gatherND float32 4D input and 1D uint32 indices`	Unsupported data type uint32 for indices
`[optional] gatherND float16 4D input and 1D uint32 indices`	Unsupported data type uint32 for indices

Subtest	Error
`[required] matmul float32 5D and 5D tensors, broadcast the two leftmost dimensions of inputB`	Failed to create session
`[required] matmul float32 5D and 5D tensors, broadcast the leftmost dimensions of inputB`	Failed to create session
`[required] matmul float32 5D and 2D tensors`	Failed to create session
`[required] matmul float16 5D and 5D tensors, broadcast the two leftmost dimensions of inputB`	Failed to create session
`[required] matmul float16 5D and 5D tensors, broadcast the leftmost dimensions of inputB`	Failed to create session
`[required] matmul float16 5D and 2D tensors`	Failed to create session

Subtest	Error
`[required] quantizeLinear float32 4D tensor broadcasting scale and zeroPoint`	ULP distance 16 (expected ≤1)
`[optional] quantizeLinear float32 tensor with int4 zeroPoint which has even size`	Context is lost
`[optional] quantizeLinear float32 2D tensor with int4 zeroPoint which has even size`	Failed to create session
`[optional] quantizeLinear float32 tensor with uint4 zeroPoint which has odd size`	Failed to create session
`[optional] quantizeLinear float16 4D tensor broadcasting scale and zeroPoint`	ULP distance 16 (expected ≤1)
`[optional] quantizeLinear float16 2D tensor with int4 zeroPoint which has even size`	Failed to create session
`[optional] quantizeLinear float16 tensor with uint4 zeroPoint which has odd size`	Failed to create session

Subtest	Error
`[optional] quantized conv2d`	Unsupported data type int32 for dequantizeLinear
`[optional] dequantizeLinear -> conv2d -> clamp -> quantizeLinear`	Unsupported data type int32 for dequantizeLinear
`[optional] dequantizeLinear -> conv2d -> relu -> quantizeLinear`	Unsupported data type int32 for dequantizeLinear
`[optional] quantized convTranspose2d`	Unsupported data type int32 for dequantizeLinear
`[optional] quantized conv2d with padding`	Unsupported data type int32 for dequantizeLinear
`[optional] quantized gemm with bias`	Unsupported data type int32 for dequantizeLinear
`[required] quantized leaky relu`	Context is lost
`[required] quantized averagePool2d`	Failed to create session
`[required] quantized softmax`	Context is lost

Subtest	Error
`[required] conv2d default + hardSigmoid`	ULP distance 7712460 (expected ≤20)
`[required] conv2d default + softplus`	ULP distance 1640755 (expected ≤36)
`[required] conv2d default + softsign`	ULP distance 11090707 (expected ≤21)
`[required] batchNormalization options.axis=0 + gelu`	ULP distance 38 (expected ≤24)

Subtest	Error
`[required] triangular float32 5D tensor default options`	Failed to create session
`[required] triangular float16 5D tensor default options`	Failed to create session

DML Execution Provider WebNN Conformance Test Failures on Qualcomm #27261

Description

Summary of Failures by Category

Detailed Failing Tests

1. batch_normalization (2 failing / 25 total)

2. batch_normalization_constant (1 failing / 3 total)

3. dequantizeLinear (12 failing / 33 total)

4. gatherElements (2 failing / 12 total)

5. gatherND (2 failing / 17 total)

6. graph_devices (1 failing / 1 total)

7. layer_normalization (1 failing / 26 total)

8. matmul (6 failing / 23 total)

9. prelu (1 failing / 28 total)

10. quantizeLinear (6 failing / 29 total)

11. qdq_subgraph (9 failing / 42 total)

12. relu (1 failing / 18 total)

13. subgraph (4 failing / 49 total)

14. triangular (2 failing / 33 total)

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

1. `batch_normalization` (2 failing / 25 total)

2. `batch_normalization_constant` (1 failing / 3 total)

3. `dequantizeLinear` (12 failing / 33 total)

4. `gatherElements` (2 failing / 12 total)

5. `gatherND` (2 failing / 17 total)

6. `graph_devices` (1 failing / 1 total)

7. `layer_normalization` (1 failing / 26 total)

8. `matmul` (6 failing / 23 total)

9. `prelu` (1 failing / 28 total)

10. `quantizeLinear` (6 failing / 29 total)

11. `qdq_subgraph` (9 failing / 42 total)

12. `relu` (1 failing / 18 total)

13. `subgraph` (4 failing / 49 total)

14. `triangular` (2 failing / 33 total)