[Performance] Performance regression in TreeEnsembleRegressor operator between v1.22.0 and v1.23.0

[junghyunpark2001]

Describe the issue

Description

We observed a severe performance regression in the TreeEnsembleRegressor operator between ONNXRuntime v1.22.0 and v1.23.0 with +104.8% kernel slowdown (more than 2x slower).

Affected Operator

TreeEnsembleRegressor

• Opset Version: 3 (ai.onnx.ml domain)
• Configuration: Large batch optimization test
• Regression: +104.8% kernel slowdown

Test Case Details

Test Case: treeensemble_coverage_tree_large_batch_optimization

Inputs:

• X tensor:
  • Data type: float32
  • Shape: [1000, 1] (1000 samples, 1 feature)

Attributes:

• aggregate_function: SUM
• n_targets: 1

Output:

• Data type: float32
• Shape: [1000, 1]

Performance:

• v1.22.0: 0.0053 ms (kernel time)
• v1.23.0: 0.0108 ms (kernel time)
• Kernel regression: +104.8% slowdown
• Confirmation: 5/10 validation runs confirmed

Root Cause: Identified Commit

Commit: 223c4a08fad0dd9dee5aae53bf142c386ddf853e

To reproduce

python script_profiling.py treeensemble_coverage_tree_large_batch_optimization 1.22.0 1.23.0

Archive.zip

Urgency

No response

Platform

Linux

OS Version

Ubuntu 24.04.3 LTS

ONNX Runtime Installation

Built from Source

ONNX Runtime Version or Commit ID

1.23

ONNX Runtime API

Python

Architecture

X64

Execution Provider

Default CPU

Execution Provider Library Version

No response

Model File

No response

Is this a quantized model?

Yes