[onert] Performance Regression: Latency Increase & Memory Reduction with MobileNetV2

[ragmani]

Description

We previously saw the following results for MobileNetV2 on CPU:
Prepare : 10.193 ms
Avg I/O : 0.081 ms
Avg Run : 10.520 ms
RSS EXECUTE : 72 160 KB

Today’s results show:
Prepare : 10.606 ms
Avg I/O : 0.084 ms
Avg Run : 21.962 ms
RSS EXECUTE : 37 484 KB

Measurement Reference

• Previous measurement: [onert/python] Support inference benchmark #15192 (comment)
• Current measurement at commit b2f193a

Steps to Reproduce

1. Run

python3 runtime/onert/sample/minimal-python/inference_benchmark.py mobilenetv2 --backends cpu --input-shape 1,224,224,3 --repeat 100

2. Compare the “Avg Run” and “EXECUTE RSS” values to previous runs

Expected Behavior

• Avg Run should remain close to ~10.5 ms
• RSS EXECUTE should remain close to ~72 MB

Actual Behavior

• Avg Run has nearly doubled to ~22 ms
• RSS EXECUTE has dropped by roughly 50%

Impact

This regression could indicate a change in memory allocation strategy or an unintended slowdown in the execution path. It needs investigation.

[hseok-oh]

Because of #15337 ?

[hseok-oh]

In my environment

commit 883d61c

===================================
MODEL_LOAD   takes 7.669 ms
PREPARE      takes 14.220 ms
EXECUTE      takes 18.793 ms
- MEAN     :  18.793 ms
- MAX      :  31.996 ms
- MIN      :  15.295 ms
- GEOMEAN  :  18.359 ms
===================================

commit b1b820e (parent commit of #15337)

===================================
MODEL_LOAD   takes 11.093 ms
PREPARE      takes 21.443 ms
EXECUTE      takes 9.965 ms
- MEAN     :  9.965 ms
- MAX      :  17.930 ms
- MIN      :  6.353 ms
- GEOMEAN  :  9.210 ms
===================================

[ragmani]

The regression on MobileNetV2 is caused by the revert of the DepthwiseConv2D optimization. We don’t have a fix at the moment, so we’ll leave the issue open and address it later.

[glistening]

@ragmani The regression is huge ( It takes twice times. ). It may be good to revert the revert of #15337 if there is a compelling reason. Could you please explain why you reverted #15337? As I was told offline several days, but I don't remember.

[ragmani]

@glistening You can find the reason in #15337 (comment).
The performance degradation may occur because the optimized DepthwiseConv2D kernel is not used.

[ragmani]

Enabling optimized DepthwiseConv2D kernel will be addressed in #15396.