Skip to content

[Test Utils] Limit tensor dimensions in Op Conformance to prevent OOM(Out of memory) - #34183

Open
Udayscode wants to merge 10 commits into
openvinotoolkit:masterfrom
Udayscode:fix/template-oom-conformance
Open

[Test Utils] Limit tensor dimensions in Op Conformance to prevent OOM(Out of memory)#34183
Udayscode wants to merge 10 commits into
openvinotoolkit:masterfrom
Udayscode:fix/template-oom-conformance

Conversation

@Udayscode

Copy link
Copy Markdown

Fixes #23556
This PR limits the maximum dimension of generated static shapes based on tensor rank to prevent out-of-memory (OOM) crashes.

The previous implementation used a fixed limit of 127 for all dimensions. For high-rank tensors (5D+), this caused memory requests to exceed 270GB, leading to std::bad_alloc errors. I have updated the logic to use adaptive limits: 127 for Rank 1-2, 32 for Rank 3-4, and 20 for Rank 5+.

Local testing on a problematic 5D Add operation showed a reduction in memory usage from crash levels to 53MB. All 5,068 conformance tests now finish successfully.

Changes

  1. Updated dimMax calculation in generate_static_shapes.cpp to be rank-dependent.
  2. Verified the fix on specific IR files that previously failed.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the op conformance runner’s static-shape generation to cap per-dimension sizes based on tensor rank, reducing the likelihood of OOM during synthetic input generation for dynamic-shape models.

Changes:

  • Made dimMax rank-dependent (127 for rank 1–2, 32 for rank 3–4, 20 for rank 5+).
  • Applied the adaptive cap consistently across generic shape generation, restricted-dim clipping, and pooling shape generation.

@t-jankowski

Copy link
Copy Markdown
Contributor

Proposed solution doesn't relate to Template Plugin code but Op Conformance test runner.
@iefode Would you take a look?

@t-jankowski t-jankowski removed their assignment Feb 26, 2026
@Udayscode Udayscode changed the title [Template Plugin] Limit tensor dimension size based on rank to prevent OOM(Out of memory) [Test Utils] Limit tensor dimensions in Op Conformance to prevent OOM(Out of memory) Feb 26, 2026
@Udayscode

Copy link
Copy Markdown
Author

@t-jankowski My apologies for the confusion in the PR title. I initially tagged it as [Template Plugin] because the OOM was specifically triggered during Template conformance runs, but the root cause is indeed in the test runner's shape generation.
@iefode please let me know if you’d like any further changes!

@t-jankowski

Copy link
Copy Markdown
Contributor

@t-jankowski My apologies for the confusion in the PR title. I initially tagged it as [Template Plugin] because the OOM was specifically triggered during Template conformance runs, but the root cause is indeed in the test runner's shape generation. @iefode please let me know if you’d like any further changes!

That's ok - no need to apologize.

@sbalandi

Copy link
Copy Markdown
Contributor

build_jenkins

@github-actions

Copy link
Copy Markdown
Contributor

This PR will be closed in a week because of 2 weeks of no activity.

@github-actions github-actions Bot added the Stale label Jun 21, 2026
@praasz

praasz commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

build_jenkins

@praasz

praasz commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

build_jenkins

@github-actions github-actions Bot removed the Stale label Jun 24, 2026
@praasz

praasz commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

build_jenkins

@praasz praasz added this to the 2026.4 milestone Jul 9, 2026
@praasz

praasz commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

build_jenkins

@praasz

praasz commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

build_jenkins

@github-actions

Copy link
Copy Markdown
Contributor

This PR will be closed in a week because of 2 weeks of no activity.

@github-actions github-actions Bot added the Stale label Jul 29, 2026
@praasz praasz added the no_stale Do not mark as stale label Jul 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

category: IE Tests OpenVINO Test: plugins and common ExternalPR External contributor no_stale Do not mark as stale Stale

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Good First Issue]: [OP CONFORMANCE][TEMPLATE] Add test in op conformance fails

6 participants