[Bug]: NPU compile_model fails with "to_shape was called on a dynamic shape" for Qwen3-0.6B INT4 (Lunar Lake)

[blairducrayoppat]

OpenVINO Version

2026.0.0-20965-c6d6a13a886-releases/2026/0

Operating System

Windows System

Device used for inference

NPU

Framework

None

Model used

Qwen/Qwen3-0.6B

Issue description

ov.Core().compile_model(model, "NPU") fails with a RuntimeError when compiling
a Qwen3-0.6B INT4 OpenVINO IR model on NPU (Intel NPU 4000, Lunar Lake). The VPUX
compiler encounters a Gather node with dynamic input dimensions (upper bounds set to
INT64_MAX) and raises a "non broadcastable dimensions" diagnostic, followed by a
to_shape was called on a dynamic shape exception.

The identical model IR compiles and runs correctly on both GPU and CPU.

Analysis

The model IR exported by optimum-intel uses dynamic dimensions for batch and sequence
length. The upper bounds for these dimensions contain INT64_MAX (9223372036854775807),
which indicates they were not resolved to concrete values during export.

The VPUX compiler encounters these unresolved dynamic dimensions at multiple nodes:

• Convert_1 (embedding input)
• Gather (embedding lookup) — first fatal diagnostic
• Power, ReduceMean, Multiply (layer norm computations)

The Gather node triggers the critical error: the VPUX compiler attempts to broadcast
dimensions 9223372036854775807 and -9223372036854775808 (signed overflow of INT64_MAX),
which fails. The compiler then calls to_shape() on the resulting dynamic PartialShape,
which is not supported and raises the exception.

Key source locations from the exception chain:

• src\core\src\partial_shape.cpp:266 — to_shape() called on dynamic shape
• src\plugins\intel_npu\src\compiler_adapter\src\ze_graph_ext_wrappers.cpp:405 — L0 graph creation failure
• src\plugins\intel_npu\src\plugin\src\plugin.cpp:879 — NPU plugin propagation

Cross-Device Comparison

Device	Result
GPU (compile_model(model, "GPU"))	PASS — compiles and runs correctly
CPU (compile_model(model, "CPU"))	PASS — compiles and runs correctly
NPU (compile_model(model, "NPU"))	FAIL — RuntimeError: to_shape was called on a dynamic shape

The GPU and CPU plugins handle the dynamic dimensions correctly. The NPU plugin does not.

Environment

Component	Version
OpenVINO	2026.0.0-20965-c6d6a13a886-releases/2026/0
OpenVINO GenAI	2026.0.0.0-2820-dab5b993a38
optimum-intel	1.27.0
nncf	3.0.0
transformers	4.51.3
OS	Windows 11 Build 26200
Hardware	Core Ultra 7 258V (Lunar Lake), Arc 140V (Xe2), NPU 4000
NPU driver	32.0.100.4514
GPU driver	32.0.101.6987

Model Details

Property	Value
Model	Qwen/Qwen3-0.6B
Format	OpenVINO IR (exported via optimum-intel 1.27.0)
Quantization	INT4, per-group (group_size=128), asymmetric
Model size on disk	367.26 MB
Inputs	4 (dynamic batch + sequence dimensions)
Outputs	1
SHA256 (openvino_model.bin)	9d25652b603f65c5a507ef9c4d35c285bbf94e116b707aa20973cff57c2226fd
SHA256 (openvino_model.xml)	c568dab243f546ff9f259fdbd1de25091a001949fee197faa1cdd3e784ba7895

Related Issues

Related issue: This was discovered during investigation of #34450 (a separate
as_convolution LLVM ABORT affecting the same model via the heterogeneous LLMPipeline
path). The two bugs are distinct — this one occurs earlier in the VPUX compiler pipeline
and surfaces as a catchable RuntimeError, whereas #34450 terminates the process with
an uncatchable SIGABRT.

• [Bug]: Compile Model: Upper bounds were not specified, got the default value - '9223372036854775807' #32466 — Same INT64_MAX upper bounds error (Upper bounds were not specified, got the default value - '9223372036854775807') on NPU with SenseVoice (ONNX). Still open, assigned to @dmatveev. Different model and export path, same VPUX compiler failure.
• [Build]: Dynamic Input Issue on NPU with GNN Inference  #26375 — Same to_shape was called on a dynamic shape error on NPU with GNN model (dynamic batch). Closed as stale. @YuChern-Intel confirmed "only static shapes are supported on NPU."
• Examples of notebooks/pixart cannot convert pixart model due to to_shape was called on a dynamic shape. #26357 — Same to_shape error on NPU with PixArt model (dynamic input dimensions). Closed as stale. Workaround suggested: reshape to static inputs.
• [Bug]: When i use Ultra9 NPU to run models, it report: get_shape was called on a descriptor::Tensor with dynamic shape #24619 — get_shape was called on a descriptor::Tensor with dynamic shape variant on NPU (HuggingFace classification model). Closed. @avitial confirmed "no dynamism support on NPU in the driver."
• [NPU][VPUX] LLVM ABORT in as_convolution pass — degenerate 0-channel shape for Qwen3-0.6B INT4 grouped quantization on Lunar Lake NPU 4000  #34450 — Separate crash (LLVM ABORT in as_convolution pass) affecting the same Qwen3-0.6B model via the heterogeneous LLMPipeline path. Different failure mode: [NPU][VPUX] LLVM ABORT in as_convolution pass — degenerate 0-channel shape for Qwen3-0.6B INT4 grouped quantization on Lunar Lake NPU 4000  #34450 is an uncatchable SIGABRT deeper in the VPUX compiler; this issue is an earlier, catchable RuntimeError on the direct compile_model("NPU") path.

Step-by-step reproduction

1. Export the model (one-time):

optimum-cli export openvino \
  --model Qwen/Qwen3-0.6B \
  --weight-format int4 \
  --group-size 128 \
  --ratio 1.0 \
  <output_dir>

2. Attempt NPU compilation:

import openvino as ov

core = ov.Core()
model = core.read_model("<output_dir>/openvino_model.xml")
print(f"Model inputs: {len(model.inputs)}, outputs: {len(model.outputs)}")
compiled = core.compile_model(model, "NPU")  # fails here

Expected Behavior

The model should compile for NPU, consistent with GPU and CPU behavior.

Actual Behavior

The VPUX compiler emits multiple [ERROR] diagnostics about unspecified upper bounds
on nodes with dynamic dimensions, then fails with a RuntimeError.

Relevant log output

**Full stdout:**

OpenVINO: 2026.0.0-20965-c6d6a13a886-releases/2026/0
Available devices: ['CPU', 'GPU', 'NPU']
Reading model...
Model read: 4 inputs, 1 outputs
Compiling for NPU...
[ERROR] 10:33:48.885 [IE::FrontEnd::importNetwork]   Upper bounds are not specified for node '__module.model.embed_tokens/ov_ext::embedding/Convert_1' (type 'Convert'): input '0' bounds are '[9223372036854775807, 9223372036854775807]'
[ERROR] 10:33:48.885 [IE::FrontEnd::importNetwork]   Upper bounds are not specified for node '__module.model.embed_tokens/ov_ext::embedding/Gather' (type 'Gather'): input '1' bounds are '[9223372036854775807, 9223372036854775807]'
[ERROR] 10:33:48.885 [IE::FrontEnd::importNetwork]   Upper bounds are not specified for node '__module.model.layers.0.input_layernorm/aten::pow/Power' (type 'Power'): input '0' bounds are '[9223372036854775807, 9223372036854775807, 1024]'
[ERROR] 10:33:48.885 [IE::FrontEnd::importNetwork]   Upper bounds are not specified for node '__module.model.layers.0.input_layernorm/aten::mean/ReduceMean' (type 'ReduceMean'): input '0' bounds are '[9223372036854775807, 9223372036854775807, 1024]'
[ERROR] 10:33:48.885 [vpux-compiler] Got Diagnostic at loc(fused<{name = "__module.model.embed_tokens/ov_ext::embedding/Gather", type = "Gather"}>["__module.model.embed_tokens/ov_ext::embedding/Gather"]) : Got non broadcastable dimensions pair : '9223372036854775807' and -9223372036854775808'
[ERROR] 10:33:48.886 [IE::FrontEnd::importNetwork]   Upper bounds are not specified for node '__module.model.layers.0.input_layernorm/aten::mul/Multiply' (type 'Multiply'): input '0' bounds are '[9223372036854775807, 9223372036854775807, 1024]'
Python exception: Exception from src\inference\src\cpp\core.cpp:113:
Exception from src\inference\src\dev\plugin.cpp:53:
Exception from src\plugins\intel_npu\src\plugin\src\plugin.cpp:879:
Exception from src\plugins\intel_npu\src\compiler_adapter\src\ze_graph_ext_wrappers.cpp:405:
L0 pfnCreate2 result: ZE_RESULT_ERROR_INVALID_ARGUMENT, code 0x78000004 - generic error code for invalid arguments . [NPU_VCL] Compiler returned msg:
Exception from src\core\src\partial_shape.cpp:266:
to_shape was called on a dynamic shape.


**Full stderr:**

loc(fused<{name = "__module.model.embed_tokens/ov_ext::embedding/Gather", type = "Gather"}>["__module.model.embed_tokens/ov_ext::embedding/Gather"]): error: Got non broadcastable dimensions pair : '9223372036854775807' and -9223372036854775808'
Traceback (most recent call last):
  File "npu_compile_attempt.py", line 21, in <module>
    compiled = core.compile_model(model, "NPU")
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "openvino/_ov_api.py", line 646, in compile_model
    super().compile_model(model, device_name, {} if config is None else config),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Exception from src\inference\src\cpp\core.cpp:113:
Exception from src\inference\src\dev\plugin.cpp:53:
Exception from src\plugins\intel_npu\src\plugin\src\plugin.cpp:879:
Exception from src\plugins\intel_npu\src\compiler_adapter\src\ze_graph_ext_wrappers.cpp:405:
L0 pfnCreate2 result: ZE_RESULT_ERROR_INVALID_ARGUMENT, code 0x78000004 - generic error code for invalid arguments . [NPU_VCL] Compiler returned msg:
Exception from src\core\src\partial_shape.cpp:266:
to_shape was called on a dynamic shape.

Issue submission checklist

• I'm reporting an issue. It's not a question.
• I checked the problem with the documentation, FAQ, open issues, Stack Overflow, etc., and have not found a solution.
• There is reproducer code and related data files such as images, videos, models, etc.

[blairducrayoppat]

Root Cause Analysis and Proposed Fix

I've completed a source code analysis of the NPU plugin compile path to trace this error to its origin and identify a fix. Full details below.

Root Cause

The NPU plugin's compile_model() in plugin.cpp has no pre-validation for unbounded dynamic shapes. When a model has dimensions with upper bound = INT64_MAX (common in LLM exports via optimum-cli), the serialized IR passes straight through to the VPUX compiler via the L0 API (pfnCreate2). The compiler's broadcast/shape analysis encounters INT64_MAX values, causing signed-integer overflow and cascading failures:

User code: core.compile_model(model, "NPU")
  → plugin.cpp :: Plugin::compile_model()
    → CompilerAdapterFactory → DriverCompilerAdapter::compile()
      → ze_graph_ext_wrappers.cpp :: getGraphDescriptor()
        → L0 API: pfnCreate2 (graph creation)
          → VPUX compiler (driver-side):
            1. Encounters Gather node with input dim upper_bound = INT64_MAX
            2. Broadcast analysis: 9223372036854775807 vs -9223372036854775808 (signed overflow)
            3. "Got non broadcastable dimensions pair..."
            4. "Upper bounds are not specified for node /Gather_221"
            5. partial_shape.cpp:266 → "to_shape was called on a dynamic shape"
          → Returns ZE_RESULT_ERROR_INVALID_ARGUMENT (0x78000004)

Source Files Traced

File	Relevance
src/plugins/intel_npu/src/plugin/src/plugin.cpp	Proposed fix location — compile_model() entry point, no dynamic shape validation
src/plugins/intel_npu/src/plugin/src/transformations.cpp	Only batch-related dynamic shape handling (hasOtherDynamicDims, handlePluginBatching) — no general validation
src/plugins/intel_npu/src/al/include/intel_npu/npu_private_properties.hpp	DYNAMIC_SHAPE_TO_STATIC definition (private, default false)
src/plugins/intel_npu/src/al/include/intel_npu/config/options.hpp	Option registration: isPublic=false, compilerSupportVersion=0.0, OptionMode::CompileTime
src/plugins/intel_npu/src/compiler_adapter/src/driver_compiler_adapter.cpp	Error path — serializes IR + flags → L0 pfnCreate2, no dynamic shape validation
src/plugins/intel_npu/src/compiler_adapter/src/plugin_compiler_adapter.cpp	VCL compiler path — also no dynamic shape validation
src/plugins/intel_npu/src/compiler_adapter/src/compiler_impl.cpp	VCL compile implementation — also no validation
src/core/src/partial_shape.cpp	Leaf error — to_shape() throws when is_dynamic()

Key Finding: DYNAMIC_SHAPE_TO_STATIC Won't Help Here

The internal option NPU_DYNAMIC_SHAPE_TO_STATIC (from npu_private_properties.hpp) applies upper bounds as static shape dimensions when enabled. However, for models with INT64_MAX upper bounds, this would attempt to create tensors with dimension 9,223,372,036,854,775,807 — which is meaningless. This option only works when upper bounds are reasonable (e.g., [1, 18, 3]).

Proposed Fix: Early Guard in plugin.cpp

Add ~25 lines in Plugin::compile_model() — an early guard that detects unbounded dynamic shapes (where dim.get_interval().has_upper_bound() returns false) and throws a clear, actionable error message.

Insertion point: After batch handling (which may resolve batch-axis dynamics) but before compiler->compile(), immediately before the OV_ITT_TASK_NEXT(PLUGIN_COMPILE_MODEL, "compile") line.

    // Early rejection of models with unbounded dynamic shapes.
    // NPU does not support models with dimensions whose upper bound is
    // INT64_MAX (i.e., fully unbounded). Attempting to compile such models
    // produces confusing internal errors in the VPUX compiler (signed
    // overflow in broadcast analysis, "to_shape was called on a dynamic
    // shape", etc.). Detect this early and provide an actionable message.
    //
    // Bounded dynamic shapes (finite upper bound) are allowed through —
    // they may work with DYNAMIC_SHAPE_TO_STATIC or future compiler support.
    {
        const auto& modelToCheck = successfullyDebatched ? batchedModel : model;
        if (modelToCheck->is_dynamic()) {
            for (const auto& param : modelToCheck->get_parameters()) {
                const auto& pshape = param->get_partial_shape();
                if (pshape.is_dynamic()) {
                    for (size_t i = 0; i < pshape.size(); ++i) {
                        const auto& dim = pshape[i];
                        if (dim.is_dynamic() && !dim.get_interval().has_upper_bound()) {
                            OPENVINO_THROW(
                                "NPU does not support models with unbounded dynamic dimensions. ",
                                "Parameter '", param->get_friendly_name(),
                                "' has dimension [", i, "] with no finite upper bound ",
                                "(upper bound is INT64_MAX). ",
                                "Please reshape the model to use static shapes before compiling ",
                                "for the NPU device:\n",
                                "    model.reshape({<static_shape>})\n",
                                "See: https://docs.openvino.ai/2025/openvino-workflow/"
                                "running-inference/changing-input-shape.html");
                        }
                    }
                }
            }
            for (const auto& result : modelToCheck->get_results()) {
                const auto& pshape = result->get_input_partial_shape(0);
                if (pshape.is_dynamic()) {
                    for (size_t i = 0; i < pshape.size(); ++i) {
                        const auto& dim = pshape[i];
                        if (dim.is_dynamic() && !dim.get_interval().has_upper_bound()) {
                            OPENVINO_THROW(
                                "NPU does not support models with unbounded dynamic dimensions. ",
                                "Result '", result->get_friendly_name(),
                                "' has output dimension [", i, "] with no finite upper bound ",
                                "(upper bound is INT64_MAX). ",
                                "Please reshape the model to use static shapes before compiling ",
                                "for the NPU device:\n",
                                "    model.reshape({<static_shape>})\n",
                                "See: https://docs.openvino.ai/2025/openvino-workflow/"
                                "running-inference/changing-input-shape.html");
                        }
                    }
                }
            }
        }
    }

Why This Insertion Point

1. After batch handling: handlePluginBatching() may resolve batch-axis dynamic dims — checking after avoids false positives on batch-dynamic models the plugin handles natively.
2. Before compilation: Catches the problem before the serialized IR reaches the VPUX compiler.
3. Checks the correct model: Uses successfullyDebatched ? batchedModel : model — the actual model that will be compiled.
4. Only rejects unbounded shapes: Bounded dynamic shapes (has_upper_bound() == true) pass through and may work with DYNAMIC_SHAPE_TO_STATIC=YES or future compiler enhancements.

All APIs used (model->is_dynamic(), get_parameters(), get_partial_shape(), dim.get_interval().has_upper_bound(), OPENVINO_THROW()) are existing public OpenVINO C++ APIs already used elsewhere in plugin.cpp.

Error Message Improvement

Before (current):

RuntimeError: to_shape was called on a dynamic shape.

After (with fix):

RuntimeError: NPU does not support models with unbounded dynamic dimensions.
Parameter 'input_ids' has dimension [1] with no finite upper bound (upper bound
is INT64_MAX). Please reshape the model to use static shapes before compiling
for the NPU device:
    model.reshape({<static_shape>})
See: https://docs.openvino.ai/2025/openvino-workflow/running-inference/changing-input-shape.html

Alternatives Considered

Option	Description	Assessment
A: Early guard in plugin.cpp (above)	Detect unbounded dynamic shapes pre-compilation, throw actionable error	Recommended — zero regression risk, single file, ~25 LOC
B: Enhance DYNAMIC_SHAPE_TO_STATIC	Reject INT64_MAX bounds or auto-reshape when option is enabled	Lower value — option is private/internal, end users can't set it
C: Fix VPUX compiler broadcast logic	Fix signed-integer overflow in driver-side compiler	Ideal but not feasible for external contributors — runs inside Intel's driver

Regression Risk

Zero. This fix only rejects models that already fail (with a confusing error). It changes no existing behavior for:

• Models with static shapes (unaffected)
• Models with bounded dynamic shapes (pass through)
• The DYNAMIC_SHAPE_TO_STATIC path (unaffected — only applies when option is enabled)

Cross-Referenced Issues

This fix would also address:

• [Bug]: Compile Model: Upper bounds were not specified, got the default value - '9223372036854775807' #32466 — SenseVoice NPU compile fails with same INT64_MAX error (same root cause)
• [Bug]: When i use Ultra9 NPU to run models, it report: get_shape was called on a descriptor::Tensor with dynamic shape #24619 — to_shape on dynamic shape (NPU) (same error signature)
• Likely [Build]: Dynamic Input Issue on NPU with GNN Inference  #26375 and Examples of notebooks/pixart cannot convert pixart model due to to_shape was called on a dynamic shape. #26357 — NPU dynamic shape compilation errors (similar pattern)

Next Steps

I have this implementation ready. Would you like me to submit this as a PR, or does the NPU team prefer to handle this internally (perhaps with a complementary fix in the VPUX compiler)?

[YuChern-Intel]

Since you have the implementation ready, you are free to submit the PR.