[TorchAO] [OVQuantizer] Update Torch AO Dependency and Refactor Executorch Tests by anzr299 · Pull Request #3842 · openvinotoolkit/nncf

anzr299 · 2026-01-15T15:33:22Z

Changes

Include tests for executorch, OVQuantizer
Remove OVQuantizer from NNCF
Move the tests in appropriate directory

Reason for changes

Migrate from torch.ao -> torchao

Notable differences in torch.ao and torchao implementation which affect test references:

Constant folding in torch.ao produced the new constant with the name _frozen_param{index}. In the new torchao implementation it keeps the same name as the node being replaced with.
The old deprecated XNNPACK Quantizer was being used before which is now changed to point to the new XNNPACKQuantizer implementation in https://github.com/pytorch/executorch/tree/main/backends/xnnpack/quantizer which now explicitly quantizes the conv_transpose2D operator and its weights.

Reference graphs changed due to above:

tests/executorch/data/ao_export_quantization_OpenVINOQuantizer/mobilenet_v3_small.dot
tests/executorch/data/ao_export_quantization_OpenVINOQuantizer/resnet18.dot
tests/executorch/data/ao_export_quantization_OpenVINOQuantizer/swin_v2_t.dot
tests/executorch/data/ao_export_quantization_OpenVINOQuantizer/synthetic_transformer.dot
tests/executorch/data/ao_export_quantization_OpenVINOQuantizer/unet.dot
tests/executorch/data/ao_export_quantization_OpenVINOQuantizer/vit_b_16.dot
tests/executorch/data/ao_export_quantization_OpenVINOQuantizer/yolo11n_sdpa_block.dot
tests/executorch/data/XNNPACKQuantizer/unet.dot (Due to difference Number 2)
tests/executorch/data/XNNPACKQuantizer/unet_ref_qconfig.json ((Due to difference Number 2))

Related tickets

CVS-176783

Tests

PTQ Conformance - PTQ-789 - Pass
Examples - https://github.com/openvinotoolkit/nncf/actions/runs/21204888584 - Pass
WC conformance - https://github.com/openvinotoolkit/nncf/actions/runs/21204893707 - Pass

pytest tests/executorch results:

All tests in tests/executorch pass

…into an/executorch_tests

examples/post_training_quantization/torch_fx/resnet18/requirements.txt

constraints.txt

AlexanderDokuchaev · 2026-01-20T15:23:58Z

src/nncf/experimental/torch/fx/quantization/quantize_pt2e.py

 from nncf.quantization.range_estimator import RangeEstimatorParameters


+def _is_openvino_quantizer_instance(obj) -> bool:


Looks like WA for mistakes in https://github.com/pytorch/executorch/blame/main/backends/openvino/quantizer/quantizer.py

It's really bad import tree.
Need some time...

There is a dependency and ownership issue around the quantization entry point that needs clarification.

Currently:

quantize_pt2e depends on OpenVINOQuantizer (from executorch).

quantize_model (also in executorch) depends on quantize_pt2e.

OpenVINOQuantizer and quantize_model are defined in the same file.

As a result, the effective call chain looks like this: executorch → nncf → executorch

This creates a circular dependency and makes it unclear what the intended public entry point is for OpenVINO PT2E quantization. Since everything lives in a single executorch file, it’s not obvious:

which function users are expected to call directly

which layer “owns” the quantization workflow (executorch or NNCF).

So my current suggest to dont block this PR:

Create ticket on it, add TODO here.

Refactor openvino backend files in executorch

Finalize quantize_pt2e

@daniil-lyakhov
@MaximProshin

Created a ticket - CVS-179719

src/nncf/torch/quantization/strip.py

AlexanderDokuchaev · 2026-01-20T15:49:53Z

src/nncf/experimental/torch/fx/quantization/quantize_pt2e.py

+    """
+    try:
+        from executorch.backends.openvino.quantizer.quantizer import OpenVINOQuantizer
+    except ModuleNotFoundError as err:


But _is_openvino_quantizer_instance function is useless.
In case to avoid circular import just move import inside a function.

And it's not safely check cause raise error. For optional dependencies it's should looks like

try: from executorch.backends.openvino.quantizer.quantizer import OpenVINOQuantizer return isinstance(obj, OpenVINOQuantizer) except ImportError: return False

And better use ImportError for checks like this, because for old version of executorch will raised ImportError.
ModuleNotFoundError is subclass of ImportError.

oh okay, I will change it

AlexanderDokuchaev · 2026-01-20T15:55:29Z

tests/executorch/requirements.txt

 pytest-xdist==3.5.0
 pytest-forked==1.6.0
 pytest-split==0.9.0
+ipython==8.37.0


ipython is not requared.
Use fastprogress==1.0.5 as it in constraints.txt #3815

Ah Alright, Done

AlexanderDokuchaev · 2026-01-20T16:32:03Z

tests/executorch/data/OpenVINOQuantizer/mobilenet_v3_small.dot

@@ -0,0 +1,1038 @@
+strict digraph {


Differ from old tests/torch/data/fx/OpenVINOQuantizer/mobilenet_v3_small.dot
Is it expected? What has changed?

It was a bug, it is changed ot be the same now

AlexanderDokuchaev · 2026-01-20T17:02:32Z

@anzr299 What version of executorch is required to run tests?

anzr299 · 2026-01-20T18:22:40Z

@anzr299 What version of executorch is required to run tests?

Executorch 1.0.1 for PTQ tests.
For Weights compression it will have to be installed from develop of executorch

AlexanderDokuchaev · 2026-01-21T14:29:01Z

examples/llm_compression/torch/distillation_qat_with_lora/requirements.txt

@@ -1,5 +1,6 @@
 tensorboard==2.13.0
 torch==2.9.0
+torchao==0.14.0


Why torch example require torchao?
It's not use anything from it.
torchao and executorch should not be required dependency for TORCH backend

It was importing torchao in src/nncf/torch/quantization/strip.py. Earlier this was torch.ao so it was hidden

https://github.com/openvinotoolkit/nncf/actions/runs/21142246987/job/60798968378

Perhaps I can make it lazy import inside the convert_to_torch_fakequantizer function

AlexanderDokuchaev · 2026-01-21T14:48:39Z

tests/post_training/requirements.txt

 optimum-onnx==0.1.0
 optimum==2.1.0
-scikit-learn>=1.2.2,<=1.5.0
+scikit-learn>=1.2.2,<=1.7.1


Set strict version of used package (get version from test job)

AlexanderDokuchaev · 2026-01-21T15:17:50Z

tests/post_training/pipelines/image_classification_base.py

+        preset = quantizer_kwargs.pop("preset", None)
+        model_type = quantizer_kwargs.pop("model_type", None)
+
+        # This logic is copied and inverted from OVQuantizer code


Why is it here?
Please use model_scope.py to setup compression arguments without any hidden behavior.

AlexanderDokuchaev · 2026-01-21T16:10:34Z

src/nncf/experimental/torch/fx/quantization/quantize_pt2e.py

 from nncf.quantization.range_estimator import RangeEstimatorParameters


+def _is_openvino_quantizer_instance(obj) -> bool:


There is a dependency and ownership issue around the quantization entry point that needs clarification.

Currently:

quantize_pt2e depends on OpenVINOQuantizer (from executorch).

quantize_model (also in executorch) depends on quantize_pt2e.

OpenVINOQuantizer and quantize_model are defined in the same file.

As a result, the effective call chain looks like this: executorch → nncf → executorch

This creates a circular dependency and makes it unclear what the intended public entry point is for OpenVINO PT2E quantization. Since everything lives in a single executorch file, it’s not obvious:

which function users are expected to call directly

which layer “owns” the quantization workflow (executorch or NNCF).

So my current suggest to dont block this PR:

Create ticket on it, add TODO here.

Refactor openvino backend files in executorch

Finalize quantize_pt2e

@daniil-lyakhov
@MaximProshin

…into an/executorch_tests

MaximProshin · 2026-01-23T06:07:21Z

Agreed to merge it after the CF to mitigate any risks with a new dependency. @anzr299 @AlexanderDokuchaev

daniil-lyakhov · 2026-01-26T14:33:10Z

constraints.txt

 # Pytorch
 torch==2.9.0
 torchvision==0.24.0
+torchao==0.14.0


Aamir, as we discussed it with Alexander Suslov, the main idea was to get rid of torch.ao and do not use any external dependencies (like torchao)
The task is to remove torch.ao imports and replace them with code in the nncf codebase, no external dependencies

anzr299 added 4 commits January 8, 2026 20:43

init

9dd119c

remove extra imports

b62071b

update workflow file; change location of data file for test

0467fda

Merge branch 'openvinotoolkit:develop' into an/executorch_tests

4090ae9

anzr299 requested a review from a team as a code owner January 15, 2026 15:33

anzr299 added 4 commits January 15, 2026 20:29

remove old quantizer test

716d1be

Merge branch 'an/executorch_tests' of https://github.com/anzr299/nncf …

a3664bc

…into an/executorch_tests

fix imports

602f773

modify data folder

096347b

anzr299 marked this pull request as draft January 15, 2026 16:44

anzr299 added 3 commits January 15, 2026 22:03

call executorch precommit on PR

9be7502

fix file location

414b74b

remove OVQuantizer

e55c599

github-actions bot added the API Public API-impacting changes label Jan 15, 2026

anzr299 added 3 commits January 15, 2026 22:21

conditional import of openvino quantizer

2f54bd5

replace all torch.ao instances with torchao

4c4cd31

micro import fix

77090db

github-actions bot added NNCF PT Pull requests that updates NNCF PyTorch NNCF PTQ Pull requests that updates NNCF PTQ labels Jan 15, 2026

anzr299 added 11 commits January 16, 2026 10:56

include torchao in requirements

2fdca07

update executorch requirements

11cc209

fix failing tests

cd320cc

fix executorch version

248f87e

fix workflow file

778ef3b

remove old ref files

9d1dc3c

add ref data

b55de80

add ipython

558ab32

modify versions of executorch torch torchao...

e1a6b31

use executorch from commit

25e088a

try fixed torch and executorch from commit

e6b67eb

anzr299 added 5 commits January 19, 2026 20:44

add torchao to fx example

663d787

fix conformance test

4ea7ca4

update refs; correct test params

82d2675

fix conformance

964435c

fix nasty bug

7ced6db

anzr299 marked this pull request as ready for review January 20, 2026 14:41

AlexanderDokuchaev requested changes Jan 20, 2026

View reviewed changes

anzr299 added 9 commits January 20, 2026 22:30

review changes

4fbe6ed

update ref correctly

7ab039c

update torchao version

1e7e2a1

fix smoothquant initialization

59d0623

revert torchao version

db81c40

review change

dcab6dd

revert torchao version to 0.14.0

0b1dbd6

old ref order is kept

987da71

Merge branch 'openvinotoolkit:develop' into an/executorch_tests

9fce258

AlexanderDokuchaev requested changes Jan 21, 2026

View reviewed changes

anzr299 added 6 commits January 21, 2026 20:30

set strict scikit learn requirement in post training test reqreuiementes

9b556dc

Merge branch 'an/executorch_tests' of https://github.com/anzr299/nncf …

fc76f1f

…into an/executorch_tests

review changes

2e825b0

Add todo for refactoring and ticket

fdc99eb

fix kwargs

e8ad45a

fix lint issue

fa98dd1

anzr299 removed the Code Freeze label Jan 22, 2026

add torchao in install test helpers

23a2b7e

resolve circular import issue

711fd78

daniil-lyakhov reviewed Jan 26, 2026

View reviewed changes

		from nncf.quantization.range_estimator import RangeEstimatorParameters


		def _is_openvino_quantizer_instance(obj) -> bool:

Conversation

anzr299 commented Jan 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Reason for changes

Related tickets

Tests

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AlexanderDokuchaev commented Jan 20, 2026

Uh oh!

anzr299 commented Jan 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

anzr299 Jan 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MaximProshin commented Jan 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Comments

anzr299 commented Jan 15, 2026 •

edited

Loading

anzr299 commented Jan 20, 2026 •

edited

Loading

anzr299 Jan 21, 2026 •

edited

Loading

MaximProshin commented Jan 23, 2026 •

edited

Loading