[OV][ITT][CPU Plugin] Align ITT markers to standard convention and add async support by tovinkere · Pull Request #33312 · openvinotoolkit/openvino

tovinkere · 2025-12-18T19:09:24Z

Standardizing CPU ITT marker + support for asynchronous execution - Part 2

This PR is the second of a series of PRs to standardize the ITT markers in OpenVINO that will be enabled by default through host-side instrumentation.

The first PR addresses the enhancements required in ITT and the framework to support the creation and propagation of IDs when asynchronous execution is in play PR#33639.
This second PR will standardize ITT markers in the CPU and enhance support to include asynchronous execution.
The third PR will enable default markers for GPU plugin to allow visibility into inference pass begin/end and operator preparation and submission within each inference.
The final PR will extend the same host side markers for NPU execution, which capturing the inference span and pipeline activity.

Summary of the current PR (PR#2)

Use the same convention standardized in PR#33639
Ensures the namespace for CPU Plugin activity falls under:
- ov::phases::inference
- ov::phases::cpu::inference
- ov::op::cpu::exec
- ov::op::cpu::details

Details:
CPU support with default enabled ITT markers was limited to synchronous execution. This PR enhances the default support to include asynchronous behavior and ensures a standardized convention is followed in namespaces used.

@aobolensk Please review as this is an enhancement of your work with synchronous execution.

See CVS-179230 benchmark data for smoke tests that include low-latency models.

+ Use the same convention standardized in PR#33311 + Ensures the namespace for CPU Plugin activity falls under: ov::phases::inference ov::phases::cpu::inference ov::op::cpu::exec ov::op::cpu::details Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

+ The value of ov:op::cpu::details won't be understood until the graph model mapping between the input model and the optimized execution model requirements are finalized. Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

src/plugins/intel_cpu/src/graph.cpp

src/plugins/intel_cpu/src/infer_request.cpp

Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

maxnick · 2026-01-15T16:43:29Z

build_jenkins

@aobolensk

…d async support (openvinotoolkit#33312) Standardizing CPU ITT marker + support for asynchronous execution - Part 2 This PR is the second of a series of PRs to standardize the ITT markers in OpenVINO that will be enabled by default through host-side instrumentation. 1. The first PR addresses the enhancements required in ITT and the framework to support the creation and propagation of IDs when asynchronous execution is in play [PR#33639](openvinotoolkit#33639). 2. This ***second PR*** will standardize ITT markers in the CPU and enhance support to include asynchronous execution. 3. The third PR will enable default markers for GPU plugin to allow visibility into inference pass begin/end and operator preparation and submission within each inference. 4. The final PR will extend the same host side markers for NPU execution, which capturing the inference span and pipeline activity. Summary of the current PR (PR#2) + Use the same convention standardized in [PR#33639](openvinotoolkit#33639) + Ensures the namespace for CPU Plugin activity falls under: - ov::phases::inference - ov::phases::cpu::inference - ov::op::cpu::exec - ov::op::cpu::details Details: CPU support with default enabled ITT markers was limited to synchronous execution. This PR enhances the default support to include asynchronous behavior and ensures a standardized convention is followed in namespaces used. @aobolensk Please review as this is an enhancement of your work with synchronous execution. See [CVS-179230](https://jira.devtools.intel.com/browse/CVS-179230) benchmark data for smoke tests that include low-latency models. --------- Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

@isanghao

… submission (#33313) - Enables default ITT markers for higher level operations such as inference pass, op preparation and submission - Follows the same guidelines to standardize the conventions for namespaces: ov::phases::gpu::inference ov::op::gpu - Supports both synchronous and asynchronous operations Enabling default GPU ITT markers using standard convention - Part 3 This PR is the **third** of a series of PRs to standardize the ITT markers in OpenVINO that will be enabled by default through host-side instrumentation. 1. The first PR addresses the enhancements required in ITT and the framework to support the creation and propagation of IDs when asynchronous execution is in play [PR#33639](#33639). 2. The second PR will standardize ITT markers in the CPU and enhance support to include asynchronous execution [PR#33312](#33312). 3. This **third** PR will enable default markers for GPU plugin to allow visibility into inference pass begin/end and operator preparation and submission within each inference. Follow standardized conventions as described in 1 and 2 4. The final PR will extend the same host side markers for NPU execution, which capturing the inference span and pipeline activity. Summary of the current PR (PR#3) Use the same convention standardized in [PR#33639](#33639) Ensures the namespace for GPU Plugin activity falls under: ov::phases::gpu::inference ov::op::gpu Details: GPU support is enabled with default ITT markers that support synchronous an asynchronous execution. This PR ensures a standardized convention is followed in namespaces used. Tickets: [CVS-179230](https://jira.devtools.intel.com/browse/CVS-179230) @isanghao Please review this as you are generally aware of what was discussed --------- Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

@isanghao

… submission (#33313) - Enables default ITT markers for higher level operations such as inference pass, op preparation and submission - Follows the same guidelines to standardize the conventions for namespaces: ov::phases::gpu::inference ov::op::gpu - Supports both synchronous and asynchronous operations Enabling default GPU ITT markers using standard convention - Part 3 This PR is the **third** of a series of PRs to standardize the ITT markers in OpenVINO that will be enabled by default through host-side instrumentation. 1. The first PR addresses the enhancements required in ITT and the framework to support the creation and propagation of IDs when asynchronous execution is in play [PR#33639](#33639). 2. The second PR will standardize ITT markers in the CPU and enhance support to include asynchronous execution [PR#33312](#33312). 3. This **third** PR will enable default markers for GPU plugin to allow visibility into inference pass begin/end and operator preparation and submission within each inference. Follow standardized conventions as described in 1 and 2 4. The final PR will extend the same host side markers for NPU execution, which capturing the inference span and pipeline activity. Summary of the current PR (PR#3) Use the same convention standardized in [PR#33639](#33639) Ensures the namespace for GPU Plugin activity falls under: ov::phases::gpu::inference ov::op::gpu Details: GPU support is enabled with default ITT markers that support synchronous an asynchronous execution. This PR ensures a standardized convention is followed in namespaces used. Tickets: [CVS-179230](https://jira.devtools.intel.com/browse/CVS-179230) @isanghao Please review this as you are generally aware of what was discussed --------- Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

tovinkere requested review from a team as code owners December 18, 2025 19:09

github-actions bot added the category: CPU OpenVINO CPU plugin label Dec 18, 2025

sys-openvino-ci added the ExternalIntelPR External contributor from Intel label Dec 18, 2025

tovinkere mentioned this pull request Dec 18, 2025

[OV][ITT][GPU Plugin] Enable default ITT markers for inference and op submission #33313

Merged

aobolensk requested review from aobolensk and maxnick December 22, 2025 05:07

aobolensk assigned maxnick and aobolensk Dec 22, 2025

aobolensk reviewed Dec 23, 2025

View reviewed changes

src/plugins/intel_cpu/src/graph.cpp Outdated Show resolved Hide resolved

src/plugins/intel_cpu/src/infer_request.cpp Show resolved Hide resolved

[OV][ITT] Addressing review feedback

21bf1d5

Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

aobolensk approved these changes Jan 19, 2026

View reviewed changes

aobolensk added under_perf_check and removed under_perf_check labels Jan 19, 2026

praasz approved these changes Jan 20, 2026

View reviewed changes

maxnick added this to the 2026.0 milestone Jan 21, 2026

aobolensk added this pull request to the merge queue Jan 21, 2026

Merged via the queue into openvinotoolkit:master with commit 967e454 Jan 21, 2026
215 of 219 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[OV][ITT][CPU Plugin] Align ITT markers to standard convention and add async support #33312

[OV][ITT][CPU Plugin] Align ITT markers to standard convention and add async support #33312
aobolensk merged 3 commits intoopenvinotoolkit:masterfrom
tovinkere:itt_markers_cpu

tovinkere commented Dec 18, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

maxnick commented Jan 15, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

tovinkere commented Dec 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

maxnick commented Jan 15, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

tovinkere commented Dec 18, 2025 •

edited

Loading