[OV][ITT] Enhance ITT MACROS to accept metadata for ID propagation - Redux by tovinkere · Pull Request #33639 · openvinotoolkit/openvino

tovinkere · 2026-01-16T07:39:04Z

#Feature enhancement - Part 1

This PR is the first of a series of PRs to standardize the ITT markers in OpenVINO that will be enabled by default through host-side instrumentation.

This first PR addresses the enhancements required in ITT and the framework to support the creation and propagation of IDs when asynchronous execution is in play.
The second PR will standardize ITT markers in the CPU and enhance support to include asynchronous execution.
The third PR will enable default markers for GPU plugin to allow visibility into inference pass begin/end and operator preparation and submission within each inference.
The final PR will extend the same host side markers for NPU execution, which capturing the inference span and pipeline activity.

##Summary of the current PR (PR#1)

Enhances base macros to also accept metadata as a pair that will be associated with a region or a task
Updated core markers, such as Read Model, Compile Model to follow a standard convention and belong to the namespace ov::phases, which is also the ITT domain
Added early exit to domain and string handle creation if a collector is not initialized and actively receiving the data

##Details:
For some plugins that use pipeline execution during asynchronous evaluation, the inference is initiated on one thread, the pipeline execution begins on another thread and is completed on a different thread. In order to get the full inference time, including host side timestamps, the stages of execution that belong to the same inference pass MUST:

Share the same inference ID, which is provided by a thread-safe global counter
This ID must be associated with the region or task of interest
And propagated by the ITT subsystem. ScopedRegion() and ScopedTask() have been enhanced to support this.

Tickets:

CVS-179230

@praasz This is resubmitted PR after the corruption of PR #33311

- This first PR addresses the enhancements required in ITT and the framework to support the creation and propagation of IDs when asynchronous execution is in play - Enhances base macros to also accept metadata as a pair that will be associated with a region or a task - Updated core markers, such as Read Model, Compile Model to follow a standard convention and belong to the namespace ov::phases, which is also the ITT domain - Added early exit to domain and string handle creation if a collector is not initialized and actively receiving the data Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

praasz · 2026-01-16T07:51:35Z

build_jenkins

- Moved the MACROS for ID initialization, ID update and global ID to the implementation file. - The class IAsyncInferRequest will always have the counter variable as a member. The update this member depends on whether ENABLE_PROFILING_ITT=BASE or FULL is set and should not happen when it is set to OFF Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

praasz · 2026-01-19T06:20:28Z

build_jenkins

aobolensk · 2026-01-19T09:02:19Z

src/inference/src/dev/iasync_infer_request.cpp

+            OV_ITT_SCOPED_REGION_BASE(ov::itt::domains::Inference,
+                                      "Inference::pipeline",
+                                      "InferenceID",
+                                      m_infer_id);  // DO NOT MODIFY!


Please, remove all markers that is should not be modified as on plugin requests

praasz · 2026-01-19T09:18:16Z

build_jenkins

src/inference/src/dev/iasync_infer_request.cpp

aobolensk · 2026-01-19T09:47:11Z

src/inference/src/itt.hpp

 OV_ITT_DOMAIN(OV, "ov");
 OV_ITT_DOMAIN(ReadTime, "ov::ReadTime");
 OV_ITT_DOMAIN(LoadTime, "ov::LoadTime");
+// Domain used for marking phases in the runtime


Comment is not descriptive. "Phases" of which process are being marked here?

aobolensk · 2026-01-19T09:49:58Z

src/common/itt/src/itt.cpp

+        // default to the parent task's ID
+        __itt_metadata_add(domain,
+                           __itt_null,
+                           __itt_string_handle_create(key ? key : "unknown"),


Is key == nullptr possible on this line? We have checked if key is null or not on line 74

aobolensk · 2026-01-19T09:50:26Z

src/common/itt/src/itt.cpp

+    // Associate the <key-value> pair with the region
+    __itt_metadata_add(domain,
+                       region_id,
+                       __itt_string_handle_create(key ? key : "unknown"),


Same question about key being null is applicable here

praasz · 2026-01-19T15:57:35Z

build_jenkins

@aobolensk

…d async support (#33312) Standardizing CPU ITT marker + support for asynchronous execution - Part 2 This PR is the second of a series of PRs to standardize the ITT markers in OpenVINO that will be enabled by default through host-side instrumentation. 1. The first PR addresses the enhancements required in ITT and the framework to support the creation and propagation of IDs when asynchronous execution is in play [PR#33639](#33639). 2. This ***second PR*** will standardize ITT markers in the CPU and enhance support to include asynchronous execution. 3. The third PR will enable default markers for GPU plugin to allow visibility into inference pass begin/end and operator preparation and submission within each inference. 4. The final PR will extend the same host side markers for NPU execution, which capturing the inference span and pipeline activity. Summary of the current PR (PR#2) + Use the same convention standardized in [PR#33639](#33639) + Ensures the namespace for CPU Plugin activity falls under: - ov::phases::inference - ov::phases::cpu::inference - ov::op::cpu::exec - ov::op::cpu::details Details: CPU support with default enabled ITT markers was limited to synchronous execution. This PR enhances the default support to include asynchronous behavior and ensures a standardized convention is followed in namespaces used. @aobolensk Please review as this is an enhancement of your work with synchronous execution. See [CVS-179230](https://jira.devtools.intel.com/browse/CVS-179230) benchmark data for smoke tests that include low-latency models. --------- Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

- Seeing if the MACROS are the cause of conditional compilation failures - Reducing string constructor overheads as the required string is already being constructed in the constructor Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

@aobolensk

…d async support (openvinotoolkit#33312) Standardizing CPU ITT marker + support for asynchronous execution - Part 2 This PR is the second of a series of PRs to standardize the ITT markers in OpenVINO that will be enabled by default through host-side instrumentation. 1. The first PR addresses the enhancements required in ITT and the framework to support the creation and propagation of IDs when asynchronous execution is in play [PR#33639](openvinotoolkit#33639). 2. This ***second PR*** will standardize ITT markers in the CPU and enhance support to include asynchronous execution. 3. The third PR will enable default markers for GPU plugin to allow visibility into inference pass begin/end and operator preparation and submission within each inference. 4. The final PR will extend the same host side markers for NPU execution, which capturing the inference span and pipeline activity. Summary of the current PR (PR#2) + Use the same convention standardized in [PR#33639](openvinotoolkit#33639) + Ensures the namespace for CPU Plugin activity falls under: - ov::phases::inference - ov::phases::cpu::inference - ov::op::cpu::exec - ov::op::cpu::details Details: CPU support with default enabled ITT markers was limited to synchronous execution. This PR enhances the default support to include asynchronous behavior and ensures a standardized convention is followed in namespaces used. @aobolensk Please review as this is an enhancement of your work with synchronous execution. See [CVS-179230](https://jira.devtools.intel.com/browse/CVS-179230) benchmark data for smoke tests that include low-latency models. --------- Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

@isanghao

… submission (#33313) - Enables default ITT markers for higher level operations such as inference pass, op preparation and submission - Follows the same guidelines to standardize the conventions for namespaces: ov::phases::gpu::inference ov::op::gpu - Supports both synchronous and asynchronous operations Enabling default GPU ITT markers using standard convention - Part 3 This PR is the **third** of a series of PRs to standardize the ITT markers in OpenVINO that will be enabled by default through host-side instrumentation. 1. The first PR addresses the enhancements required in ITT and the framework to support the creation and propagation of IDs when asynchronous execution is in play [PR#33639](#33639). 2. The second PR will standardize ITT markers in the CPU and enhance support to include asynchronous execution [PR#33312](#33312). 3. This **third** PR will enable default markers for GPU plugin to allow visibility into inference pass begin/end and operator preparation and submission within each inference. Follow standardized conventions as described in 1 and 2 4. The final PR will extend the same host side markers for NPU execution, which capturing the inference span and pipeline activity. Summary of the current PR (PR#3) Use the same convention standardized in [PR#33639](#33639) Ensures the namespace for GPU Plugin activity falls under: ov::phases::gpu::inference ov::op::gpu Details: GPU support is enabled with default ITT markers that support synchronous an asynchronous execution. This PR ensures a standardized convention is followed in namespaces used. Tickets: [CVS-179230](https://jira.devtools.intel.com/browse/CVS-179230) @isanghao Please review this as you are generally aware of what was discussed --------- Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

@isanghao

… submission (#33313) - Enables default ITT markers for higher level operations such as inference pass, op preparation and submission - Follows the same guidelines to standardize the conventions for namespaces: ov::phases::gpu::inference ov::op::gpu - Supports both synchronous and asynchronous operations Enabling default GPU ITT markers using standard convention - Part 3 This PR is the **third** of a series of PRs to standardize the ITT markers in OpenVINO that will be enabled by default through host-side instrumentation. 1. The first PR addresses the enhancements required in ITT and the framework to support the creation and propagation of IDs when asynchronous execution is in play [PR#33639](#33639). 2. The second PR will standardize ITT markers in the CPU and enhance support to include asynchronous execution [PR#33312](#33312). 3. This **third** PR will enable default markers for GPU plugin to allow visibility into inference pass begin/end and operator preparation and submission within each inference. Follow standardized conventions as described in 1 and 2 4. The final PR will extend the same host side markers for NPU execution, which capturing the inference span and pipeline activity. Summary of the current PR (PR#3) Use the same convention standardized in [PR#33639](#33639) Ensures the namespace for GPU Plugin activity falls under: ov::phases::gpu::inference ov::op::gpu Details: GPU support is enabled with default ITT markers that support synchronous an asynchronous execution. This PR ensures a standardized convention is followed in namespaces used. Tickets: [CVS-179230](https://jira.devtools.intel.com/browse/CVS-179230) @isanghao Please review this as you are generally aware of what was discussed --------- Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

tovinkere · 2026-02-24T07:16:35Z

@praasz @aobolensk Closing this PR and reopening where the async helpers are associated with NPU markers.

### Details: #Feature enhancement - Part 1 - *Core Domains:* This is a part of PR# [33639]( #33639) which has been split. - This PR addresses the standardization of Read/Compiler phases of the runtime to use the new domain names - Also reverts the SyncInferenceCPU marker type from REGION_BASE to TASK_BASE as the second PR in this series will add the necessary hooks for asynchronous inferences. ### Tickets: - [CVS-179230](https://jira.devtools.intel.com/browse/CVS-179230) ### AI Assistance: - *AI assistance used: no* --------- Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

…vinotoolkit#34278) ### Details: #Feature enhancement - Part 1 - *Core Domains:* This is a part of PR# [33639]( openvinotoolkit#33639) which has been split. - This PR addresses the standardization of Read/Compiler phases of the runtime to use the new domain names - Also reverts the SyncInferenceCPU marker type from REGION_BASE to TASK_BASE as the second PR in this series will add the necessary hooks for asynchronous inferences. ### Tickets: - [CVS-179230](https://jira.devtools.intel.com/browse/CVS-179230) ### AI Assistance: - *AI assistance used: no* --------- Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

tovinkere requested a review from a team as a code owner January 16, 2026 07:39

praasz self-assigned this Jan 16, 2026

praasz approved these changes Jan 16, 2026

View reviewed changes

praasz added this to the 2026.0 milestone Jan 16, 2026

praasz assigned aobolensk Jan 16, 2026

praasz requested a review from aobolensk January 16, 2026 07:51

This was referenced Jan 16, 2026

[OV][ITT][CPU Plugin] Align ITT markers to standard convention and add async support #33312

Merged

[OV][ITT][GPU Plugin] Enable default ITT markers for inference and op submission #33313

Merged

github-actions bot added category: inference OpenVINO Runtime library - Inference category: Core OpenVINO Core (aka ngraph) labels Jan 16, 2026

sys-openvino-ci added the ExternalIntelPR External contributor from Intel label Jan 16, 2026

aobolensk reviewed Jan 19, 2026

View reviewed changes

Merge branch 'master' into itt_markers_extn_core2

ca698fc

praasz added the Code Freeze label Jan 19, 2026

aobolensk reviewed Jan 19, 2026

View reviewed changes

praasz removed the Code Freeze label Jan 19, 2026

praasz added 3 commits January 19, 2026 12:33

Fix code style iasync_infer_request.cpp

9ffcf50

Update code style iasync_infer_request.cpp

7406de7

Update iasync_infer_request.cpp

550863d

tovinkere added 2 commits January 30, 2026 10:51

[OV][ITT] Addresses reviewer feedback

30bef83

Signed-off-by: Vasanth Tovinkere <vasanth.tovinkere@intel.com>

tovinkere requested review from a team as code owners February 9, 2026 17:26

github-actions bot added the category: CPU OpenVINO CPU plugin label Feb 9, 2026

tovinkere mentioned this pull request Feb 24, 2026

[OV][ITT] Updates the core runtime phases with new domain names #34278

Merged

tovinkere closed this Feb 24, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[OV][ITT] Enhance ITT MACROS to accept metadata for ID propagation - Redux#33639

[OV][ITT] Enhance ITT MACROS to accept metadata for ID propagation - Redux#33639
tovinkere wants to merge 8 commits intoopenvinotoolkit:masterfrom
tovinkere:itt_markers_extn_core2

tovinkere commented Jan 16, 2026 •

edited

Loading

Uh oh!

praasz commented Jan 16, 2026

Uh oh!

praasz commented Jan 19, 2026

Uh oh!

aobolensk Jan 19, 2026

Uh oh!

praasz commented Jan 19, 2026

Uh oh!

Uh oh!

aobolensk Jan 19, 2026

Uh oh!

aobolensk Jan 19, 2026

Uh oh!

aobolensk Jan 19, 2026

Uh oh!

praasz commented Jan 19, 2026

Uh oh!

tovinkere commented Feb 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

tovinkere commented Jan 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Tickets:

Uh oh!

praasz commented Jan 16, 2026

Uh oh!

praasz commented Jan 19, 2026

Uh oh!

aobolensk Jan 19, 2026

Choose a reason for hiding this comment

Uh oh!

praasz commented Jan 19, 2026

Uh oh!

Uh oh!

aobolensk Jan 19, 2026

Choose a reason for hiding this comment

Uh oh!

aobolensk Jan 19, 2026

Choose a reason for hiding this comment

Uh oh!

aobolensk Jan 19, 2026

Choose a reason for hiding this comment

Uh oh!

praasz commented Jan 19, 2026

Uh oh!

tovinkere commented Feb 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

tovinkere commented Jan 16, 2026 •

edited

Loading