[GPU] Make singleton OCL Context by isanghao · Pull Request #32738 · openvinotoolkit/openvino

isanghao · 2025-11-07T12:11:56Z

Details:

Make OCL context as singleton
GenAI tries to change its behavior to multi ov::Core, while buffers can be shared across ov::Core
We had two choices
- explicit context sharing among multiple ov::Core
- singleton OCL context
The expectation is that single OCL context is created for one process. So it should be fine to use singleton ocl context.
Singleton OCL context was implemented with weak ptr. It was design choice to release cl_context before plugin unload.
As Plugin class owns cl_context through shared_ptr, singleton cl_context will be freed when all Core instances are destructed.

Tickets:

CVS-178537

src/inference/include/openvino/runtime/core.hpp

Copilot

Pull request overview

This PR implements singleton OCL context for the Intel GPU plugin to enable buffer sharing across multiple ov::Core instances. The change ensures that a single OCL context is created per process rather than per Core instance, allowing GenAI applications to use multiple Core instances while sharing GPU resources efficiently.

Key changes:

Moved OCL context initialization from instance-level to static singleton in Plugin::get_default_contexts()
Updated documentation across C++, Python, JavaScript, and Node.js bindings to reflect the new multi-Core behavior
Added test to verify singleton OCL context behavior across multiple Core instances

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
`src/plugins/intel_gpu/tests/functional/behavior/infer_request.cpp`	Added test case verifying that OCL contexts from different Core instances are identical
`src/plugins/intel_gpu/src/plugin/plugin.cpp`	Moved `m_default_contexts` and `m_default_contexts_once` from member variables to static variables for singleton behavior
`src/plugins/intel_gpu/include/intel_gpu/plugin/plugin.hpp`	Removed member variable declarations that are now static in the implementation
`src/inference/include/openvino/runtime/core.hpp`	Updated documentation to clarify that multiple Core instances now share underlying device resources
`src/bindings/python/src/pyopenvino/core/core.cpp`	Updated Python binding documentation to reflect resource sharing behavior
`src/bindings/python/src/openvino/_ov_api.pyi`	Updated Python type stub documentation for consistency
`src/bindings/js/node/lib/addon.ts`	Updated JavaScript/TypeScript documentation to reflect resource sharing
`docs/sphinx_setup/api/nodejs_api/openvino-node/interfaces/Core.rst`	Updated Node.js API documentation for consistency

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

StefaniaHergane · 2026-01-15T12:20:00Z

Please check behavior of sample app hello_query_device - it is executed in NPU CI and hangs on some Windows machines.

isanghao · 2026-01-21T06:59:00Z

This is not required for 2026.0 because genai is still using singleton core. openvinotoolkit/openvino.genai#2952

This can be merged in 2026.1 timeframe.

isanghao · 2026-01-21T07:47:00Z

src/inference/src/cpp/core.cpp

        ov::DeviceIDParser parser(device_name);
        std::string devName = parser.get_device_name();
-
+        _impl->get_plugin(devName).cleanup();


@praasz could you review this part? I added new plugin API cleanup to ensure context cleanup before unloading plugin. Without this, it caused hang on windows.

@isanghao
I agree with @olpipi idea to clean up in GPU plugin dtor if required

Thanks for the feedback @praasz @olpipi. I realized that this new strategy does not need explicit cleanup. I reverted the changes.

olpipi · 2026-01-21T10:48:48Z

src/inference/src/cpp/core.cpp

@@ -265,17 +271,17 @@ void Core::unload_plugin(const std::string& device_name) {
    OV_CORE_CALL_STATEMENT({
        ov::DeviceIDParser parser(device_name);
        std::string devName = parser.get_device_name();
-
+        _impl->get_plugin(devName).cleanup();


What reason to add this new method to plugins api? Can you perform the same steps inside of ov::intel_gpu::Plugin dtor?

Thanks for the feedback @praasz @olpipi. I realized that this new strategy does not need explicit cleanup. I reverted the changes.

ahnyoung-paul

LGTM

jade-cho

LGTM

### Details: - Make OCL context as singleton - GenAI tries to change its behavior to multi ov::Core, while buffers can be shared across ov::Core - We had two choices - explicit context sharing among multiple ov::Core - singleton OCL context - The expectation is that single OCL context is created for one process. So it should be fine to use singleton ocl context. - Singleton OCL context was implemented with weak ptr. It was design choice to release cl_context before plugin unload. - As Plugin class owns cl_context through shared_ptr, singleton cl_context will be freed when all Core instances are destructed. ### Tickets: - CVS-178537

github-actions bot added the category: GPU OpenVINO GPU plugin label Nov 7, 2025

isanghao added the do_not_merge label Nov 7, 2025

Wovchena approved these changes Nov 10, 2025

View reviewed changes

Wovchena mentioned this pull request Nov 10, 2025

[GENAI]Introduce add_extension to genai. openvinotoolkit/openvino.genai#2952

Open

isanghao force-pushed the singleton_ocl_context branch from e56e26f to 5a3c56e Compare November 24, 2025 11:33

github-actions bot added category: inference OpenVINO Runtime library - Inference category: CPP API OpenVINO CPP API bindings labels Nov 24, 2025

isanghao changed the title ~~WIP: [GPU] Make singleton OCL Context~~ [GPU] Make singleton OCL Context Nov 24, 2025

isanghao removed the do_not_merge label Nov 24, 2025

isanghao marked this pull request as ready for review November 24, 2025 11:34

isanghao requested review from a team as code owners November 24, 2025 11:34

p-durandin reviewed Nov 24, 2025

View reviewed changes

src/inference/include/openvino/runtime/core.hpp Outdated Show resolved Hide resolved

Wovchena reviewed Nov 24, 2025

View reviewed changes

src/inference/include/openvino/runtime/core.hpp Outdated Show resolved Hide resolved

isanghao requested review from a team as code owners November 25, 2025 03:47

isanghao requested review from ValentinaKats and removed request for a team November 25, 2025 03:47

github-actions bot added category: Python API OpenVINO Python bindings category: docs OpenVINO documentation category: JS API OpenVino JS API Bindings labels Nov 25, 2025

Wovchena approved these changes Nov 25, 2025

View reviewed changes

Wovchena requested a review from Copilot November 25, 2025 05:16

Copilot AI reviewed Nov 25, 2025

View reviewed changes

p-durandin approved these changes Nov 25, 2025

View reviewed changes

jade-cho approved these changes Dec 2, 2025

View reviewed changes

hyunback approved these changes Dec 2, 2025

View reviewed changes

isanghao enabled auto-merge December 15, 2025 04:29

isanghao force-pushed the singleton_ocl_context branch from ceee6c1 to e1b7306 Compare December 17, 2025 03:30

isanghao added 3 commits January 16, 2026 11:15

[GPU] Make singleton OCL Context

efc629e

fix for hang on windows

ff30c9f

revert unnecessary change

d4d1f9e

isanghao force-pushed the singleton_ocl_context branch from ad172d0 to ff30c9f Compare January 20, 2026 05:08

isanghao added this pull request to the merge queue Jan 21, 2026

geunhwan added this to the 2026.0 milestone Jan 21, 2026

geunhwan added the Code Freeze label Jan 21, 2026

isanghao removed this pull request from the merge queue due to a manual request Jan 21, 2026

isanghao added do_not_merge and removed Code Freeze labels Jan 21, 2026

isanghao modified the milestones: 2026.0, 2026.1 Jan 21, 2026

isanghao commented Jan 21, 2026

View reviewed changes

olpipi reviewed Jan 21, 2026

View reviewed changes

isanghao added 3 commits January 27, 2026 18:33

remove unnecessary explicit cleanup

982c9b6

Merge branch 'master' into singleton_ocl_context

64aec45

Merge branch 'master' into singleton_ocl_context

8f3fcd4

ahnyoung-paul approved these changes Feb 5, 2026

View reviewed changes

jade-cho approved these changes Feb 5, 2026

View reviewed changes

isanghao added this pull request to the merge queue Feb 5, 2026

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Feb 5, 2026

Wovchena added this pull request to the merge queue Feb 5, 2026

Merged via the queue into openvinotoolkit:master with commit 8322000 Feb 5, 2026
212 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[GPU] Make singleton OCL Context#32738

[GPU] Make singleton OCL Context#32738
Wovchena merged 6 commits intoopenvinotoolkit:masterfrom
isanghao:singleton_ocl_context

isanghao commented Nov 7, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

StefaniaHergane commented Jan 15, 2026 •

edited

Loading

Uh oh!

Uh oh!

isanghao commented Jan 21, 2026

Uh oh!

isanghao Jan 21, 2026

Uh oh!

praasz Jan 22, 2026

Uh oh!

isanghao Jan 27, 2026

Uh oh!

olpipi Jan 21, 2026

Uh oh!

isanghao Jan 27, 2026

Uh oh!

ahnyoung-paul left a comment

Uh oh!

jade-cho left a comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

11 participants

Conversation

isanghao commented Nov 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Details:

Tickets:

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

StefaniaHergane commented Jan 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

isanghao commented Jan 21, 2026

Uh oh!

isanghao Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

praasz Jan 22, 2026

Choose a reason for hiding this comment

Uh oh!

isanghao Jan 27, 2026

Choose a reason for hiding this comment

Uh oh!

olpipi Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

isanghao Jan 27, 2026

Choose a reason for hiding this comment

Uh oh!

ahnyoung-paul left a comment

Choose a reason for hiding this comment

Uh oh!

jade-cho left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

11 participants

isanghao commented Nov 7, 2025 •

edited

Loading

StefaniaHergane commented Jan 15, 2026 •

edited

Loading