Skip to content

Fix GIL crash in image generation pipelines on repeated generate() calls#3383

Merged
Wovchena merged 7 commits intoopenvinotoolkit:masterfrom
sgonorov:fix-t2i-generation-flux-bug
Mar 13, 2026
Merged

Fix GIL crash in image generation pipelines on repeated generate() calls#3383
Wovchena merged 7 commits intoopenvinotoolkit:masterfrom
sgonorov:fix-t2i-generation-flux-bug

Conversation

@sgonorov
Copy link
Contributor

Fix pybind11::handle::dec_ref() GIL crash that occurs on the second generate() call in image generation pipelines (Text2Image, Image2Image, Inpainting) when a Python-backed generator (e.g. TorchGenerator) is passed.

  • Introduce GilSafeGeneratorWrapper in py_utils.cpp that acquires the GIL before destroying the underlying Python generator, preventing unsafe Py_DECREF when the old generator is overwritten during subsequent generate() calls with py::gil_scoped_release active
  • Add GIL-safe destructor to TorchGenerator as defense-in-depth for its py::object members
  • Remove unused params_have_torch_generator helper
  • Work around ov::Core::read_model infinite recursion caused by a template overload in OpenVINO 2026.1.0 core.hpp by using std::as_const() on non-const AnyMap arguments across tokenizer, whisper, and RAG pipelines

Issue CVS-181541

@sgonorov sgonorov self-assigned this Feb 24, 2026
@github-actions github-actions bot added category: image generation Image generation pipelines category: whisper Whisper pipeline category: GHA CI based on Github actions category: tokenizers Tokenizer class or submodule update category: Python API Python API for GenAI category: GGUF GGUF file reader category: RAG RAG pipeline components labels Feb 24, 2026
@sgonorov sgonorov force-pushed the fix-t2i-generation-flux-bug branch from 7ad558a to 656595e Compare February 24, 2026 10:40
@sgonorov sgonorov marked this pull request as ready for review February 26, 2026 09:52
@sgonorov sgonorov force-pushed the fix-t2i-generation-flux-bug branch from 656595e to 028a6d3 Compare March 9, 2026 09:30
@sgonorov sgonorov requested a review from Wovchena March 9, 2026 09:31
@sgonorov sgonorov added this to the 2026.1 milestone Mar 9, 2026
@Wovchena Wovchena requested review from Copilot and likholat March 9, 2026 09:38
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes a Python GIL-related crash when reusing image generation pipelines with a Python-backed RNG generator across repeated generate() calls, and adds a workaround for an OpenVINO ov::Core::read_model recursion issue in 2026.1.0 when passing non-const AnyMap.

Changes:

  • Wrap Python-backed ov::genai::Generator instances in a GIL-safe wrapper to ensure safe destruction while py::gil_scoped_release is active.
  • Add multi-call image generation regression tests and wire them into Linux/Windows CI.
  • Use std::as_const() on AnyMap passed to core.read_model(...) in tokenizer/whisper/RAG to avoid the OpenVINO 2026.1.0 overload recursion.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 10 comments.

Show a summary per file
File Description
tests/python_tests/test_image_generation_multi_call.py New regression tests covering repeated generate() calls with/without generators and callbacks
src/python/py_utils.cpp Adds GilSafeGeneratorWrapper and wraps Generator conversion in py_object_to_any()
src/python/py_image_generation_pipelines.cpp Adds GIL-safe cleanup in TorchGenerator destructor; removes unused helper
src/cpp/src/whisper/pipeline_static.cpp Uses std::as_const(properties_copy) for read_model(...)
src/cpp/src/whisper/pipeline.cpp Uses std::as_const(properties_copy) for read_model(...)
src/cpp/src/whisper/models/statefull_decoder.cpp Uses std::as_const(properties) for read_model(...)
src/cpp/src/tokenizer/tokenizer_impl.cpp Uses std::as_const(filtered_properties) for tokenizer/detokenizer read_model(...)
src/cpp/src/rag/text_embedding_pipeline.cpp Uses std::as_const(properties) for read_model(...)
.github/workflows/windows.yml Adds the new multi-call test file to the Windows Python test run
.github/workflows/linux.yml Adds the new multi-call test file to the Linux Python test run
.github/CODEOWNERS Assigns code owners for the new test and Python image generation bindings file

You can also share your feedback on Copilot code review. Take the survey.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 5 comments.


You can also share your feedback on Copilot code review. Take the survey.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.


You can also share your feedback on Copilot code review. Take the survey.

@sgonorov sgonorov force-pushed the fix-t2i-generation-flux-bug branch from b62c949 to f82e1d7 Compare March 11, 2026 17:48
@sgonorov sgonorov requested review from Wovchena and Copilot March 11, 2026 17:54
@sgonorov sgonorov force-pushed the fix-t2i-generation-flux-bug branch from f82e1d7 to 18cb8a0 Compare March 12, 2026 12:54
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 5 comments.

@Wovchena Wovchena enabled auto-merge March 12, 2026 13:03
@sgonorov sgonorov force-pushed the fix-t2i-generation-flux-bug branch from 18cb8a0 to 06ec871 Compare March 13, 2026 08:45
Copilot AI review requested due to automatic review settings March 13, 2026 09:39
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated 1 comment.

@Wovchena Wovchena added this pull request to the merge queue Mar 13, 2026
Merged via the queue into openvinotoolkit:master with commit df2eb49 Mar 13, 2026
275 of 300 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

category: GGUF GGUF file reader category: GHA CI based on Github actions category: image generation Image generation pipelines category: Python API Python API for GenAI category: RAG RAG pipeline components category: tokenizers Tokenizer class or submodule update category: whisper Whisper pipeline Code Freeze

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants