added python tool to convert hugging face tokenizer to openvino ir format#132
Merged
xipingyan merged 9 commits intoxipingyan:master_tr_module_genaifrom Mar 16, 2026
Merged
Conversation
Add samples/tools/ directory with Python dev-only utilities: - hf_tokenizer_to_ov_ir.py: convert HF tokenizer to OV IR format - requirements.txt: Python dependencies (transformers, openvino-tokenizers) - README.md: usage documentation These tools are for development use only and are not included in release builds. Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
- Move qwen3_omni_case_compare.py into samples/tools/ - Rename requirements.txt to requirements_hf_tokenizer_to_ov.txt (per-tool naming convention for independent dependencies) - Update tools/README.md with both tools documented - Update bat launcher and modeling README with new paths - qwen3_omni_case_compare.py uses stdlib only, no extra requirements Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
xzhan34
force-pushed
the
master_tr_module_genai
branch
from
d34d6e3 to
175570a
Compare
xipingyan
requested changes
Mar 16, 2026
Co-authored-by: Xiping Yan <xiping.yan@intel.com>
Co-authored-by: Xiping Yan <xiping.yan@intel.com>
Co-authored-by: Xiping Yan <xiping.yan@intel.com>
Co-authored-by: Xiping Yan <xiping.yan@intel.com>
xipingyan
reviewed
Mar 16, 2026
There was a problem hiding this comment.
Pull request overview
This PR adds a small “dev tools” area under src/cpp/src/modeling/samples/tools with Python utilities for tokenizer conversion and Qwen3-Omni case comparison, and updates modeling documentation to reference the new tools location.
Changes:
- Add
hf_tokenizer_to_ov.py(+ requirements file) to convert HuggingFace tokenizers to OpenVINO IR. - Add
qwen3_omni_case_compare.pyto run multi-device / multi-precision C++ (and optionally Python) comparison runs and emit a JSON report. - Add tools README and adjust
src/cpp/src/modeling/README.mdto point to the newtools/directory and updated script path.
Reviewed changes
Copilot reviewed 4 out of 5 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
src/cpp/src/modeling/samples/tools/requirements_hf_tokenizer_to_ov.txt |
Adds pip dependencies for tokenizer conversion tool. |
src/cpp/src/modeling/samples/tools/hf_tokenizer_to_ov.py |
New HF tokenizer → OpenVINO IR conversion script. |
src/cpp/src/modeling/samples/tools/qwen3_omni_case_compare.py |
New case-comparison harness across devices/precisions with reporting. |
src/cpp/src/modeling/samples/tools/README.md |
Documents the added dev tools and how to run them. |
src/cpp/src/modeling/README.md |
Updates modeling docs to reference the new tools/ path; currently has a Markdown formatting issue. |
You can also share your feedback on Copilot code review. Take the survey.
xipingyan
reviewed
Mar 16, 2026
| padding_side: str | None = None, | ||
| ): | ||
| try: | ||
| from transformers import AutoTokenizer |
Owner
There was a problem hiding this comment.
move from transformers import AutoTokenizerto header？
There was a problem hiding this comment.
Pull request overview
Adds a small set of dev-only Python tools under src/cpp/src/modeling/samples/tools to support tokenizer IR conversion and Qwen3-Omni case/perf comparisons, and updates modeling docs to reference these tools.
Changes:
- Add
hf_tokenizer_to_ov.pyplus a dedicatedrequirements_*.txtfor converting HF tokenizers to OpenVINO IR. - Add
qwen3_omni_case_compare.pyharness to run multiple Qwen3-Omni cases across devices/precisions and emit a JSON report. - Add dev-tools README and update
src/cpp/src/modeling/README.mdto reference the newtools/location.
Reviewed changes
Copilot reviewed 4 out of 5 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| src/cpp/src/modeling/samples/tools/requirements_hf_tokenizer_to_ov.txt | Declares Python deps for tokenizer conversion tool |
| src/cpp/src/modeling/samples/tools/hf_tokenizer_to_ov.py | New tokenizer→IR conversion script |
| src/cpp/src/modeling/samples/tools/qwen3_omni_case_compare.py | New multi-case comparison/perf harness for Qwen3-Omni |
| src/cpp/src/modeling/samples/tools/README.md | Documents dev tools and usage |
| src/cpp/src/modeling/README.md | Updates modeling docs to reference tools/ |
You can also share your feedback on Copilot code review. Take the survey.
| try: | ||
| from transformers import AutoTokenizer | ||
| except ImportError: | ||
| sys.exit("ERROR: 'transformers' is required. Install: pip install -r requirements.txt") |
src/cpp/src/modeling/README.md
Outdated
Comment on lines
7
to
9
| ├── models/ # Model implementations | ||
|
|
||
| ``` |
xzhan34
force-pushed
the
master_tr_module_genai
branch
4 times, most recently
from
4ea16b0 to
c9fad36
Compare
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
xzhan34
force-pushed
the
master_tr_module_genai
branch
from
c9fad36 to
caf2bf0
Compare
xipingyan
approved these changes
Mar 16, 2026
xipingyan
merged commit 68a3599
into
xipingyan:master_tr_module_genai
49 of 50 checks passed
xipingyan
added a commit
that referenced
this pull request
Mar 16, 2026
…rmat (#132) * Add dev tools: HuggingFace tokenizer to OpenVINO IR converter Add samples/tools/ directory with Python dev-only utilities: - hf_tokenizer_to_ov_ir.py: convert HF tokenizer to OV IR format - requirements.txt: Python dependencies (transformers, openvino-tokenizers) - README.md: usage documentation These tools are for development use only and are not included in release builds. Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com> * Move qwen3_omni_case_compare.py to tools/ and reorganize - Move qwen3_omni_case_compare.py into samples/tools/ - Rename requirements.txt to requirements_hf_tokenizer_to_ov.txt (per-tool naming convention for independent dependencies) - Update tools/README.md with both tools documented - Update bat launcher and modeling README with new paths - qwen3_omni_case_compare.py uses stdlib only, no extra requirements Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com> * Update src/cpp/src/modeling/samples/tools/README.md Co-authored-by: Xiping Yan <xiping.yan@intel.com> * Update src/cpp/src/modeling/samples/tools/README.md Co-authored-by: Xiping Yan <xiping.yan@intel.com> * Update src/cpp/src/modeling/README.md Co-authored-by: Xiping Yan <xiping.yan@intel.com> * Update src/cpp/src/modeling/README.md Co-authored-by: Xiping Yan <xiping.yan@intel.com> * Move openvino and openvino_tokenizers imports to module level * Move transformers import to module level * README: add collapsible sections, Linux examples, and generalize paths Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com> --------- Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com> Co-authored-by: Xiping Yan <xiping.yan@intel.com>
xipingyan
added a commit
that referenced
this pull request
Mar 18, 2026
* Support QWen VL video inputs (#2514)
1. Enable video preprocessing for Qwen VL model.
Add ov::Property<std::vector<ov::Tensor>> videos{"videos"};
2. Support: mix images and videos input.
3. The main updates for Qwen-VL series models:
-- For video: For 2-in-1 merging, if 9 images are input, only 5 images
are actually processed.
-- For image: For 2-in-1 merging, we only double each image, so if we
input 9 images, we only actually process 9 images.
-- Introduce "`If`" node, merge video and image preprocess into one OV
subgroup.
**tickets**: CVS-173219
---------
Signed-off-by: xipingya <xiping.yan@intel.com>
Signed-off-by: xiping.yan <xiping.yan@intel.com>
Co-authored-by: Wanglei Shen <wanglei.shen@intel.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Chen Peter <peter.chen@intel.com>
Co-authored-by: Artur Paniukov <chgk1101@gmail.com>
Co-authored-by: Roman Kazantsev <roman.kazantsev@intel.com>
* Correct the test ID
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update condition for enum value conflicting with windows headers (#2979)
## Description
This PR updates the preprocessor condition for the deprecated
`ALTERNATE` enum value in `KVCrushAnchorPointMode` to avoid conflicts
with Windows headers and exclude it on all Windows platforms
CVS-175618
## Checklist:
- [ ] Tests have been updated or added to cover the new code - N/A
- [x] This patch fully addresses the ticket
- [ ] I have made corresponding changes to the documentation - N/A
* Trigger CI
* Revert "Trigger CI"
This reverts commit 96d778b344c06b59d41c4a22a35ad96869480270.
* Fix for deadlock in python callback
* draft module genai POC.
Signed-off-by: xipingya <xiping.yan@intel.com>
* Draft commit.
Signed-off-by: xipingya <xiping.yan@intel.com>
* tmp backup
Signed-off-by: xipingya <xiping.yan@intel.com>
* Fix load yaml error on Ubuntu24.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* module connect finish.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* finish module sort.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* param + img_preprocessing, test pass.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* Data flow work pass.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* Add TextEncoderModule initiate and run (#5)
* add text encoder
* fix cmt
* add initiate call
* rebase and pass input
* simplify interface export. only keep pipeline export. (#6)
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* Enable print each module config (#7)
* Remove module type convert.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* Add print yaml config of each module.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
---------
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* add ut (#8)
Enable pipeline and module's Unit Test.
* add description to config print (#9)
* Enable img preprocess and Enable unit test (#13)
* draft
* draft.
* tmp
* enable test.
Signed-off-by: xipingya <xiping.yan@intel.com>
* draft test img preprocess.
Signed-off-by: xipingya <xiping.yan@intel.com>
* enable test.
Signed-off-by: xipingya <xiping.yan@intel.com>
* update readme
* fix input name
---------
Signed-off-by: xipingya <xiping.yan@intel.com>
* Xp/fix img preprocess test bug (#14)
* Print diff value.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* update test script
* ut modules
* compare threshold change to 0.01.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
---------
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* Add vision encoder (#11)
* Fix memory leak os IBaseModule
This issue is caused by the circular reference of
InputModule and Output module. Use weak_ptr can
solve this issue.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* Add modular vision encoder to genai
Add modular vision encoder to genai.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
---------
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* Introduce log (#15)
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* Add model type to module desc (#16)
Add model type to module desc. Modules can use this param
to initialize according to model.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* add TextEmbeddingModule (#10)
* add TextEmbeddingModule
* rebase, add module test, fix cmt
* small fix
* Add normalize prompt to TextEncoderModule (#17)
Add normalize prompt to TextEncoderModule.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* PythonAPI and auto padding unit test's input and output nodes. (#18)
* padding input output module
* python api draft
* fix rebase conflict issue
* python api draft
Signed-off-by: xipingya <xiping.yan@intel.com>
* export ModulePipeline for python api.
Signed-off-by: xipingya <xiping.yan@intel.com>
* only input 4 dim image.
Signed-off-by: xipingya <xiping.yan@intel.com>
* python test pass
Signed-off-by: xipingya <xiping.yan@intel.com>
* add readme
---------
Signed-off-by: xipingya <xiping.yan@intel.com>
* Add EmbeddingMerger module (#19)
* add merger module
* small fix
* Add all completed module to pipeline test (#20)
Add all completed module for now to pipeline test.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* Use CPU to test pipeline (#21)
Use CPU to test pipeline.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* Add module pipe line test with torch llm (#22)
Add module pipe line test with torch llm.
Please follow the README in samples/python/module_genai
to run the test.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* enable output position id on vision_embedding. (#24)
Signed-off-by: xipingya <xiping.yan@intel.com>
* Add LLMInference module (#25)
to do:
1. wip how to pass position id list parameter between two modules.
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* fix pr25 build issue (#26)
* try to fix
Signed-off-by: xipingya <xiping.yan@intel.com>
* Fix build error.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* just output 16 new tokens.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* Eanble batch and single test for llm
Signed-off-by: xiping.yan <xiping.yan@intel.com>
---------
Signed-off-by: xipingya <xiping.yan@intel.com>
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* llm module: delete dead code. (#27)
with pr#26, this code is dead now, deleted it.
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* fix build error (#29)
* Xp/py api add auto padding (#28)
* update python api description
* update: auto padding param and result
Signed-off-by: xipingya <xiping.yan@intel.com>
* tmp
* Enable new interface to pass path and config context string.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* support to pass yaml config context to MOdulePipline.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* update python test,
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* Move auto padding to utils.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* in unit test, add checking yaml format.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
---------
Signed-off-by: xipingya <xiping.yan@intel.com>
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* cpp full pipeline test pass (#30)
* llm module: fix the wrong moudule name in the error message. (#31)
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* Draft yaml config for z-image. (#32)
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* Add ZImageDenoiserLoopModule (#33)
* Add Transformer module to module pipeline
Add Transformer module to module pipeline.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* Add zimage transformer python test with optimum-intel
Add zimage transformer python test with optimum-intel.
This test uses text encoder and vae decoder from optimum-intel,
and the transformer is from module genai pipeline.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* Fix params auto padding issue when multiple inputs exist
Fix params auto padding issue when multiple inputs exist.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* Rename TransformerModule to ZImageDenoiserLoopModule
Rename TransformerModule to ZImageDenoiserLoopModule.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* Move tensor operations to other file
Move tensor operations to other file and reduce
the inference steps of z image denoiser test.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
---------
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* Fix module pipeline precision issue (#35)
Fix module pipeline precision issue.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* Add VAEDecoderModule module (#34)
* Add VAEDecoderModule module
cpp unit test passed.
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* enhanced zimage transformer python test with vae decoder
1. rename python unit name to z_image.
2. rename python unit script file.
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* VAEDecoderModule: add enable_postprocess flag parameter for tiling support
1. unit test with disabel postprocess flag passed.
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
---------
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* fix z_image python pipeline issue. (#37)
rename self.pipe.get_output("image_output") -> self.pipe.get_output("image")
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* Make z image denoise module to return 4d tensor (#39)
* Make z image denoise module to return 4d tensor
Make z image denoise module to return 4d tensor.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* Update src/cpp/src/module_genai/modules/md_zimage_denoiser_loop.cpp
---------
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
Co-authored-by: Xiping Yan <xiping.yan@intel.com>
* Xp/enable zimg tiling (#38)
* tmp
* implement blend_v blend_h
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* ut test pass.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* fix merge bug
* split parse yaml.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* pass submodule yaml success.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* tmp
* 1: introduce PipelineDesc,
2: wrapper each module's common constructor declare.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* reuse modulepipelineimpl for sub_pipeline
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* test pass.
* fix concate issue.
* draft post-process.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* draft post model
* draft
* fix build issue
* simplify compare_big_tensor
* python pipeline run pass. still have accuracy issue.
* test without tiling
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* support batch, need to input latents
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* update post_process.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* fix test error.
* local test pass with enable tiling.
Signed-off-by: xipingya <xiping.yan@intel.com>
* Update samples/cpp/module_genai/ut_modules/VAEDecoderTilingModule.cpp
---------
Signed-off-by: xiping.yan <xiping.yan@intel.com>
Signed-off-by: xipingya <xiping.yan@intel.com>
* migrate to gtest (#40)
* draft first gtest example.
Signed-off-by: xipingyan <xiping.yan@intel.com>
* test pass.
Signed-off-by: xipingyan <xiping.yan@intel.com>
* ignore dumped file
* remove old test
* fix build error
---------
Signed-off-by: xipingyan <xiping.yan@intel.com>
* Xp/fix tiling accuracy (#43)
* reduce duplicated calc in loop
* fix blend_v offset error
* vae_decoder_tiling: fix build issue. (#44)
/src/cpp/src/module_genai/modules/md_vae_decoder_tiling.cpp:119:45: required from here
/usr/include/c++/13/bits/stl_construct.h:119:7: error: no matching function for call to
‘ov::Model::Model(std::vector<std::shared_ptr<ov::Node> >, std::vector<std::shared_ptr<ov::op::v0::Parameter> >)’
119 | ::new((void*)__p) _Tp(std::forward<_Args>(__args)...);
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* Fix z image denoiser loop module error timestep issue (#42)
* Add python test for pipeline module
Add python test for pipeline module.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* Fix z image denoiser loop module error timestep issue
Fix z image denoiser loop module error timestep issue.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* Move accuracy test with python to other repo
Move accuracy test with python to other repo.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* Update src/cpp/src/image_generation/schedulers/z_image_flow_match_euler_discrete.hpp
---------
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
Co-authored-by: Xiping Yan <xiping.yan@intel.com>
* Add ClipTextEncoderModule (#41)
* rebase and initial version
* fix output idx
* update
* dix cmt
* Add negative prompt operation (#45)
Add negative prompt operation.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* fix function signature mismatch issue. (#47)
this can cause build issue under windows enviroment.
module_genai_ut_app.dir/manifest.res" failed (exit code 1120) with the following output:
main.cpp.obj : error LNK2019: unresolved external symbol "void __cdecl test_genai_module_ut_modules(int,char * * const)"
main.cpp.obj : error LNK2019: unresolved external symbol "void __cdecl test_genai_module_ut_pipelines(int,char * * const)"
Hint on symbols that are defined and could potentially match:
"int __cdecl test_genai_module_ut_pipelines(int,char * * const)" (?test_genai_module_ut_pipelines@@YAHHQEAPEAD@Z)
samples\cpp\module_genai\module_genai_ut_app.exe : fatal error LNK1120: 2 unresolved externals
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* Support async execute based on oneTBB flow (#46)
* Support async execute based on oneTBB flow
Signed-off-by: xipingya <xiping.yan@intel.com>
* test generate asyn interface.
* update gtest: wrapper test data(images) and test model.
Signed-off-by: xipingya <xiping.yan@intel.com>
* Need to cpy ov tokenizer.
Signed-off-by: xipingya <xiping.yan@intel.com>
---------
Signed-off-by: xipingya <xiping.yan@intel.com>
* enable profiling. (#49)
Signed-off-by: xipingya <xiping.yan@intel.com>
* add SaveImageModule for module genai (#48)
SaveImageModule added to align with comfyui SaveImage node.
to avoid external dependency, the saved image type is bmp format.
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* update (#50)
* Add gtest unit test (#51)
Add VisionEncoder and ZImageDenoiserLoop gtest unit test.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* add yaml (#53)
* module pipeline yaml config string validator. (#54)
added a simple pipeline yaml validator to our module pipeline as a skeleton.
1. only check must have 3 nodes in the config string.
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* Add fake edge to support sync nodes execution under async mode. (#52)
* tmp
* enable fake edge to process sync nodes under asyn mode.
Signed-off-by: xipingya <xiping.yan@intel.com>
* Move init_sub_pipeline to base class.
Signed-off-by: xipingya <xiping.yan@intel.com>
* add async submoudle test
Signed-off-by: xipingya <xiping.yan@intel.com>
* update input_node
---------
Signed-off-by: xipingya <xiping.yan@intel.com>
* fix test fail issue. (#56)
Signed-off-by: xipingya <xiping.yan@intel.com>
* fix ModulePipelineValidator.cpp one unit test failure issue (#57)
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* Add latent image module (#55)
* Add latent image module
Add latent image module.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* Rename LatentImageModule to RandomLatentImageModule
Rename LatentImageModule to RandomLatentImageModule.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* Fix VisionEncoderTest issue
Fix VisionEncoderTest issue.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* Update tests/module_genai/cpp/modules/RandomLatentImageModule.cpp
---------
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
Co-authored-by: Xiping Yan <xiping.yan@intel.com>
* enable input ov_module for LLM module. (#59)
* enable input ov_module.
Signed-off-by: xipingya <xiping.yan@intel.com>
* tmp
* tmp
* local test pass.
Signed-off-by: xipingyan <xiping.yan@intel.com>
* Revert "enable input ov_module."
This reverts commit 739057b152e598914b9b350298e41796a2b4af90.
* remove old test.
---------
Signed-off-by: xipingya <xiping.yan@intel.com>
Signed-off-by: xipingyan <xiping.yan@intel.com>
* Fix denoiser loop test accuracy issue (#61)
Fix denoiser loop test accuracy issue.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* update test script, add some debug or profiling macro (#62)
* migrate cliptextencodermodule gtest (#60)
* migrate
* fix cmt
* Refactor module register and enable new arch for dummy module test. (#63)
* 1: Remove FakeModuleA/B/C/D. Just keep DummyModuleBase a module type.
2: Simplify new Module registor.
For example for add new Module;
1: Add new module type in module_type.hpp, for example: X(ResultModule, 50)
2: Add "GENAI_REGISTER_MODULE_SAME(ResultModule);" in new Module implimentation.
3: Refactor generate_asyn based on new dummy module.
Signed-off-by: xipingyan <xiping.yan@intel.com>
* unify name to dummy.
Signed-off-by: xipingyan <xiping.yan@intel.com>
---------
Signed-off-by: xipingyan <xiping.yan@intel.com>
* Support dummy module for end to end test. (#64)
* draft
* test
* 2 test files can use dummy module together via registor table.
Signed-off-by: xipingyan <xiping.yan@intel.com>
* Unify dummy test.
* Add dumpy mudule guide.
Signed-off-by: xipingyan <xiping.yan@intel.com>
* asyn -> async
---------
Signed-off-by: xipingyan <xiping.yan@intel.com>
* ComfyUI API and Workflow Json support (#58)
* ModulePipeline::validate_config_string handle empty string case
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* code refactor: use utils_image to replace load_image
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* fix a runtime issue ClipTextEncoderModule
When CFG is disabled (guidance_scale <= 1.0), set empty vector instead of uninitialized tensor
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* modular genai: comfyui api json parser class with validator.
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* modular genai: comfyui workflow json to api json convertion class.
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* modular genai: comfyui api json to genai yaml convertion class.
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* module genai: expose comfyui json to yaml convertion api.
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* modular genai: added unit test for comfyui json to yaml convertion
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* modular genai: sample application with comfyui json support
for both comfyui api json and workflow json.
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
---------
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* Eanble all DtatType test. (#66)
Signed-off-by: xipingyan <xiping.yan@intel.com>
* Optimize Z Image module pipeline (#65)
Optimize Z Image module pipeline non inferring time.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* gtest migration (#67)
* migrate SaveImageModule.cpp to gtest unit test directory
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* migrate VAEDecoderModule.cpp to gtest unit test directory
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* openvino genai: windows enviroment build issue fixing
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
---------
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* migrate all left ut to gtest. (#68)
* 1: EmbeddingMergerModuleTest finish.
Signed-off-by: xipingyan <xiping.yan@intel.com>
* TextEmbeddingModuleTest
* TextEncoderModuleTest
Signed-off-by: xipingyan <xiping.yan@intel.com>
* Tiling
---------
Signed-off-by: xipingyan <xiping.yan@intel.com>
* Enable tiling size param (#69)
* update code format.
* remove useless header
* enable tile_size input with param
Signed-off-by: xipingyan <xiping.yan@intel.com>
---------
Signed-off-by: xipingyan <xiping.yan@intel.com>
* c++ api python binding (#70)
* module pipeline new c++ api python binding
1. api: validate_config, validate_config_string
2. api: comfyui_json_to_yaml, comfyui_json_string_to_yaml
3. struct: ValidationResult
4. struct: ConfigModelsMap
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* python unit test for modular pipeline python binding
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* update z_image python sample application
1. support new c++ api python binding.
2. update sample application to support comfyui json file
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
---------
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* Enable some type check and add get_input function. (#71)
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* add tile_size support for comfyui z_image pipeline (#72)
* yaml: fix issue for global_context case
The check std::filesystem::path(content).has_parent_path() is incorrectly detecting the YAML content
as a file path because "global_context:" contains a colon which Windows interprets as a drive letter pattern.
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* comfyui: expose tile_size to application
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* update z_image python pipeline app with tile_size support
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* update c++ comfyui sample application with tile_size support
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
---------
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* fixed windows build issue with msvc. (#73)
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* enable modular_genai sample to be compiled by default (#75)
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* Wan diffusion (#74)
* Update random latent module to fit wan model
Update random latent module to fit wan model.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* Add UniPCMultistepScheduler
Add UniPCMultistepScheduler for Wan 2.1 model.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* Use TEST_P to test unipc multistep scheduler
Use TEST_P to test unipc multistep scheduler.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
---------
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* add new submoule: SaveVideoModule (#76)
* add SaveVideoModule to support avi container with mjpeg video format
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* gtest unit test for SaveVideoModule submodule
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
---------
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* fixed windows build issue from PR #74. (#77)
unipc_multistep_scheduler.cpp(502,53): error C2065: 'M_PI': undeclared identifier [openvino_genai_obj.vcxproj]
nipc_multistep_scheduler.cpp(502,101): error C2065: 'M_PI': undeclared identifier [openvino_genai_obj.vcxproj]
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* comfyui json file support code refactors (#78)
* comfyui api json parser: refine get_validation_errors_string() implemenation
refine get_validation_errors_string to get more detailed error message when comfyui validation failed
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* comfyui pipeline code refactor
added a common convert_parsed_json_to_yaml for both json string and json file.
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* workflow_json code refactor: reduce duplicate code for maintaince
Pretty names for node titles when title is not specified in workflow JSON
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* for comfyui workflow json, to save the converted api json for debug purpose.
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* comfyui yaml_converter code refactor: topological_sort_nodes
with topological_sort_nodes, Iterate over sorted nodes and call each node's generator
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
---------
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* Add Wan 2.1 support to denoiser loop module (#79)
Add Wan 2.1 support to denoiser loop module.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* qwen_2.5_vl modular pipeline python sample applicaton (#81)
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* Add readme for modular genai. (#80)
* Add readme for modular genai.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* refactor samples.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* sample qwen pass.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* update readme.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* update z-image test.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
---------
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* yaml-cpp dependency management (#83)
provides yaml-cpp either from system installation or via FetchContent to
resolve windows python genai wheel pacakge runtime yaml-cpp external dependency.
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* fix PR79 module naming issue (#84)
change the comfyui code per PR79 change: ZImageDenoiserLoopModule->DenoiserLoopModule
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* update python samples (#82)
* update python samples
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* update load image
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* test qwen-vl pass.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* update readme.
* Update samples/python/module_genai/md_visual_language_chat.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* rename get_parameter_module_outputs
* update z-image readme
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* Update samples/python/module_genai/utils/utils.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* t
---------
Signed-off-by: xiping.yan <xiping.yan@intel.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update clip text encoder to support Wan 2.1 (#87)
Update clip text encoder to support Wan 2.1.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* openvino genai build python wheel BKM (#88)
* openvino genai build python wheel BKM
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* Update README_Module_GenAI.md
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
Co-authored-by: Xiping Yan <xiping.yan@intel.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* z-image cpp sample test pass. (#89)
* remove strange input name,
use: prompt and image as inputs.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* enable get_output(), for only an output case.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* output node, return output with name, this name is based on output node specification.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* add com_utils
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* Update src/cpp/src/module_genai/utils/com_utils.hpp
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Signed-off-by: xiping.yan <xiping.yan@intel.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update vae decoder to support Wan 2.1 (#91)
Update vae decoder to support Wan 2.1.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* comfyui standalone python sample application (#90)
to align with cpp sample, seperate the comfyui python code from md_image_generation.py
as a standalone application.
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* z-image with tiling for cpp samples. (#92)
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* model load once if multiple text encoder modules with the same model_path and device (#93)
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* wan2.1 comfyui json support with yaml pipeline config generation (#86)
* enhanced vae decoder tiling module's latents input support
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* SaveVideoModule update to support another 5D tensor format.
1. [B, F, H, W, C] format (channels-last, e.g., from VideoProcessor)
2. [B, C, F, H, W] format (channels-first, e.g., from VAE decoder)
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* wan2.1 text to video comfyui json support
both workflow json and api json supported. added new class type support:
1. SaveAnimatedWEBP
2. SaveWEBM
3. EmptyHunyuanLatentVideo
4. ModelSamplingSD3
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* support wan2.1 text to video comfyui json to yaml conversion
both comfyui wan2.1 t2v workflow jsoa and api json supportted
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* update comfyui cpp sample application to support wan2.1
1. comfyui wan2.1 t2v json files (both workflow and api json files) added
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
---------
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* remove WA wan 2.1 GPU T5 inference issue. (#95)
this issue is root caused: the model is should be f32 and inference precision is f32.
otherwise, nan issue happen and the output video will be full of white frames
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* wan2.1 vae tiling feature support (#96)
* unified blend operation for both 4D image and 5D video tensor type.
updated zimage pipeline to use new blend operation.
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* update wan 2.1 vae to support tile functionality
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* update VAEDecoderTilingModule and VAEDecoderModule to support both zimage and wan2.1
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* update wan2.1 comfyui yaml generation to enable and support vae tiling feature
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* update comfyui both cpp and python with use_ting argument to control wan2.1 vae tiling behavior
with addtional change to make model_path is a reuired argument.
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* update md_video_generation sample to use wan2.1 vae tiling feature
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* Update src/cpp/src/module_genai/modules/md_vae_decoder_tiling.cpp
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Xp/wan denoise loop multiple model (#94)
* asyn load/release model weights.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* enable 2 param: check_dynamic_load_weights and check_cache_dir
Signed-off-by: xiping.yan <xiping.yan@intel.com>
# Conflicts:
# src/cpp/src/module_genai/modules/md_clip_text_encoder.cpp
* local test pass.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* run work, output avi is white.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* Apply suggestion from @Copilot
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Unit test pass.
* update samples.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* default use split model.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* refactor code.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* add cpp samples. not test.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* update test.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* fix cpp samples issue.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* update readme.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* Update samples/python/module_genai/md_video_generation.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update src/cpp/src/module_genai/modules/md_denoiser_loop/splitted_model_infer.cpp
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update src/cpp/src/module_genai/modules/md_denoiser_loop/class.cpp
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update src/cpp/src/module_genai/modules/md_denoiser_loop/class.cpp
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update src/cpp/src/module_genai/modules/md_clip_text_encoder.cpp
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update tests/module_genai/cpp/modules/DenoiserLoopModule.cpp
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update src/cpp/src/module_genai/modules/md_clip_text_encoder.hpp
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update src/cpp/src/module_genai/modules/md_denoiser_loop/splitted_model_infer.cpp
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update samples/cpp/module_genai/README.md
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update samples/cpp/module_genai/README.md
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update src/cpp/src/module_genai/modules/md_denoiser_loop/splitted_model_infer.cpp
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* rename contain_ to contains_
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* fir ci issue.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
---------
Signed-off-by: xiping.yan <xiping.yan@intel.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Fix crash issue when using splitted model (#98)
Fix crash issue when using splitted model.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* Enable remote tensor for splitted model infer. (#99)
* remote context.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* Enable remote tensor.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* use genai_debug to print log.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
---------
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* windows build issue fix: convert filesystem::path to string in CSplittedModelInfer::create call (#101)
fix type mismatch:
CSplittedModelInfer::create expects const std::string& but transformer_model_path
is std::filesystem::path. Added .string() conversion to fix MSVC C2664 error.
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* Refactor model cache from global singleton to pipeline-scoped resource cache (#102)
Replace ClipTextEncoderResourceCache (global singleton) with a generic
PipelineResourceCache owned by PipelineDesc. This ensures cached resources
(compiled models, infer requests) are automatically released when the
pipeline is destroyed, avoiding stale global state across pipeline lifetimes.
- Add PipelineResourceCache with thread-safe, type-erased get_or_create()
- Move cache ownership into PipelineDesc via std::unique_ptr
- Update ClipTextEncoderModule to use pipeline-scoped cache
- Remove module-specific global cache class and its includes
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* Xp/enable release weights (#100)
* remote context.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* Enable remote tensor.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* use genai_debug to print log.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* draft
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* disable multiple thread load/release weight.
issue:
what(): Exception from src/plugins/intel_gpu/src/runtime/ocl/ocl_stream.cpp:376:
[GPU] clFinish, error code: -5 CL_OUT_OF_RESOURCES
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* add macro, easy to debug.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* Default disable dynamic load weigths, performance is low.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* Apply suggestion from @Copilot
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* enable dynamic load model wights unit test.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* skipped, only for dynamic load weights.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* dynamic load model weigths, not create inferrequest.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* Apply suggestion from @Copilot
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* add build option: ENABLE_DYNAMIC_LOAD_MODEL_WEIGHTS, default off;
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* fix build issue.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* Update src/cpp/src/module_genai/modules/md_denoiser_loop/splitted_model_infer.cpp
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* rename to release_fun
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* Fix dangling reference in async lambda captures in thread_helper.hpp (#104)
* Initial plan
* Fix dangling reference in thread_helper.hpp async lambda captures
Co-authored-by: xipingyan <47800531+xipingyan@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: xipingyan <47800531+xipingyan@users.noreply.github.com>
* -#if !ENABLE_DYNAMIC_LOAD_MODEL_WEIGHTS
+#ifndef ENABLE_DYNAMIC_LOAD_MODEL_WEIGHTS
* Fix GPU device skip condition in DenoiserLoopModule test to use substring match (#103)
* Initial plan
* Fix GPU skip condition in test to use case-insensitive substring match
Co-authored-by: xipingyan <47800531+xipingyan@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: xipingyan <47800531+xipingyan@users.noreply.github.com>
* Update src/cpp/src/module_genai/modules/md_denoiser_loop/class.cpp
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* remove duplicated: check_cache_dir
---------
Signed-off-by: xiping.yan <xiping.yan@intel.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
* rename ENABLE_DYNAMIC_LOAD_MODEL_WEIGHTS - > ENABLE_DYNAMIC_WEIGHT_MANAGEMENT (#105)
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* Enable Qwen3.5 image preprocessor (#106)
Enable Qwen3.5 image preprocessor.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* Qwen3 5 vision text encode (#108)
* Enable text encoder for Qwen 3.5
Enable text encoder for Qwen 3.5.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* Enable vision encoder module for Qwen 3.5
Enable vision encoder module for Qwen 3.5.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* Update src/cpp/src/module_genai/modules/md_vision_encoder.cpp
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
Co-authored-by: Xiping Yan <xiping.yan@intel.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* added sdpa llm inference for qwen3.5 model support (#107)
* added modeling code from openvino-new-arch repo.
this is the code base dependendcy for supporing qwen3.5 model.
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* added sdpa llm inference for qwen3.5 model support.
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* added a unit test for LLMInferenceSDPAModule for Qwen3.5
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* added compiler flag -DENABLE_OPENVINO_NEW_ARCH to control build behavior
to support both public openvino runtime and new arch for qwen3.5 support
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
---------
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* Fix build error. (#109)
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* Qwen3 5 pipeline (#111)
* Use Qwen 3.5 0.8B to test
Use Qwen 3.5 0.8B to test.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* Add Qwen 3.5 cpp pipeline sample
Add Qwen 3.5 cpp pipeline sample.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
---------
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* Use quantized Qwen 3.5 vision model (#113)
Use quantized Qwen 3.5 vision model.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* qwen3.5 llm inference moduel performanece data dump. (#114)
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* Xp/enable qwen3 5 video preprocess (#110)
* porting qwen3_vl parts implementation.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* draft commit.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* Fix unit test fail.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* introuduce vision_preprocess.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* build pass.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* Fix build error.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* test pass.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* enable video preprocess.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* tmp
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* fix bug. should malloc vector firstly.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* to qwen3_vl to type convert
* remove useless codes.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* Apply suggestion from @Copilot
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Apply suggestion from @Copilot
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Apply suggestion from @Copilot
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* dst image with batch
* Apply suggestion from @Copilot
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Apply suggestion from @Copilot
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Apply suggestion from @Copilot
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* tmp
* Apply suggestion from @Copilot
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Apply suggestion from @Copilot
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Apply suggestion from @Copilot
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Use quantized Qwen 3.5 vision model (#113)
Use quantized Qwen 3.5 vision model.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* add desc for video output
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* Apply suggestion from @Copilot
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* add some utils function.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* Eanbe a new video module.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* add video preprocess test.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* test pass.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* Apply suggestion from @Copilot
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Apply suggestion from @Copilot
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Apply suggestion from @Copilot
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* update based on copilot.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* update namespace
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* Apply suggestions from code review
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Signed-off-by: xiping.yan <xiping.yan@intel.com>
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: ZiniuLin <109139916+ZiniuLin@users.noreply.github.com>
* windows build regression fix: multiple compilation errors under MSVC C++17 (#115)
- processing_qwen3_5.cpp: pass std::vector<float>& instead of float*
to resize_bilinear_to_chw()
- qwen3_5preprocessor.cpp: add static_cast<size_t>() for int32_t config
values in ov::Shape{} brace-init to fix C2398 narrowing conversion
- qwen3_vl/vision_preprocess.cpp: add static_cast<size_t>() for int
return values in ImageSize{} to fix C2397 narrowing conversion
- VideoPreprocesModule.cpp: replace C++20 designated initializers with
C++17-compatible positional aggregate initialization (C7555)
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* Xp/enable audio process (#116)
* draft
1: rename model -> models
2: audio_preprocess.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* add name space.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* test fail.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* test pass.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* add copilot instruction.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* Update samples/cpp/module_genai/md_omni.cpp
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update samples/cpp/module_genai/md_omni.cpp
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* fix copilot issue.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* Update src/cpp/src/module_genai/modules/md_audio_preprocess.cpp
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* fix copilots comments.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
---------
Signed-off-by: xiping.yan <xiping.yan@intel.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Enable VL model for Qwen 3-Omni (#117)
Enable VL model for Qwen 3-Omni.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* Xp/update module pipeline api (#118)
* update api: add params: const ov::AnyMap& properties = {},
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* enable ov_cache_dir to sdpa llm.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* pass string for cache_dir
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* Update src/cpp/include/openvino/genai/module_genai/pipeline.hpp
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update src/python/py_module_pipeline.cpp
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Apply suggestion from @Copilot
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Apply suggestion from @Copilot
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Signed-off-by: xiping.yan <xiping.yan@intel.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* add param: warmup and perf (#121)
* add param: warmup and perf
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* add header
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* Apply suggestions from code review
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Signed-off-by: xiping.yan <xiping.yan@intel.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Enable audio input for Qwen 3-Omni (#120)
Enable audio input for Qwen 3-Omni.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* add profiling (#122)
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* fix build issue for public ov (#123)
* profiling vlm qwen3 omni and Optimized preprocessing model based on ov model. (#125)
* 1: profiling update,
2: enable ov to preprocess.
Signed-off-by: xipingya <xiping.yan@intel.com>
* test pass.
Signed-off-by: xipingya <xiping.yan@intel.com>
* Update src/cpp/src/module_genai/modules/models/qwen3_5/qwen3_5preprocessor.cpp
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update src/cpp/src/module_genai/modules/models/qwen3_omni/qwen3_utils.hpp
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* use single core.
Signed-off-by: xipingya <xiping.yan@intel.com>
* Update src/cpp/src/module_genai/modules/models/qwen3_vl/vision_preprocess.cpp
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Signed-off-by: xipingya <xiping.yan@intel.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Enable video input for Qwen3-Omni (#124)
* Enable video input for Qwen3-Omni
Enable video input for Qwen3-Omni.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* Update src/cpp/src/module_genai/modules/models/qwen3_5/qwen3_5preprocessor.hpp
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
Co-authored-by: Xiping Yan <xiping.yan@intel.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* fix merge conflict error.
Signed-off-by: xipingya <xiping.yan@intel.com>
* fix link error.
Signed-off-by: xipingya <xiping.yan@intel.com>
* fix ffmpeg7 issue (#127)
* fix ffmpeg7 issue
* enable install
* ffmpeg support 4/5/6/7
Signed-off-by: xipingya <xiping.yan@intel.com>
* add pipeline path, move some files to this path.
Signed-off-by: xipingya <xiping.yan@intel.com>
* update readme.
Signed-off-by: xipingya <xiping.yan@intel.com>
* unit test readme.
Signed-off-by: xipingya <xiping.yan@intel.com>
* remove useless changes.
Signed-off-by: xipingya <xiping.yan@intel.com>
* fix config name conflict.
Signed-off-by: xipingya <xiping.yan@intel.com>
* calling script cvt convert model in cpp source codes are not allowed. (#130)
remove them.
Signed-off-by: xipingya <xiping.yan@intel.com>
* Enable TTS for Qwen3-Omni (#128)
Enable TTS for Qwen3-Omni.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
# Conflicts:
# .gitignore
# src/cpp/src/modeling/models/qwen3_tts/modeling_qwen3_tts_speech_decoder.cpp
* fix conflict
Signed-off-by: xipingya <xiping.yan@intel.com>
* reduce diff
Signed-off-by: xipingya <xiping.yan@intel.com>
* throw exception directly.
Signed-off-by: xipingya <xiping.yan@intel.com>
* 1: add namespace for unit test.
2: Fix xeon test fail.
Signed-off-by: xipingya <xiping.yan@intel.com>
* qwen3.5 vision encode skipped. beacase required new-arch ov.
Signed-off-by: xipingya <xiping.yan@intel.com>
* Increase VAEDecoderTestData threahold for xeon test
Signed-off-by: xipingya <xiping.yan@intel.com>
* fix u8 diff with thr 1.
Signed-off-by: xipingya <xiping.yan@intel.com>
* text encode, replace wqen3.5 35b with 0.8b
Signed-off-by: xipingya <xiping.yan@intel.com>
* cpu/gpu diff, fix
Signed-off-by: xiping.yan <xiping.yan@intel.com>
* added python tool to convert hugging face tokenizer to openvino ir format (#132)
* Add dev tools: HuggingFace tokenizer to OpenVINO IR converter
Add samples/tools/ directory with Python dev-only utilities:
- hf_tokenizer_to_ov_ir.py: convert HF tokenizer to OV IR format
- requirements.txt: Python dependencies (transformers, openvino-tokenizers)
- README.md: usage documentation
These tools are for development use only and are not included in
release builds.
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* Move qwen3_omni_case_compare.py to tools/ and reorganize
- Move qwen3_omni_case_compare.py into samples/tools/
- Rename requirements.txt to requirements_hf_tokenizer_to_ov.txt
(per-tool naming convention for independent dependencies)
- Update tools/README.md with both tools documented
- Update bat launcher and modeling README with new paths
- qwen3_omni_case_compare.py uses stdlib only, no extra requirements
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* Update src/cpp/src/modeling/samples/tools/README.md
Co-authored-by: Xiping Yan <xiping.yan@intel.com>
* Update src/cpp/src/modeling/samples/tools/README.md
Co-authored-by: Xiping Yan <xiping.yan@intel.com>
* Update src/cpp/src/modeling/README.md
Co-authored-by: Xiping Yan <xiping.yan@intel.com>
* Update src/cpp/src/modeling/README.md
Co-authored-by: Xiping Yan <xiping.yan@intel.com>
* Move openvino and openvino_tokenizers imports to module level
* Move transformers import to module level
* README: add collapsible sections, Linux examples, and generalize paths
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
---------
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
Co-authored-by: Xiping Yan <xiping.yan@intel.com>
* Fix inference issue when only provides text for Qwen3-Omni (#131)
Fix inference issue when only provides text for Qwen3-Omni.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* Wrapper LLM infer sdpa.
Signed-off-by: xipingya <xiping.yan@intel.com>
* print input infor.
Signed-off-by: xipingya <xiping.yan@intel.com>
* tmp update
Signed-off-by: xipingya <xiping.yan@intel.com>
* fix again
Signed-off-by: xipingya <xiping.yan@intel.com>
* device default to cpu
* Add Qwen3-Omni model save (#135)
* Add Qwen3-Omni model save
Add Qwen3-Omni model save.
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
* Update samples/cpp/video_generation/CMakeLists.txt
* Update samples/cpp/video_generation/CMakeLists.txt
* Update samples/cpp/video_generation/CMakeLists.txt
* Update samples/cpp/visual_language_chat/CMakeLists.txt
* Update samples/cpp/visual_language_chat/CMakeLists.txt
* Update samples/cpp/visual_language_chat/CMakeLists.txt
---------
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
Co-authored-by: Xiping Yan <xiping.yan@intel.com>
* fix core issue
* Add Qwen3-VL model implementation and sample (#133)
- Add Qwen3-VL text model (modeling_qwen3_vl_text.cpp/hpp)
- Add Qwen3-VL vision model (modeling_qwen3_vl_vision.cpp/hpp)
- Add modeling_qwen3_vl.cpp sample: image+text inference with
vision encoder + text decoder, supporting quantization options
- Update samples CMakeLists.txt to build modeling_qwen3_vl target
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* Remove Qwen3 Omni/TTS codes and add modeling_private submodule support (#134)
* Remove Qwen3 Omni/TTS code and add ENABLE_MODELING_PRIVATE build guard
Changes:
- Remove models/qwen3_omni/: audio, vision, text, TTS modeling and processing (19 files)
- Remove models/qwen3_tts/: code predictor, speech decoder, talker, TTS pipeline (9 files)
- Remove samples: modeling_qwen3_omni.cpp, modeling_qwen3_omni_tts_min.cpp, CMakeLists.txt
- Remove tools: qwen3_omni_case_compare.py
- CMakeLists.txt: exclude modeling_private/ sources when ENABLE_MODELING_PRIVATE is OFF
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* added modeling_private submodule with cmake files change.
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Update cmake/features.cmake
Co-authored-by: Xiping Yan <xiping.yan@intel.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
---------
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Xiping Yan <xiping.yan@intel.com>
* fix model address
* move qwen3_omni to private
Signed-off-by: xipingya <xiping.yan@intel.com>
---------
Signed-off-by: xipingya <xiping.yan@intel.com>
Signed-off-by: xiping.yan <xiping.yan@intel.com>
Signed-off-by: Ziniu Lin <ziniu.lin@intel.com>
Signed-off-by: Zhang, Xiaolin <xiaolin.zhang@intel.com>
Signed-off-by: xipingyan <xiping.yan@intel.com>
Co-authored-by: Wanglei Shen <wanglei.shen@intel.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Chen Peter <peter.chen@intel.com>
Co-authored-by: Artur Paniukov <chgk1101@gmail.com>
Co-authored-by: Roman Kazantsev <roman.kazantsev@intel.com>
Co-authored-by: Yaroslav Tarkan <yaroslav.tarkan@intel.com>
Co-authored-by: Vladimir Zlobin <vladimir.zlobin@intel.com>
Co-authored-by: Stanislav Gonorovskii <stanislav.gonorovskii@intel.com>
Co-authored-by: binbin Deng <108676127+plusbang@users.noreply.github.com>
Co-authored-by: ZiniuLin <109139916+ZiniuLin@users.noreply.github.com>
Co-authored-by: xzhan34 <xiaolin.zhang@intel.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
CVS-###
Fixes #(issue)
Checklist: