input prompts only, test qwen3 omni.#119
input prompts only, test qwen3 omni.#119xipingyan wants to merge 2 commits intomaster_modular_genaifrom
Conversation
There was a problem hiding this comment.
Pull request overview
This PR adds a new text-only (prompt-only) pipeline configuration YAML for the Qwen3-Omni model. The configuration strips away the vision/audio modules from the full config.yaml to create a simplified pipeline that only processes text prompts through the TextEncoderModule → LLMInferenceSDPAModule → ResultModule chain, similar to the existing Qwen3.5-0.8B/config_text.yaml.
Changes:
- Added
config_prompt.yamlfor Qwen3-Omni with a text-only pipeline (prompt encoding → LLM inference → result collection), omitting vision and audio preprocessing stages. - Defined GPU device allocation for both the
TextEncoderModuleandLLMInferenceSDPAModulewith model paths pointing to the Qwen3-Omni-4B test models. - Configured the pipeline to generate up to 512 tokens and collect the output through a
ResultModule.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| source: "pipeline_params.prompt" | ||
| outputs: | ||
| - name: "input_ids" | ||
| type: "OVTensor" |
There was a problem hiding this comment.
The prompt_encoder is missing the mask output declaration. The TextEncoderModule::run() method unconditionally writes to this->outputs["mask"].data for the QWEN3_OMNI model type (see src/cpp/src/module_genai/modules/md_text_encoder.cpp:159). While this won't crash (the map auto-inserts), it's inconsistent with every other TextEncoderModule config in the codebase — for example, the analogous text-only config at samples/cpp/module_genai/config_yaml/Qwen3.5-0.8B/config_text.yaml:21-22 and the sibling config.yaml at samples/cpp/module_genai/config_yaml/Qwen3-Omni/config.yaml:48-49 both declare the mask output. Add the mask output to maintain consistency and properly document the module's outputs.
| type: "OVTensor" | |
| type: "OVTensor" | |
| - name: "mask" | |
| type: "OVTensor" |
| type: "ParameterModule" | ||
| outputs: | ||
| - name: "prompt" | ||
| type: "String" |
There was a problem hiding this comment.
A blank line is missing between the pipeline_params and prompt_encoder module definitions. All other pipeline YAML configs in this directory consistently use blank lines to separate top-level modules (e.g., Qwen3.5-0.8B/config_text.yaml:10, Qwen3-Omni/config.yaml:12). Add a blank line after line 9 for consistency.
| type: "String" | |
| type: "String" |
Signed-off-by: xiping.yan <xiping.yan@intel.com>
…en3_omni_text_input_only_samples
This pull request adds a new configuration YAML file for the Qwen3-Omni model pipeline in the
samples/cpp/module_genai/config_yamldirectory. The configuration defines the structure and flow for prompt encoding, LLM inference, and result formatting, enabling streamlined integration and testing of the Qwen3-Omni model.New Qwen3-Omni pipeline configuration:
config_prompt.yamlto specify the pipeline modules and their parameters for the Qwen3-Omni model, including prompt encoding, LLM inference, and result collection.TextEncoderModuleandLLMInferenceSDPAModule, supporting efficient model execution.ResultModule, ensuring generated text is properly collected and formatted.ISSUE: