Skip to content

input prompts only, test qwen3 omni.#119

Open
xipingyan wants to merge 2 commits intomaster_modular_genaifrom
xp/qwen3_omni_text_input_only_samples
Open

input prompts only, test qwen3 omni.#119
xipingyan wants to merge 2 commits intomaster_modular_genaifrom
xp/qwen3_omni_text_input_only_samples

Conversation

@xipingyan
Copy link
Copy Markdown
Owner

@xipingyan xipingyan commented Mar 7, 2026

This pull request adds a new configuration YAML file for the Qwen3-Omni model pipeline in the samples/cpp/module_genai/config_yaml directory. The configuration defines the structure and flow for prompt encoding, LLM inference, and result formatting, enabling streamlined integration and testing of the Qwen3-Omni model.

New Qwen3-Omni pipeline configuration:

  • Added config_prompt.yaml to specify the pipeline modules and their parameters for the Qwen3-Omni model, including prompt encoding, LLM inference, and result collection.
  • Defined module chaining and device allocation (GPU) for both the TextEncoderModule and LLMInferenceSDPAModule, supporting efficient model execution.
  • Provided explicit model paths and parameters for both encoder and LLM inference modules, facilitating reproducible testing and deployment.
  • Established output structure via the ResultModule, ensuring generated text is properly collected and formatted.

ISSUE:

[INFO] Running module: pipeline_params
[INFO]     Pass prompt to output port
[INFO] Running module: prompt_encoder
[INFO] Running module: llm
[ERROR] Exception from src/inference/src/cpp/infer_request.cpp:224:
Check 'TRShape::broadcast_merge_into(result_shape, input_shapes[input_port], broadcast_spec)' failed at src/core/shape_inference/include/select_shape_inference.hpp:40:
While validating node 'opset1::Select Select_401311 () -> ()' with friendly_name 'Select_401311':
Argument shapes are inconsistent.

Copilot AI review requested due to automatic review settings March 7, 2026 08:04
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a new text-only (prompt-only) pipeline configuration YAML for the Qwen3-Omni model. The configuration strips away the vision/audio modules from the full config.yaml to create a simplified pipeline that only processes text prompts through the TextEncoderModuleLLMInferenceSDPAModuleResultModule chain, similar to the existing Qwen3.5-0.8B/config_text.yaml.

Changes:

  • Added config_prompt.yaml for Qwen3-Omni with a text-only pipeline (prompt encoding → LLM inference → result collection), omitting vision and audio preprocessing stages.
  • Defined GPU device allocation for both the TextEncoderModule and LLMInferenceSDPAModule with model paths pointing to the Qwen3-Omni-4B test models.
  • Configured the pipeline to generate up to 512 tokens and collect the output through a ResultModule.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

source: "pipeline_params.prompt"
outputs:
- name: "input_ids"
type: "OVTensor"
Copy link

Copilot AI Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The prompt_encoder is missing the mask output declaration. The TextEncoderModule::run() method unconditionally writes to this->outputs["mask"].data for the QWEN3_OMNI model type (see src/cpp/src/module_genai/modules/md_text_encoder.cpp:159). While this won't crash (the map auto-inserts), it's inconsistent with every other TextEncoderModule config in the codebase — for example, the analogous text-only config at samples/cpp/module_genai/config_yaml/Qwen3.5-0.8B/config_text.yaml:21-22 and the sibling config.yaml at samples/cpp/module_genai/config_yaml/Qwen3-Omni/config.yaml:48-49 both declare the mask output. Add the mask output to maintain consistency and properly document the module's outputs.

Suggested change
type: "OVTensor"
type: "OVTensor"
- name: "mask"
type: "OVTensor"

Copilot uses AI. Check for mistakes.
type: "ParameterModule"
outputs:
- name: "prompt"
type: "String"
Copy link

Copilot AI Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A blank line is missing between the pipeline_params and prompt_encoder module definitions. All other pipeline YAML configs in this directory consistently use blank lines to separate top-level modules (e.g., Qwen3.5-0.8B/config_text.yaml:10, Qwen3-Omni/config.yaml:12). Add a blank line after line 9 for consistency.

Suggested change
type: "String"
type: "String"

Copilot uses AI. Check for mistakes.
Signed-off-by: xiping.yan <xiping.yan@intel.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants