Skip to content

index.mdx

Latest commit

 

History

History
113 lines (91 loc) · 4.13 KB

index.mdx

File metadata and controls

113 lines (91 loc) · 4.13 KB

import BasicGenerationConfiguration from '@site/docs/use-cases/_shared/_basic_generation_configuration.mdx'; import ChatScenario from '@site/docs/use-cases/_shared/_chat_scenario.mdx'; import GenerationConfigurationWorkflow from '@site/docs/use-cases/_shared/_generation_configuration_workflow.mdx'; import Streaming from '@site/docs/use-cases/_shared/_streaming.mdx';

Additional Usage Options

:::tip Check out Python and C++ visual language chat samples. :::

Use Image or Video Tags in Prompt

The prompt can contain <ov_genai_image_i> with i replaced with an actual zero based index to refer to an image. Reference to images used in previous prompts isn't implemented. A model's native image tag can be used instead of <ov_genai_image_i>. These tags are:

  1. InternVL2: <image>\n
  2. llava-1.5-7b-hf: <image>
  3. LLaVA-NeXT: <image>
  4. LLaVA-NeXT-Video: <image>
  5. nanoLLaVA: <image>\n
  6. nanoLLaVA-1.5: <image>\n
  7. MiniCPM-o-2_6: <image>./</image>\n
  8. MiniCPM-V-2_6: <image>./</image>\n
  9. Phi-3-vision: <|image_i|>\n - the index starts with one
  10. Phi-4-multimodal-instruct: <|image_i|>\n - the index starts with one
  11. Qwen2-VL: <|vision_start|><|image_pad|><|vision_end|>
  12. Qwen2.5-VL: <|vision_start|><|image_pad|><|vision_end|>
  13. Qwen3-VL: <|vision_start|><|image_pad|><|vision_end|>
  14. gemma-3-4b-it: <start_of_image>

Model's native video tag can be used to refer to a video. These tags are:

  1. LLaVA-NeXT-Video: <video>
  2. Qwen2-VL: <|vision_start|><|video_pad|><|vision_end|>
  3. Qwen2.5-VL: <|vision_start|><|video_pad|><|vision_end|>
  4. Qwen3-VL: <|vision_start|><|video_pad|><|vision_end|>

If the prompt doesn't contain image or video tags, but images or videos are provided, the tags are prepended to the prompt.

Use Different Generation Parameters

Similar to text generation, VLM pipelines support various generation parameters to control the text output.

```python import openvino_genai as ov_genai 
      pipe = ov_genai.VLMPipeline(model_path, "CPU")

      # Get default configuration
      config = pipe.get_generation_config()

      # Modify parameters
      config.max_new_tokens = 100
      config.temperature = 0.7
      config.top_k = 50
      config.top_p = 0.9
      config.repetition_penalty = 1.2

      # Generate text with custom configuration
      output = pipe.generate(prompt, images, config)
      ```
  </TabItemPython>
  <TabItemCpp>
      ```cpp
      int main() {
          ov::genai::VLMPipeline pipe(model_path, "CPU");

          // Get default configuration
          auto config = pipe.get_generation_config();

          // Modify parameters
          config.max_new_tokens = 100;
          config.temperature = 0.7f;
          config.top_k = 50;
          config.top_p = 0.9f;
          config.repetition_penalty = 1.2f;

          // Generate text with custom configuration
          auto output = pipe.generate(prompt, images, config);
      }
      ```
  </TabItemCpp>
  <TabItemJS>
      ```javascript
      import { VLMPipeline }  from 'openvino-genai-node';

      const pipe = await VLMPipeline(modelPath, "CPU", {});

      // Create custom generation configuration
      const config = {
          max_new_tokens: 100,
          temperature: 0.7,
          top_k: 50,
          top_p: 0.9,
          repetition_penalty: 1.2
      };

      // Generate text with custom configuration
      const output = await pipe.generate(prompt, {
          images: images,
          generationConfig: config
      });
      ```
  </TabItemJS>