[Bug]: gemma-3-12b-it-int8-ov generates gibberish when extracting text from an image

[gaynetdinov]

OpenVINO Version

2025.4.1

Operating System

ubuntu 24.04.03 LTS

Device used for inference

GPU

Framework

None

Model used

https://huggingface.co/OpenVINO/gemma-3-12b-it-int8-ov

Issue description

A simple test with text test passes, but a request to extra text from an image fails. The same behaviour I've seen in llm-scaler btw, so maybe I'm doing something wrong?

docker-compose

services:
  ovms:
    image: openvino/model_server:latest-gpu
    container_name: ovms
    restart: "unless-stopped"
    group_add:
      - "993"
    devices:
      - /dev/dri:/dev/dri

    ports:
      - "8350:8000"
    volumes:
      - ./models:/models:rw
    command:
      - --source_model
      - "OpenVINO/gemma-3-12b-it-int8-ov"
      - --model_repository_path
      - "models"
      - --task
      - "text_generation"
      - --rest_port
      - "8000"
      - --cache_size
      - "4"
      - --target_device
      - "GPU.1"



  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui-ov
    restart: unless-stopped
    ports:
      - "5081:8080"    
    volumes:
      - open-webui-data-ov:/app/backend/data
    environment:
      TZ: "Europe/Berlin"

volumes:
  open-webui-data-ov:

It goes endlessly until I hit stop in the UI, but even after that the ovms server is generating endless the logs I attached below. At this time my sparkle b60 is taking off with its single blower fan.

Step-by-step reproduction

• docker compose up
• go to localhost:5081
• open new chat for gemma
• upload any image and ask what is it or ask to extract text from it

Relevant log output

damirca@homelab:~/ov$ docker compose up
WARN[0000] Found orphan containers ([ov-ov_bench-1]) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
Attaching to open-webui-ov, ovms
open-webui-ov  | Loading WEBUI_SECRET_KEY from file, not provided as an environment variable.
open-webui-ov  | Loading WEBUI_SECRET_KEY from .webui_secret_key
ovms           | [2026-02-19 13:19:28.078][1][serving][info][server.cpp:88] OpenVINO Model Server 2025.4.1.7bc56cf8a
ovms           | [2026-02-19 13:19:28.078][1][serving][info][server.cpp:89] OpenVINO backend 2025.4.1.0rc1
ovms           | Path already exists on local filesystem. Skipping download to path: models/OpenVINO/gemma-3-12b-it-int8-ov
ovms           | Model: OpenVINO/gemma-3-12b-it-int8-ov downloaded to: models/OpenVINO/gemma-3-12b-it-int8-ov
ovms           | Graph: graph.pbtxt created in: models/OpenVINO/gemma-3-12b-it-int8-ov
ovms           | [2026-02-19 13:19:28.114][1][serving][info][pythoninterpretermodule.cpp:37] PythonInterpreterModule starting
ovms           | [2026-02-19 13:19:28.179][1][serving][info][pythoninterpretermodule.cpp:50] PythonInterpreterModule started
ovms           | [2026-02-19 13:19:28.364][1][modelmanager][info][modelmanager.cpp:156] Available devices for Open VINO: CPU, GPU.0, GPU.1
ovms           | [2026-02-19 13:19:28.365][1][serving][info][capimodule.cpp:40] C-APIModule starting
ovms           | [2026-02-19 13:19:28.365][1][serving][info][capimodule.cpp:42] C-APIModule started
ovms           | [2026-02-19 13:19:28.366][1][serving][info][grpcservermodule.cpp:110] GRPCServerModule starting
ovms           | [2026-02-19 13:19:28.366][1][serving][info][grpcservermodule.cpp:114] GRPCServerModule started
ovms           | [2026-02-19 13:19:28.366][1][serving][info][grpcservermodule.cpp:115] Port was not set. GRPC server will not be started.
ovms           | [2026-02-19 13:19:28.366][1][serving][info][httpservermodule.cpp:35] HTTPServerModule starting
ovms           | [2026-02-19 13:19:28.366][1][serving][info][httpservermodule.cpp:39] Will start 12 REST workers
ovms           | [2026-02-19 13:19:28.366][51][serving][info][drogon_http_server.cpp:137] Binding REST server to address: 0.0.0.0:8000
ovms           | [2026-02-19 13:19:28.416][1][serving][info][drogon_http_server.cpp:164] REST server listening on port 8000 with 12 unary threads and 12 streaming threads
ovms           | [2026-02-19 13:19:28.416][1][serving][info][http_server.cpp:248] API key not provided via --api_key_file or API_KEY environment variable. Authentication will be disabled.
ovms           | [2026-02-19 13:19:28.416][1][serving][info][httpservermodule.cpp:52] HTTPServerModule started
ovms           | [2026-02-19 13:19:28.416][1][serving][info][httpservermodule.cpp:53] Started REST server at 0.0.0.0:8000
ovms           | [2026-02-19 13:19:28.416][1][serving][info][servablemanagermodule.cpp:51] ServableManagerModule starting
ovms           | [2026-02-19 13:19:28.418][1][serving][info][mediapipegraphdefinition.cpp:423] MediapipeGraphDefinition initializing graph nodes
ovms           | [2026-02-19 13:19:28.418][1][modelmanager][info][servable_initializer.cpp:443] Initializing Visual Language Model Continuous Batching servable
open-webui-ov  | INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
open-webui-ov  | INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
open-webui-ov  | WARNI [open_webui.env]
open-webui-ov  |
open-webui-ov  | WARNING: CORS_ALLOW_ORIGIN IS SET TO '*' - NOT RECOMMENDED FOR PRODUCTION DEPLOYMENTS.
open-webui-ov  |
open-webui-ov  | WARNI [langchain_community.utils.user_agent] USER_AGENT environment variable not set, consider setting it to identify your requests.
open-webui-ov  |
open-webui-ov  |  ██████╗ ██████╗ ███████╗███╗   ██╗    ██╗    ██╗███████╗██████╗ ██╗   ██╗██╗
open-webui-ov  | ██╔═══██╗██╔══██╗██╔════╝████╗  ██║    ██║    ██║██╔════╝██╔══██╗██║   ██║██║
open-webui-ov  | ██║   ██║██████╔╝█████╗  ██╔██╗ ██║    ██║ █╗ ██║█████╗  ██████╔╝██║   ██║██║
open-webui-ov  | ██║   ██║██╔═══╝ ██╔══╝  ██║╚██╗██║    ██║███╗██║██╔══╝  ██╔══██╗██║   ██║██║
open-webui-ov  | ╚██████╔╝██║     ███████╗██║ ╚████║    ╚███╔███╔╝███████╗██████╔╝╚██████╔╝██║
open-webui-ov  |  ╚═════╝ ╚═╝     ╚══════╝╚═╝  ╚═══╝     ╚══╝╚══╝ ╚══════╝╚═════╝  ╚═════╝ ╚═╝
open-webui-ov  |
open-webui-ov  |
open-webui-ov  | v0.8.0 - building the best AI user interface.
open-webui-ov  |
open-webui-ov  | https://github.com/open-webui/open-webui
open-webui-ov  |
Fetching 30 files: 100%|██████████| 30/30 [00:00<00:00, 18586.28it/s]
Loading weights: 100%|██████████| 103/103 [00:00<00:00, 3041.28it/s, Materializing param=pooler.dense.weight]
open-webui-ov  | BertModel LOAD REPORT from: /app/backend/data/cache/embedding/models/models--sentence-transformers--all-MiniLM-L6-v2/snapshots/c9745ed1d9f207416be6d2e6f8de32d1f16199bf
open-webui-ov  | Key                     | Status     |  |
open-webui-ov  | ------------------------+------------+--+-
open-webui-ov  | embeddings.position_ids | UNEXPECTED |  |
open-webui-ov  |
open-webui-ov  | Notes:
open-webui-ov  | - UNEXPECTED   :can be ignored when loading from different task/architecture; not ok if you expect identical arch.
open-webui-ov  | INFO:     Started server process [1]
open-webui-ov  | INFO:     Waiting for application startup.
open-webui-ov  | 2026-02-19 14:19:37.924 | INFO     | open_webui.utils.logger:start_logger:165 - GLOBAL_LOG_LEVEL: INFO
open-webui-ov  | 2026-02-19 14:19:37.926 | INFO     | open_webui.main:lifespan:615 - Installing external dependencies of functions and tools...
open-webui-ov  | 2026-02-19 14:19:38.109 | INFO     | open_webui.utils.plugin:install_frontmatter_requirements:423 - No requirements found in frontmatter.
open-webui-ov  | /usr/local/lib/python3.11/site-packages/jwt/api_jwt.py:371: InsecureKeyLengthWarning: The HMAC key is 16 bytes long, which is below the minimum recommended length of 32 bytes for SHA256. See RFC 7518 Section 3.2.
open-webui-ov  |   decoded = self.decode_complete(
open-webui-ov  | 2026-02-19 14:19:40.025 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:54190 - "GET /api/version HTTP/1.1" 200
ovms           | [2026-02-19 13:19:45.225][173][llm_executor][info][llm_executor.hpp:90] All requests: 0; Scheduled requests: 0;
ovms           | [2026-02-19 13:19:45.226][1][modelmanager][info][mediapipegraphdefinition.cpp:184] Mediapipe: OpenVINO/gemma-3-12b-it-int8-ov inputs:
ovms           | name: input; mapping: ; shape: (); precision: UNDEFINED; layout: ...
ovms           | [2026-02-19 13:19:45.226][1][modelmanager][info][mediapipegraphdefinition.cpp:185] Mediapipe: OpenVINO/gemma-3-12b-it-int8-ov outputs:
ovms           | name: output; mapping: ; shape: (); precision: UNDEFINED; layout: ...
ovms           | [2026-02-19 13:19:45.226][1][modelmanager][info][mediapipegraphdefinition.cpp:186] Mediapipe: OpenVINO/gemma-3-12b-it-int8-ov kfs pass through: false
ovms           | [2026-02-19 13:19:45.226][1][modelmanager][info][pipelinedefinitionstatus.hpp:59] Mediapipe: OpenVINO/gemma-3-12b-it-int8-ov state changed to: AVAILABLE after handling: ValidationPassedEvent:
ovms           | [2026-02-19 13:19:45.227][174][modelmanager][info][modelmanager.cpp:1200] Started model manager thread
ovms           | [2026-02-19 13:19:45.227][175][modelmanager][info][modelmanager.cpp:1219] Started cleaner thread
ovms           | [2026-02-19 13:19:45.227][1][serving][info][servablemanagermodule.cpp:55] ServableManagerModule started
open-webui-ov  | 2026-02-19 14:20:57.469 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:54525 - "GET /_app/version.json HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:21:57.461 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:54791 - "GET /_app/version.json HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:22:57.467 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:54893 - "GET /_app/version.json HTTP/1.1" 200





open-webui-ov  | 2026-02-19 14:23:57.464 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55124 - "GET /_app/version.json HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:24:12.906 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55140 - "GET /static/loader.js HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:24:12.906 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55141 - "GET /static/custom.css HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:24:12.911 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55140 - "GET /static/splash.png HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:24:13.190 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55140 - "GET /api/config HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:24:13.193 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55141 - "GET /static/favicon.png HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:24:13.231 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55141 - "GET /api/v1/auths/ HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:24:13.235 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55141 - "GET /api/version HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:24:13.255 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55140 - "GET /api/config HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:24:13.257 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55141 - "GET /api/v1/users/user/settings HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:24:13.325 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55141 - "GET /api/v1/configs/banners HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:24:13.335 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55140 - "GET /api/v1/tools/ HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:24:13.337 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55143 - "GET /api/v1/users/user/settings HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:24:13.354 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55144 - "GET /api/v1/users/69c3ac32-2ba2-44e2-82cf-ef48f1c0b62c/profile/image HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:24:13.358 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55146 - "GET /api/v1/chats/all/tags HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:24:13.359 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55145 - "GET /api/v1/folders/ HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:24:13.369 | INFO     | open_webui.routers.openai:get_all_models:482 - get_all_models()
open-webui-ov  | 2026-02-19 14:24:13.371 | INFO     | open_webui.routers.ollama:get_all_models:322 - get_all_models()
open-webui-ov  | 2026-02-19 14:24:13.436 | ERROR    | open_webui.routers.ollama:send_get_request:104 - Connection error: Cannot connect to host host.docker.internal:11434 ssl:default [Name or service not known]
open-webui-ov  | 2026-02-19 14:24:13.440 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55145 - "GET /api/v1/chats/pinned HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:24:13.453 | ERROR    | open_webui.routers.ollama:send_get_request:104 - Connection error: Cannot connect to host host.docker.internal:11434 ssl:default [Name or service not known]
open-webui-ov  | 2026-02-19 14:24:13.464 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55143 - "GET /api/models HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:24:13.469 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55144 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:24:13.596 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55145 - "GET /api/v1/functions/ HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:24:13.597 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55143 - "GET /api/v1/tools/ HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:24:13.612 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55146 - "GET /api/v1/models/model/profile/image?id=OpenVINO/gemma-3-12b-it-int8-ov&lang=en-US HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:24:13.613 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55144 - "GET /api/v1/models/model/profile/image?id=undefined&lang=en-US HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:24:13.626 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55140 - "GET /api/v1/functions/ HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:24:13.632 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55145 - "POST /api/v1/tasks/active/chats HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:24:13.751 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55145 - "GET /api/v1/chats/?page=2 HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:24:13.840 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55145 - "GET /api/v1/chats/90d317bd-a191-4df9-be20-df886b111c49 HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:24:14.200 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55145 - "GET /api/models HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:24:17.181 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55145 - "POST /api/v1/chats/new HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:24:17.193 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55145 - "GET /api/v1/tools/ HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:24:17.268 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55145 - "GET /api/v1/chats/?page=2 HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:24:17.286 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55145 - "POST /api/v1/chats/244ea17a-29c1-40d7-807e-c02d8ff1200f HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:24:17.293 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55145 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:24:17.354 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55145 - "POST /api/chat/completions HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:24:17.365 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55145 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200
ovms           | [2026-02-19 13:24:17.904][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 0.2%;
ovms           | [2026-02-19 13:24:18.260][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 0.2%;
ovms           | [2026-02-19 13:24:18.616][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 0.2%;
ovms           | [2026-02-19 13:24:18.973][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 0.3%;
ovms           | [2026-02-19 13:24:19.330][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 0.3%;
ovms           | [2026-02-19 13:24:19.688][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 0.4%;
ovms           | [2026-02-19 13:24:20.044][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 0.4%;
ovms           | [2026-02-19 13:24:20.401][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 0.5%;
ovms           | [2026-02-19 13:24:20.768][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 0.6%;
ovms           | [2026-02-19 13:24:21.135][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 0.6%;
ovms           | [2026-02-19 13:24:21.524][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 0.7%;
ovms           | [2026-02-19 13:24:21.887][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 0.7%;
ovms           | [2026-02-19 13:24:21.887][173][llm_executor][info][llm_executor.hpp:90] All requests: 0; Scheduled requests: 0;
open-webui-ov  | 2026-02-19 14:24:21.916 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55145 - "POST /api/chat/completed HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:24:21.928 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55145 - "POST /api/v1/chats/244ea17a-29c1-40d7-807e-c02d8ff1200f HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:24:21.935 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55145 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200
ovms           | [2026-02-19 13:24:22.477][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 1.9%;
ovms           | [2026-02-19 13:24:22.861][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 2.0%;
ovms           | [2026-02-19 13:24:23.241][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 2.1%;
ovms           | [2026-02-19 13:24:23.618][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 2.1%;
ovms           | [2026-02-19 13:24:24.019][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 2.2%;
ovms           | [2026-02-19 13:24:24.427][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 2.2%;
ovms           | [2026-02-19 13:24:24.827][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 2.2%;
ovms           | [2026-02-19 13:24:25.227][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 2.3%;
ovms           | [2026-02-19 13:24:25.630][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 2.3%;
open-webui-ov  | 2026-02-19 14:24:26.001 | INFO     | open_webui.routers.files:upload_file_handler:230 - file.content_type: image/png False
open-webui-ov  | 2026-02-19 14:24:26.008 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55145 - "POST /api/v1/files/?process=false HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:24:26.015 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55145 - "GET /api/v1/files/2d13f197-c7c7-44fe-8557-75e678610491/process/status?stream=true HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:24:26.027 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55145 - "GET /api/v1/files/2d13f197-c7c7-44fe-8557-75e678610491/content HTTP/1.1" 200
ovms           | [2026-02-19 13:24:26.041][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 2.4%;
ovms           | [2026-02-19 13:24:26.450][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 2.5%;
ovms           | [2026-02-19 13:24:26.857][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 2.5%;
ovms           | [2026-02-19 13:24:27.260][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 2.6%;
ovms           | [2026-02-19 13:24:27.669][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 2.6%;
ovms           | [2026-02-19 13:24:28.074][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 2.7%;
ovms           | [2026-02-19 13:24:28.485][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 2.7%;
ovms           | [2026-02-19 13:24:28.899][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 2.7%;
open-webui-ov  | 2026-02-19 14:24:28.902 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55145 - "POST /api/v1/chats/244ea17a-29c1-40d7-807e-c02d8ff1200f HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:24:28.908 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55145 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:24:28.921 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55145 - "POST /api/chat/completions HTTP/1.1" 200
open-webui-ov  | 2026-02-19 14:24:28.929 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55145 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200
ovms           | [2026-02-19 13:24:29.480][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 5.0%;
ovms           | [2026-02-19 13:24:29.905][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 5.1%;
ovms           | [2026-02-19 13:24:30.337][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 5.1%;
ovms           | [2026-02-19 13:24:30.770][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 5.2%;
ovms           | [2026-02-19 13:24:31.200][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 5.3%;
ovms           | [2026-02-19 13:24:31.635][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 5.5%;
ovms           | [2026-02-19 13:24:32.064][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 5.6%;
ovms           | [2026-02-19 13:24:32.496][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 5.6%;
ovms           | [2026-02-19 13:24:32.936][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 5.8%;
ovms           | [2026-02-19 13:24:33.362][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 5.9%;
ovms           | [2026-02-19 13:24:33.785][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 6.0%;
ovms           | [2026-02-19 13:24:34.202][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 6.1%;
ovms           | [2026-02-19 13:24:34.637][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 6.1%;
ovms           | [2026-02-19 13:24:35.073][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 6.3%;
ovms           | [2026-02-19 13:24:35.509][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 6.4%;
ovms           | [2026-02-19 13:24:35.945][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 6.5%;
ovms           | [2026-02-19 13:24:36.391][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 6.6%;
ovms           | [2026-02-19 13:24:36.827][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 6.7%;
ovms           | [2026-02-19 13:24:37.257][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 6.8%;
ovms           | [2026-02-19 13:24:37.676][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 6.9%;
ovms           | [2026-02-19 13:24:38.086][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 7.0%;
ovms           | [2026-02-19 13:24:38.500][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 7.1%;
ovms           | [2026-02-19 13:24:38.913][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 7.2%;
ovms           | [2026-02-19 13:24:39.324][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 7.3%;
ovms           | [2026-02-19 13:24:39.743][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 7.5%;
ovms           | [2026-02-19 13:24:40.177][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 7.6%;
ovms           | [2026-02-19 13:24:40.607][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 7.6%;
ovms           | [2026-02-19 13:24:41.039][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 7.7%;
ovms           | [2026-02-19 13:24:41.474][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 7.8%;
ovms           | [2026-02-19 13:24:41.906][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 8.0%;
ovms           | [2026-02-19 13:24:42.343][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 8.0%;
ovms           | [2026-02-19 13:24:42.773][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 8.1%;
ovms           | [2026-02-19 13:24:43.212][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 8.3%;
ovms           | [2026-02-19 13:24:43.646][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 8.4%;
ovms           | [2026-02-19 13:24:44.078][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 8.5%;
ovms           | [2026-02-19 13:24:44.509][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 8.5%;
ovms           | [2026-02-19 13:24:44.949][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 8.6%;
ovms           | [2026-02-19 13:24:45.391][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 8.8%;
ovms           | [2026-02-19 13:24:45.829][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 8.9%;
ovms           | [2026-02-19 13:24:46.261][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 9.0%;
ovms           | [2026-02-19 13:24:46.722][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 9.1%;
ovms           | [2026-02-19 13:24:47.166][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 9.2%;
ovms           | [2026-02-19 13:24:47.599][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 9.3%;
ovms           | [2026-02-19 13:24:48.014][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 9.4%;
open-webui-ov  | 2026-02-19 14:24:48.393 | WARNING  | open_webui.utils.middleware:response_handler:4336 - Task was cancelled!
open-webui-ov  | 2026-02-19 14:24:48.398 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55342 - "POST /api/tasks/stop/6fc57fec-29ad-48af-a019-0f0ba16b080b HTTP/1.1" 200
ovms           | [2026-02-19 13:24:48.430][173][llm_executor][info][llm_executor.hpp:66] All requests: 2; Scheduled requests: 2; Cache usage 9.5%;
ovms           | [2026-02-19 13:24:48.832][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 5.1%;
ovms           | [2026-02-19 13:24:49.228][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 5.2%;
ovms           | [2026-02-19 13:24:49.627][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 5.2%;
ovms           | [2026-02-19 13:24:50.035][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 5.3%;
ovms           | [2026-02-19 13:24:50.454][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 5.4%;
ovms           | [2026-02-19 13:24:50.871][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 5.4%;
ovms           | [2026-02-19 13:24:51.288][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 5.5%;
ovms           | [2026-02-19 13:24:51.723][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 5.5%;
ovms           | [2026-02-19 13:24:52.133][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 5.6%;
ovms           | [2026-02-19 13:24:52.556][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 5.6%;
ovms           | [2026-02-19 13:24:52.975][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 5.6%;
ovms           | [2026-02-19 13:24:53.396][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 5.7%;
ovms           | [2026-02-19 13:24:53.821][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 5.8%;
ovms           | [2026-02-19 13:24:54.225][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 5.8%;
ovms           | [2026-02-19 13:24:54.637][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 5.9%;
ovms           | [2026-02-19 13:24:55.056][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 5.9%;
ovms           | [2026-02-19 13:24:55.472][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 6.0%;
ovms           | [2026-02-19 13:24:55.888][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 6.1%;
ovms           | [2026-02-19 13:24:56.303][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 6.1%;
ovms           | [2026-02-19 13:24:56.732][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 6.1%;
ovms           | [2026-02-19 13:24:57.143][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 6.2%;
open-webui-ov  | 2026-02-19 14:24:57.468 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.2.22:55352 - "GET /_app/version.json HTTP/1.1" 200
ovms           | [2026-02-19 13:24:57.553][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 6.2%;
ovms           | [2026-02-19 13:24:57.969][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 6.3%;
ovms           | [2026-02-19 13:24:58.383][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 6.3%;
ovms           | [2026-02-19 13:24:58.800][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 6.4%;
ovms           | [2026-02-19 13:24:59.208][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 6.5%;
ovms           | [2026-02-19 13:24:59.609][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 6.5%;
ovms           | [2026-02-19 13:25:00.024][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 6.6%;
ovms           | [2026-02-19 13:25:00.431][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 6.6%;
ovms           | [2026-02-19 13:25:00.834][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 6.6%;
ovms           | [2026-02-19 13:25:01.233][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 6.7%;
ovms           | [2026-02-19 13:25:01.634][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 6.7%;
ovms           | [2026-02-19 13:25:02.062][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 6.8%;
ovms           | [2026-02-19 13:25:02.472][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 6.9%;
ovms           | [2026-02-19 13:25:02.896][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 6.9%;
ovms           | [2026-02-19 13:25:03.337][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 7.0%;
ovms           | [2026-02-19 13:25:03.748][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 7.1%;
ovms           | [2026-02-19 13:25:04.149][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 7.1%;
ovms           | [2026-02-19 13:25:04.552][173][llm_executor][info][llm_executor.hpp:66] All requests: 1; Scheduled requests: 1; Cache usage 7.1%;

Issue submission checklist

• I'm reporting an issue. It's not a question.
• I checked the problem with the documentation, FAQ, open issues, Stack Overflow, etc., and have not found a solution.
• There is reproducer code and related data files such as images, videos, models, etc.

[mlukasze]

it's known issue, sorry if affected your experience with OpenVINO.
we are working on it - we may not be able to introduce fix with the release that is upcoming (due too short window to test the solution properly), but we will do our best for the next one

Ref. CVS-171180

[p-wysocki]

Hi @gaynetdinov! Thank you for reporting the issue. We're currently implementing a fix for the image comprehension. I just tested your image with #34111 and your exact model gemma-3-12b-it-int8-ov and this is the output I got:

Based on the image, this is a screenshot of a conversation with an AI chatbot, likely OpenAI's ChatGPT. 

Here's a breakdown of what's happening:

*   **The User:** The user is interacting with the chatbot, initially asking "How can I assist?". They then ask "how many shutter units do I have".
*   **The Chatbot:** The chatbot responds with "You have 14 shutter units in your home."
*   **Follow-up:** The user then asks the chatbot to "list them".
*   **Chatbot Response:** The chatbot provides a numbered list of 14 shutter units, including names like "Shutter Unit HWR," "Shutter Unit Bathroom," etc.

It appears the user is using the chatbot to manage or get information about their home's shutter units.



It's a demonstration of a conversational AI system capable of understanding and responding to specific requests.

I'm sorry you encountered it, we're working hard on including this fix with OpenVINO 26.1, but as @mlukasze mentioned, we can't promise it.

If you wish feel free to build OpenVINO with my branch and then it should work, but please keep in mind that this feature is not approved yet and may have some bugs/may change later.