Qwen 2.5 based model does not accept Images

[BW-Projects]

Hello,
when running SGLang with "Qwen/Qwen2.5-VL-7B-Instruct", it works without a problem sending images as messages, however when using RolmOCR, which is a finetune of Qwen2.5-VL (https://huggingface.co/reducto/RolmOCR) I can't send images or more precise, they are ignored as the image_data is None in the request when compared to the original qwen2.5 requests.
The request is the same

response = await self.client.chat.completions.create(
                temperature=temperature,
                model=model,
                messages=[{"role": "user", "content": [
                        {"type": "text", "text": MARKDOWN_OCR_PROMPT_no_title},
                        {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{base64_image}"}}
                            ]
                        }
                    ],
                max_completion_tokens=max_tokens,
                timeout=timeout
            )

Does anybody knows whats going on? The architecture is set correctly in the config.json of the RolmOCR.

[JustinTong0323]

Set chat-template to qwen2-vl, unless server could not recognize multi-modal data in messages.

[JustinTong0323]

Launch server with "--chat-template=qwen2-vl "

[jingyang-virtue]

What's the general rule here? I'm having similar issue with a finetuned gemma3-4b-it model.

EDIT: I found the necessary chat template I need to specify in https://docs.sglang.ai/supported_models/multimodal_language_models.html. But I'm still curious why this is needed.