Deepseek-V3 with vision

[yuanpinz]

How was Deepseek-V3 with vision implemented?

[Saint-lsy]

Thanks for your attention to our work. Deepseek-V3 is evaluated by api calling. You can refer to the Deepseek official API docs https://api-docs.deepseek.com/

[yuanpinz]

Thanks for your attention to our work. Deepseek-V3 is evaluated by api calling. You can refer to the Deepseek official API docs https://api-docs.deepseek.com/

Would you provide more details? As far as I know, Deepseek-V3 is a LLM and do not support multimodal input.

[Saint-lsy]

In our implementation, we use a third-party API (https://chatanywhere.apifox.cn/doc-2694962). You can try this api with multimodal input.

[Lapis0x0]

I am the administrator of this API site. To my knowledge, our DeepSeek v3 still lacks vision capabilities. Did you use any specific engineering solution for multimodal concatenation?

[Lapis0x0]

Additionally, grok3 does not appear to be a vlm either. Please refer to: https://docs.x.ai/developers/models

[Saint-lsy]

@Lapis0x0 @yuanpinz Thanks for your reply! In our implementation, we use this code for all proprietary models. The requests all succeeded without any errors, and we didn't realize earlier that some of these models actually don't support multimodal (image) inputs.

def build_message_with_image_and_text(text_prompt: str, image_b64: str):
    return [
        {
            "role": "user",
            "content": [
                {"type": "text", "text": text_prompt},
                {
                    "type": "image_url",
                    "image_url": {
                        "url": f"data:image/jpeg;base64,{image_b64}"
                    }
                },
            ],
        }
    ]

def get_model_response(messages):
    response = client.chat.completions.create(
        model="grok-3" #or deepseek v3,
        messages=messages,
    )
    return response.choices[0].message.content
......
messages = [
    {"role": "system", "content": "You are a helpful assistant"},
] + build_message_with_image_and_text(text_prompt, image_b64)
answer = get_model_response(messages)

We've been using the same multimodal message format (text + base64 image) for all proprietary models, and surprisingly, even models that don't officially support vision inputs returned successful responses without errors.