Vertex AI Gemini: The ChatClient/LLM does not make use of the provided tool/s

[thomasflad]

Bug description
The ChatClient/LLM does not make use of the provided tool/s when using Gemini

Environment

• Spring AI 1.0.0-M8
• Spring Boot 3.4.5
• Gemini 2.0 Flash

Steps to reproduce

• Define a tool with name and description (@tool)
• Use VertexAiGeminiChatModel for ChatClient
• Call prompt with tools() and toolContext()
• User prompt with Image

Expected behavior
When running the same ChatClient via OpenAI it makes use of the tool as expected. Gemini seems to ignore the provided tool. I'm not sure if something is wrong on my side. ("vertex.ai.gemini.chat.options.tool-names" config is provided).