[Bug]: Non-streaming tool calling broken with gemini-3-pro-preview

[CapCap]

Bug Description

When calling run on a FunctionAgent without streaming, no response is returned, and no tool calls are performed. The same code, when run with gemini-2.5-pro, works just fine.

I have added working code that uses lower-level tools (llm.achat_with_tools) which works to help compare

I also tested with streaming and it does work, hence I think it might be similar or related to the changes in #20207

Version

"llama-index>=0.14.8", "llama-index-core>=0.14.8", "llama-index-llms-google-genai>=0.7.3"
(all latest versions at time of publishing)

Steps to Reproduce

This works:

async def works():
    llm = GoogleGenAI(
        api_key=GOOGLE_AISTUDIO_API_KEY,
        model="gemini-3-pro-preview",
        is_function_calling_model=True,
    )

    tool = FunctionTool.from_defaults(fn=multiply)

    chat_history = [
        ChatMessage(role="system", content="Use the multiply tool when asked to multiply numbers."),
        ChatMessage(role="user", content='What is 5 times 7?')
    ]

    response = await llm.achat_with_tools([tool], chat_history=chat_history)
    print(f"Response: {response}")

This does not:

    llm = GoogleGenAI(
        api_key=GOOGLE_AISTUDIO_API_KEY,
        model="gemini-3-pro-preview",
        is_function_calling_model=True,
    )

    tool = FunctionTool.from_defaults(fn=multiply)

    agent = FunctionAgent(
        name="TestAgent",
        description="A test agent",
        tools=[tool],
        llm=llm,
    )

    response = await agent.run(
        chat_history=[
            ChatMessage(role="system", content="Use the multiply tool when asked to multiply numbers."),
        ],
        user_msg=ChatMessage(role=MessageRole.USER, content="What is 5 times 7?"),
    )

While with streaming it does work

    llm = GoogleGenAI(
        api_key=GOOGLE_AISTUDIO_API_KEY,
        model="gemini-3-pro-preview",
        is_function_calling_model=True,
    )

    tool = FunctionTool.from_defaults(fn=multiply)

    agent = FunctionAgent(
        name="TestAgent",
        description="A test agent",
        tools=[tool],
        llm=llm,
        streaming=True,
    )

    handler = agent.run(
        chat_history=[
            ChatMessage(role="system", content="Use the multiply tool when asked to multiply numbers."),
        ],
        user_msg=ChatMessage(role=MessageRole.USER, content="What is 5 times 7?"),
    )
    async for event in handler.stream_events():
        print(event)

[dosubot]

This issue is caused by how FunctionAgent handles tool calling: even in non-streaming mode, it uses the LLM's async streaming interface (astream_chat_with_tools). For gemini-3-pro-preview, this can result in no response or tool calls unless streaming is enabled, because the model's API may not reliably emit events unless a streaming session is active. This is why your lower-level call to llm.achat_with_tools works (it uses the non-streaming path), and why enabling streaming on the agent resolves the problem.

Recent changes in PR #20207 updated Gemini tool calling logic, but the agent's architecture still relies on streaming internally, even if you don't request streaming. This difference in code paths explains the behavior you're seeing.

As a workaround, you can either use streaming mode with FunctionAgent, or stick to direct calls to llm.achat_with_tools for non-streaming scenarios with gemini-3-pro-preview. If you need non-streaming agent orchestration, you may need to adapt the agent to use the non-streaming LLM interface for Gemini models.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[CapCap]

More info: explicitly setting streaming=False, fixes it