Skip to content

[Bug] Tools cannot be used with structured output in Gemini #2225

Open
@kidrah-bhq

Description

Description

Whenever a tool is added with structured output in Gemini, it throws an error

Steps to Reproduce

from agno.agent import Agent
from agno.models.google import Gemini
from pydantic import BaseModel, Field
from agno.tools.duckduckgo import DuckDuckGoTools


class ChatResponseModel(BaseModel):
    response: str = Field(description="Direct response to the answer")

structured_output_agent = Agent(
    model=Gemini(),
    structured_outputs=True,
    response_model=ChatResponseModel,
    tools=[DuckDuckGoTools()],
    show_tool_calls=True,
)

structured_output_agent.print_response("Hi")

Agent Configuration (if applicable)

Provide relevant agent configuration.

Expected Behavior

What did you expect to happen?
It should provide the output as per the provided structure

Actual Behavior

What actually happened instead?
Received an error
If I remove the tool or the response_model, it works fine

Screenshots or Logs (if applicable)

Include any relevant screenshots or error logs that demonstrate the issue.
Error from Gemini API: 400 INVALID_ARGUMENT. {'error': {'code': 400, 'message': "For controlled generation of only function calls (forced function calling), please set 'tool_config.function_calling_config.mode' field to ANY instead of populating 'response_mime_type' and 'response_schema' fields. For more
details, see: https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/function-calling#tool-config", 'status': 'INVALID_ARGUMENT'}}

Environment

  • OS: (e.g. macOS, Windows 11)
  • Browser (if relevant): (e.g. Chrome 108, Firefox 107)
  • Agno Version: (e.g. v1.0.0) 1.1.6
  • External Dependency Versions: (e.g., yfinance 0.2.52)
  • Additional Environment Details: (e.g., Python 3.10)

Possible Solutions (optional)

Suggest any ideas you might have to fix or address the issue.

Additional Context

Add any other context or details about the problem here.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions