Skip to content

Bug: Wrong number of tool calls using evals-cli #40

@beaufortfrancois

Description

@beaufortfrancois

Try I'd like a small Pesto pizza for https://googlechromelabs.github.io/webmcp-tools/demos/pizza-maker/ using the evals-cli tool.

Vercel backend gives me the following tool calls:

  1. call: {"type":"tool-call","toolCallId":"Cdd9DKnzVP2PiLC3","toolName":"set_pizza_style","input":{"style":"Pesto"},"providerMetadata":{"google":{"thoughtSignature":"..."}}}
  2. call: {"type":"tool-call","toolCallId":"oJlgIdNgaY3BteLG","toolName":"set_pizza_style","input":{"style":"Pesto"},"providerMetadata":{"google":{"thoughtSignature":"..."}}}
  3. call: {"type":"tool-call","toolCallId":"EdJOFFtzL97582Hv","toolName":"set_pizza_size","input":{"size":"Small"}}

Gemini backend only returns the first one:

'{"parts":[{"functionCall":{"name":"set_pizza_style","args":{"style":"Pesto"}},"thoughtSignature":"..."}],"role":"model"}'

Using the Model Context Inspector extension, I get the 2 appropriate tool calls:

User prompt: "I'd like a small Pesto pizza"
AI calling tool "set_pizza_size" with {"size":"Small"}
Tool "set_pizza_size" result: Set pizza size to Small.
AI calling tool "set_pizza_style" with {"style":"Pesto"}
Tool "set_pizza_style" result: Changed pizza style to Pesto
AI result: I have a small pesto pizza for you!

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions