Skip to content

[Feature Request] Support stream + tool call + structured output #3782

@Wendong-Fan

Description

@Wendong-Fan

Required prerequisites

Motivation

when using ChatAgent in streaming mode with both response_format (structured output) and tools enabled, the agent
fails to return proper tool calls and structured content. The same configuration works correctly in non-streaming mode.

code to reproduce:

from pydantic import BaseModel, Field

from camel.agents import ChatAgent
from camel.models import ModelFactory
from camel.toolkits import MathToolkit
from camel.types import ModelPlatformType, ModelType


class Result(BaseModel):
    sum_result: str = Field(description="Result of the addition")
    product_result: str = Field(description="Result of the multiplication")
    division_result: str = Field(description="Result of the division")
    capital_result: str = Field(description="Result of the capital search")

streaming_model = ModelFactory.create(
    model_platform=ModelPlatformType.DEFAULT,
    model_type=ModelType.DEFAULT,
    model_config_dict={
        "stream": True,
        "stream_options": {"include_usage": True},
    },
)

agent = ChatAgent(
    system_message="You are a helpful assistant. ",
    model=streaming_model,
    tools=MathToolkit().get_tools(),
    stream_accumulate=False,  # Delta mode
)

streaming_response = agent.step(
    "Calculate: 1) 123.45 + 678.90  2) 100 * 3.14159  3) 1000 / 7, "
    "also search what is the capital of Germany",
    response_format=Result,
)

# Collect content from streaming chunks
content_parts = []
for chunk in streaming_response:
    if chunk.msgs[0].content:
        content_parts.append(chunk.msgs[0].content)

# Verify we got some structured response
full_content = "".join(content_parts)

print(full_content)

Suggested Fix:

  1. Preserve used_prompt_formatting and original_response_format in _stream method
  2. Apply _apply_prompt_based_parsing after stream completion when used_prompt_formatting is True
  3. Review _stream_response and _process_stream_chunks_with_accumulator to ensure proper handling of tool calls followed
    by structured output generation

also need to validate other model platforms including gemini, anthropic etc..

Solution

No response

Alternatives

No response

Additional context

No response

Metadata

Metadata

Assignees

Labels

P0Task with high level priority

Type

No type

Projects

Status

No status

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions