-
Notifications
You must be signed in to change notification settings - Fork 3k
docs: document Agent run metadata, runtime exit_conditions, SearchableToolset notes, and MockChatGenerator #11873
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 3 commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
8a6a343
docs: document Agent run metadata, runtime exit_conditions, and MockC…
julian-risch 9eb4f03
docs: absorb SearchableToolset duplicate-name note and extend reserve…
julian-risch 881f4e9
Refine overview of MockChatGenerator
julian-risch ba75479
Remove "e.g." from agent.mdx example code
julian-risch fa9b5f4
Merge branch 'main' into docs/v3-simple-doc-updates-3
julian-risch 03479d9
Merge branch 'main' into docs/v3-simple-doc-updates-3
davidsbatista 6e3212c
Merge branch 'main' into docs/v3-simple-doc-updates-3
davidsbatista File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -38,16 +38,33 @@ The `Agent` returns a dictionary containing: | |
|
|
||
| - `messages`: the full conversation history, | ||
| - `last_message`: the final `ChatMessage` from the agent, | ||
| - `step_count`: the number of steps the agent ran, | ||
| - `token_usage`: aggregated token usage summed across every LLM call in the run, | ||
| - `tool_call_counts`: how many times each tool was invoked, keyed by tool name, | ||
| - Additional dynamic keys based on `state_schema`. | ||
|
|
||
| ### Run Metadata | ||
|
|
||
| The `step_count`, `token_usage`, and `tool_call_counts` outputs are populated automatically during a run. They are added to the agent's `state_schema` behind the scenes, so tools registered with `inputs_from_state` can read them mid-run. They are outputs only — they cannot be passed as inputs to `run()` or `run_async()`, and using them as keys in your own `state_schema` raises a `ValueError`. See [State](./state.mdx#schema-definition) for details. | ||
|
|
||
| ```python | ||
| response = agent.run(messages=[ChatMessage.from_user("What is 7 * (4 + 2)?")]) | ||
|
|
||
| print(response["step_count"]) # e.g. 2 | ||
| print( | ||
| response["token_usage"], | ||
| ) # e.g. {"prompt_tokens": 512, "completion_tokens": 86, ...} | ||
| print(response["tool_call_counts"]) # e.g. {"calculator": 1} | ||
| ``` | ||
|
|
||
| ## Parameters | ||
|
|
||
| `chat_generator` is the only mandatory parameter — an instance of a Chat Generator that supports tools. All other parameters are optional. | ||
|
|
||
| - `tools`: A list of tool or toolset instances the agent can call. Supported types: [`Tool`](../../tools/tool.mdx), [`ComponentTool`](../../tools/componenttool.mdx), [`PipelineTool`](../../tools/pipelinetool.mdx), [`MCPTool`](../../tools/mcptool.mdx), [`Toolset`](../../tools/toolset.mdx), [`MCPToolset`](../../tools/mcptoolset.mdx), [`SearchableToolset`](../../tools/searchabletoolset.mdx). | ||
| - `tools`: A list of tool or toolset instances the agent can call. Supported types: [`Tool`](../../tools/tool.mdx), [`ComponentTool`](../../tools/componenttool.mdx), [`PipelineTool`](../../tools/pipelinetool.mdx), [`MCPTool`](../../tools/mcptool.mdx), [`Toolset`](../../tools/toolset.mdx), [`MCPToolset`](../../tools/mcptoolset.mdx), [`SearchableToolset`](../../tools/searchabletoolset.mdx). Tool names must be unique; duplicate names are detected at the start of each agent step, before the chat generator is called. | ||
| - `system_prompt`: A plain string or Jinja2 template used as the system message for every run. If the template contains Jinja2 variables, those variables become additional inputs to `run()`. | ||
| - `user_prompt`: A Jinja2 template appended to the user-provided messages on each run. Template variables become additional inputs to `run()`. Use `required_variables` to enforce which variables must be provided. | ||
| - `exit_conditions`: List of conditions that cause the agent to stop. Use `”text”` to stop when the LLM replies without a tool call, or a tool name to stop once that tool has been executed. Defaults to `[“text”]`. | ||
| - `exit_conditions`: List of conditions that cause the agent to stop. Use `”text”` to stop when the LLM replies without a tool call, or a tool name to stop once that tool has been executed. Defaults to `[“text”]`. Exit conditions are evaluated at runtime rather than validated at initialization, so a condition can name a tool that is only loaded later — for example, a tool passed at runtime via `run(tools=...)` or one discovered by a [`SearchableToolset`](../../tools/searchabletoolset.mdx). | ||
|
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The addition could also be left out. |
||
| - `state_schema`: Defines the agent's runtime state — a dict mapping key names to type configs (e.g. `{“docs”: {“type”: list[Document]}}`). Tools can read from and write to state keys via `inputs_from_state` and `outputs_to_state`. See [State](./state.mdx) for full details. | ||
| - `streaming_callback`: A callback invoked for each streamed token. Use the built-in `print_streaming_chunk` for console output. | ||
| - `max_agent_steps`: Maximum number of LLM + tool call iterations before the agent stops. Defaults to `100`. | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
109 changes: 109 additions & 0 deletions
109
docs-website/docs/pipeline-components/generators/mockchatgenerator.mdx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,109 @@ | ||
| --- | ||
| title: "MockChatGenerator" | ||
| id: mockchatgenerator | ||
| slug: "/mockchatgenerator" | ||
| description: "A Chat Generator that returns predefined responses without calling any API, for tests and quick prototypes." | ||
| --- | ||
|
|
||
| # MockChatGenerator | ||
|
|
||
| A Chat Generator that returns predefined responses without calling any API, for tests and quick prototypes. | ||
|
|
||
| <div className="key-value-table"> | ||
|
|
||
| | | | | ||
| | --- | --- | | ||
| | **Most common position in a pipeline** | In place of a real Chat Generator, in tests and prototypes | | ||
| | **Mandatory init variables** | None | | ||
| | **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects | | ||
| | **Output variables** | `replies`: A list of generated `ChatMessage` objects | | ||
| | **API reference** | [Generators](/reference/generators-api) | | ||
| | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/generators/chat/mock.py | | ||
| | **Package name** | `haystack-ai` | | ||
|
|
||
| </div> | ||
|
|
||
| ## Overview | ||
|
|
||
| `MockChatGenerator` is a deterministic, zero-cost drop-in replacement for real Chat Generators such as `OpenAIChatGenerator`. It implements `run`, `run_async`, streaming callbacks, and serialization but never contacts an external service, which makes it ideal for unit tests, smoke tests, and quick prototypes. | ||
|
|
||
| The response is selected based on how the component is configured: | ||
|
|
||
| - **Fixed response**: Pass a single string or `ChatMessage` via `responses`. The same reply is returned on every call. A `ChatMessage` passed as a response must have the `assistant` role. | ||
| - **Cycling responses**: Pass a list of strings and/or `ChatMessage` objects via `responses`. Each call returns the next item, wrapping around to the start once the list is exhausted. This is useful to drive multi-step flows such as Agents, where the first call returns a tool call and a later call returns the final answer. | ||
| - **Dynamic response**: Pass a `response_fn` callable that receives the input messages and returns the reply as a string or an assistant `ChatMessage`. Use this when the reply should depend on the input. To support serialization, pass a named function. | ||
| - **Echo (default)**: With no configuration, the component echoes back the text of the last message that has text content, so it is usable out of the box. | ||
|
|
||
| `responses` and `response_fn` are mutually exclusive. | ||
|
|
||
| Further optional parameters: | ||
|
|
||
| - `model`: The model name reported in the response metadata. Defaults to `"mock-model"`. | ||
| - `meta`: Additional metadata merged into the `meta` of every returned `ChatMessage`. A per-response `ChatMessage`'s own metadata takes precedence. | ||
| - `streaming_callback`: An optional callback invoked with `StreamingChunk` objects reconstructed from the predefined response. It lets the mock exercise streaming code paths without a real model. | ||
|
|
||
| ## Usage | ||
|
|
||
| ### On its own | ||
|
|
||
| ```python | ||
| from haystack.components.generators.chat import MockChatGenerator | ||
| from haystack.dataclasses import ChatMessage | ||
|
|
||
| # Fixed response | ||
| generator = MockChatGenerator(responses="Hello, this is a mock response.") | ||
| result = generator.run([ChatMessage.from_user("Hi!")]) | ||
| print(result["replies"][0].text) # "Hello, this is a mock response." | ||
|
|
||
| # Echo mode (default): returns the last message with text content | ||
| generator = MockChatGenerator() | ||
| result = generator.run([ChatMessage.from_user("Repeat after me")]) | ||
| print(result["replies"][0].text) # "Repeat after me" | ||
| ``` | ||
|
|
||
| ### Driving an Agent | ||
|
|
||
| Pass `ChatMessage` objects (rather than plain strings) to return tool calls or reasoning content. With cycling responses, you can script a full agent loop without a real model: | ||
|
|
||
| ```python | ||
| from haystack.components.agents import Agent | ||
| from haystack.components.generators.chat import MockChatGenerator | ||
| from haystack.dataclasses import ChatMessage, ToolCall | ||
| from haystack.tools import tool | ||
|
|
||
|
|
||
| @tool | ||
| def search(query: str) -> str: | ||
| """Search for information.""" | ||
| return f"Results for: {query}" | ||
|
|
||
|
|
||
| generator = MockChatGenerator( | ||
| responses=[ | ||
| ChatMessage.from_assistant( | ||
| tool_calls=[ToolCall(tool_name="search", arguments={"query": "Haystack"})], | ||
| ), | ||
| "Here is the final answer.", | ||
| ], | ||
| ) | ||
|
|
||
| agent = Agent(chat_generator=generator, tools=[search]) | ||
| result = agent.run(messages=[ChatMessage.from_user("Tell me about Haystack")]) | ||
| print(result["last_message"].text) # "Here is the final answer." | ||
| ``` | ||
|
|
||
| ### Input-dependent responses | ||
|
|
||
| ```python | ||
| from haystack.components.generators.chat import MockChatGenerator | ||
| from haystack.dataclasses import ChatMessage | ||
|
|
||
|
|
||
| def shout_back(messages: list[ChatMessage]) -> str: | ||
| return messages[-1].text.upper() | ||
|
|
||
|
|
||
| generator = MockChatGenerator(response_fn=shout_back) | ||
| result = generator.run([ChatMessage.from_user("hello")]) | ||
| print(result["replies"][0].text) # "HELLO" | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The addition could also be left out.