Skip to content
Merged
21 changes: 19 additions & 2 deletions docs-website/docs/pipeline-components/agents-1/agent.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -38,16 +38,33 @@ The `Agent` returns a dictionary containing:

- `messages`: the full conversation history,
- `last_message`: the final `ChatMessage` from the agent,
- `step_count`: the number of steps the agent ran,
- `token_usage`: aggregated token usage summed across every LLM call in the run,
- `tool_call_counts`: how many times each tool was invoked, keyed by tool name,
- Additional dynamic keys based on `state_schema`.

### Run Metadata

The `step_count`, `token_usage`, and `tool_call_counts` outputs are populated automatically during a run. They are added to the agent's `state_schema` behind the scenes, so tools registered with `inputs_from_state` can read them mid-run. They are outputs only — they cannot be passed as inputs to `run()` or `run_async()`, and using them as keys in your own `state_schema` raises a `ValueError`. See [State](./state.mdx#schema-definition) for details.

```python
response = agent.run(messages=[ChatMessage.from_user("What is 7 * (4 + 2)?")])

print(response["step_count"]) # e.g. 2
print(
response["token_usage"],
) # e.g. {"prompt_tokens": 512, "completion_tokens": 86, ...}
print(response["tool_call_counts"]) # e.g. {"calculator": 1}
```

## Parameters

`chat_generator` is the only mandatory parameter — an instance of a Chat Generator that supports tools. All other parameters are optional.

- `tools`: A list of tool or toolset instances the agent can call. Supported types: [`Tool`](../../tools/tool.mdx), [`ComponentTool`](../../tools/componenttool.mdx), [`PipelineTool`](../../tools/pipelinetool.mdx), [`MCPTool`](../../tools/mcptool.mdx), [`Toolset`](../../tools/toolset.mdx), [`MCPToolset`](../../tools/mcptoolset.mdx), [`SearchableToolset`](../../tools/searchabletoolset.mdx).
- `tools`: A list of tool or toolset instances the agent can call. Supported types: [`Tool`](../../tools/tool.mdx), [`ComponentTool`](../../tools/componenttool.mdx), [`PipelineTool`](../../tools/pipelinetool.mdx), [`MCPTool`](../../tools/mcptool.mdx), [`Toolset`](../../tools/toolset.mdx), [`MCPToolset`](../../tools/mcptoolset.mdx), [`SearchableToolset`](../../tools/searchabletoolset.mdx). Tool names must be unique; duplicate names are detected at the start of each agent step, before the chat generator is called.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The addition could also be left out.

- `system_prompt`: A plain string or Jinja2 template used as the system message for every run. If the template contains Jinja2 variables, those variables become additional inputs to `run()`.
- `user_prompt`: A Jinja2 template appended to the user-provided messages on each run. Template variables become additional inputs to `run()`. Use `required_variables` to enforce which variables must be provided.
- `exit_conditions`: List of conditions that cause the agent to stop. Use `”text”` to stop when the LLM replies without a tool call, or a tool name to stop once that tool has been executed. Defaults to `[“text”]`.
- `exit_conditions`: List of conditions that cause the agent to stop. Use `”text”` to stop when the LLM replies without a tool call, or a tool name to stop once that tool has been executed. Defaults to `[“text”]`. Exit conditions are evaluated at runtime rather than validated at initialization, so a condition can name a tool that is only loaded later — for example, a tool passed at runtime via `run(tools=...)` or one discovered by a [`SearchableToolset`](../../tools/searchabletoolset.mdx).

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The addition could also be left out.

- `state_schema`: Defines the agent's runtime state — a dict mapping key names to type configs (e.g. `{“docs”: {“type”: list[Document]}}`). Tools can read from and write to state keys via `inputs_from_state` and `outputs_to_state`. See [State](./state.mdx) for full details.
- `streaming_callback`: A callback invoked for each streamed token. Use the built-in `print_streaming_chunk` for console output.
- `max_agent_steps`: Maximum number of LLM + tool call iterations before the agent stops. Defaults to `100`.
Expand Down
9 changes: 9 additions & 0 deletions docs-website/docs/pipeline-components/agents-1/state.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,15 @@ The schema defines what data can be stored and how values are updated. Each sche

If you don't specify a handler, State automatically assigns a default based on the type.

:::info Reserved keys
The `Agent` manages some state keys itself and rejects them in a user-provided `state_schema` with a `ValueError`:

- The run-metadata keys `step_count`, `token_usage`, and `tool_call_counts`, which the Agent populates automatically during a run: tools can read them mid-run via `inputs_from_state`, and they are returned in the result dictionary.
- The hook-facing keys `continue_run` (set by an `on_exit` hook to keep the Agent running), `tools` (the tools available in the current step, for hooks to inspect), and `hook_context` (the request-scoped resources passed to `Agent.run(hook_context={...})`).

If one of your state keys clashes, rename it (for example, `my_token_usage`).
:::

### Default Handlers

State provides two built-in merge behaviors (importable from `haystack.components.agents.state`):
Expand Down
1 change: 1 addition & 0 deletions docs-website/docs/pipeline-components/generators.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ Generators are responsible for generating text after you give them a prompt. The
| [LlamaStackChatGenerator](generators/llamastackchatgenerator.mdx) | Enables chat completions using an LLM model made available via Llama Stack server | ✅ |
| [MetaLlamaChatGenerator](generators/metallamachatgenerator.mdx) | Enables chat completion with any model hosted available with Meta Llama API. | ✅ |
| [MistralChatGenerator](generators/mistralchatgenerator.mdx) | Enables chat completion using Mistral's text generation models. | ✅ |
| [MockChatGenerator](generators/mockchatgenerator.mdx) | Returns predefined responses without calling any API — a deterministic, zero-cost stand-in for real Chat Generators in tests and prototypes. | ✅ |
| [NvidiaChatGenerator](generators/nvidiachatgenerator.mdx) | Enables chat completion using Nvidia-hosted models. | ✅ |
| [NvidiaGenerator](generators/nvidiagenerator.mdx) | Provides an interface for generating text using LLMs self-hosted with NVIDIA NIM or models hosted on the NVIDIA API catalog. | ❌ |
| [OllamaChatGenerator](generators/ollamachatgenerator.mdx) | Enables chat completion using an LLM running on Ollama. | ✅ |
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
---
title: "MockChatGenerator"
id: mockchatgenerator
slug: "/mockchatgenerator"
description: "A Chat Generator that returns predefined responses without calling any API, for tests and quick prototypes."
---

# MockChatGenerator

A Chat Generator that returns predefined responses without calling any API, for tests and quick prototypes.

<div className="key-value-table">

| | |
| --- | --- |
| **Most common position in a pipeline** | In place of a real Chat Generator, in tests and prototypes |
| **Mandatory init variables** | None |
| **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects |
| **Output variables** | `replies`: A list of generated `ChatMessage` objects |
| **API reference** | [Generators](/reference/generators-api) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/generators/chat/mock.py |
| **Package name** | `haystack-ai` |

</div>

## Overview

`MockChatGenerator` is a deterministic, zero-cost drop-in replacement for real Chat Generators such as `OpenAIChatGenerator`. It implements `run`, `run_async`, streaming callbacks, and serialization but never contacts an external service, which makes it ideal for unit tests, smoke tests, and quick prototypes.

The response is selected based on how the component is configured:

- **Fixed response**: Pass a single string or `ChatMessage` via `responses`. The same reply is returned on every call. A `ChatMessage` passed as a response must have the `assistant` role.
- **Cycling responses**: Pass a list of strings and/or `ChatMessage` objects via `responses`. Each call returns the next item, wrapping around to the start once the list is exhausted. This is useful to drive multi-step flows such as Agents, where the first call returns a tool call and a later call returns the final answer.
- **Dynamic response**: Pass a `response_fn` callable that receives the input messages and returns the reply as a string or an assistant `ChatMessage`. Use this when the reply should depend on the input. To support serialization, pass a named function.
- **Echo (default)**: With no configuration, the component echoes back the text of the last message that has text content, so it is usable out of the box.

`responses` and `response_fn` are mutually exclusive.

Further optional parameters:

- `model`: The model name reported in the response metadata. Defaults to `"mock-model"`.
- `meta`: Additional metadata merged into the `meta` of every returned `ChatMessage`. A per-response `ChatMessage`'s own metadata takes precedence.
- `streaming_callback`: An optional callback invoked with `StreamingChunk` objects reconstructed from the predefined response. It lets the mock exercise streaming code paths without a real model.

## Usage

### On its own

```python
from haystack.components.generators.chat import MockChatGenerator
from haystack.dataclasses import ChatMessage

# Fixed response
generator = MockChatGenerator(responses="Hello, this is a mock response.")
result = generator.run([ChatMessage.from_user("Hi!")])
print(result["replies"][0].text) # "Hello, this is a mock response."

# Echo mode (default): returns the last message with text content
generator = MockChatGenerator()
result = generator.run([ChatMessage.from_user("Repeat after me")])
print(result["replies"][0].text) # "Repeat after me"
```

### Driving an Agent

Pass `ChatMessage` objects (rather than plain strings) to return tool calls or reasoning content. With cycling responses, you can script a full agent loop without a real model:

```python
from haystack.components.agents import Agent
from haystack.components.generators.chat import MockChatGenerator
from haystack.dataclasses import ChatMessage, ToolCall
from haystack.tools import tool


@tool
def search(query: str) -> str:
"""Search for information."""
return f"Results for: {query}"


generator = MockChatGenerator(
responses=[
ChatMessage.from_assistant(
tool_calls=[ToolCall(tool_name="search", arguments={"query": "Haystack"})],
),
"Here is the final answer.",
],
)

agent = Agent(chat_generator=generator, tools=[search])
result = agent.run(messages=[ChatMessage.from_user("Tell me about Haystack")])
print(result["last_message"].text) # "Here is the final answer."
```

### Input-dependent responses

```python
from haystack.components.generators.chat import MockChatGenerator
from haystack.dataclasses import ChatMessage


def shout_back(messages: list[ChatMessage]) -> str:
return messages[-1].text.upper()


generator = MockChatGenerator(response_fn=shout_back)
result = generator.run([ChatMessage.from_user("hello")])
print(result["replies"][0].text) # "HELLO"
```
8 changes: 8 additions & 0 deletions docs-website/docs/tools/searchabletoolset.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,14 @@ subsequent iterations.
`SearchableToolset` does not support adding new tools after initialization or merging with other toolsets. Use `catalog` to provide all tools upfront.
:::

### Warm-up

`SearchableToolset` builds its search index during `warm_up()`. When used with an [`Agent`](../pipeline-components/agents-1/agent.mdx), constructing the Agent does not trigger this — warm-up happens when you call `Agent.warm_up()` or automatically at run time.

All tool names in the catalog must be unique: `warm_up()` raises a `ValueError` if the catalog contains tools with duplicate names, since a search hit could otherwise resolve to the wrong tool.

The Agent evaluates its `exit_conditions` at runtime, so an exit condition can name any tool in the catalog, even one the agent has not discovered yet.

## Usage

### Basic usage with an Agent
Expand Down
1 change: 1 addition & 0 deletions docs-website/sidebars.js
Original file line number Diff line number Diff line change
Expand Up @@ -436,6 +436,7 @@ export default {
'pipeline-components/generators/llamastackchatgenerator',
'pipeline-components/generators/metallamachatgenerator',
'pipeline-components/generators/mistralchatgenerator',
'pipeline-components/generators/mockchatgenerator',
'pipeline-components/generators/nvidiachatgenerator',
'pipeline-components/generators/nvidiagenerator',
'pipeline-components/generators/ollamachatgenerator',
Expand Down