v2.14.1

Bug Fixes

• Fixed a mypy issue in the OpenAIChatGenerator and its handling of stream responses. This issue only occurs with mypy >=1.16.0.
• Fix type comparison in schema validation by replacing is not with != when checking the type List[ChatMessage]. This prevents false mismatches due to Python's is operator comparing object identity instead of equality.

v2.14.1-rc1

v2.14.1-rc1

v2.14.0

⭐️ Highlights

Enhancements for Complex Agentic Systems

We've improved agent workflows with better message handling and streaming support. Agent component now returns a last_message output for quick access to the final message, and can use a streaming_callback to emit tool results in real time. You can use the updated print_streaming_chunk or write your own callback function to enable ToolCall details during streaming.

from haystack.components.websearch import SerperDevWebSearch
from haystack.components.agents import Agent
from haystack.components.generators.utils import print_streaming_chunk
from haystack.tools import tool, ComponentTool
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage

web_search = ComponentTool(name="web_search", component=SerperDevWebSearch(top_k=5))
wiki_search = ComponentTool(name="wiki_search", component=SerperDevWebSearch(top_k=5, allowed_domains=["https://www.wikipedia.org/"]))

research_agent = Agent(
    chat_generator=OpenAIChatGenerator(model="gpt-4o-mini"),
    system_prompt="""
    You are a research agent that can find information on web or specifically on wikipedia. 
    Use wiki_search tool if you need facts and use web_search tool for latest news on topics.
    Use one tool at a time, use the other tool if the retrieved information is not enough.
    Summarize the retrieved information before returning response to the user.
    """,
    tools=[web_search, wiki_search],
    streaming_callback=print_streaming_chunk
)

result = research_agent.run(messages=[ChatMessage.from_user("Can you tell me about Florence Nightingale's life?")])

Enabling streaming with print_streaming_chunk function looks like this:

[TOOL CALL]
Tool: wiki_search 
Arguments: {"query":"Florence Nightingale"}

[TOOL RESULT]
{'documents': [{'title': 'List of schools in Nottinghamshire', 'link': 'https://www.wikipedia.org/wiki/List_of_schools_in_Nottinghamshire', 'position': 1, 'id': 'a6d0fe00f1e0cd06324f80fb926ba647878fb7bee8182de59a932500aeb54a5b', 'content': 'The Florence Nightingale Academy, Eastwood; The Flying High Academy, Mansfield; Forest Glade Primary School, Sutton-in-Ashfield; Forest Town Primary School ...', 'blob': None, 'score': None, 'embedding': None, 'sparse_embedding': None}], 'links': ['https://www.wikipedia.org/wiki/List_of_schools_in_Nottinghamshire']}
...

Print the last_message

print("Final Answer:", result["last_message"].text)
>>> Final Answer: Florence Nightingale (1820-1910) was a pioneering figure in nursing and is often hailed as the founder of modern nursing. She was born...

Additionally, AnswerBuilder stores all generated messages in all_messages meta field of GeneratedAnswer and supports a new last_message_only mode for lightweight flows where only the final message needs to be processed.

Visualizing Pipelines with SuperComponents

We extended pipeline.draw() and pipeline.show(), which save pipeline diagrams to images files or display them in Jupyter notebooks. You can now pass super_component_expansion=True to expand any SuperComponents and draw more detailed visualizations.

Here is an example with a pipeline containing MultiFileConverter and DocumentPreprocssor SuperComponents. After installing the dependencies that the MultiFileConverter needs for all supported file formats via pip install haystack-ai pypdf markdown-it-py mdit_plain trafilatura python-pptx python-docx jq openpyxl tabulate pandas, you can run:

from pathlib import Path

from haystack import Pipeline
from haystack.components.converters import MultiFileConverter
from haystack.components.preprocessors import DocumentPreprocessor
from haystack.components.writers import DocumentWriter
from haystack.document_stores.in_memory import InMemoryDocumentStore

document_store = InMemoryDocumentStore()

pipeline = Pipeline()
pipeline.add_component("converter", MultiFileConverter())
pipeline.add_component("preprocessor", DocumentPreprocessor())
pipeline.add_component("writer", DocumentWriter(document_store = document_store))
pipeline.connect("converter", "preprocessor")
pipeline.connect("preprocessor", "writer")

# expanded pipeline that shows all components
path = Path("expanded_pipeline.png")
pipeline.draw(path=path, super_component_expansion=True)

# original pipeline
path = Path("original_pipeline.png")
pipeline.draw(path=path)

SentenceTransformersSimilarityRanker with PyTorch, ONNX, and OpenVINO

We added a new SentenceTransformersSimilarityRanker component that uses the Sentence Transformers library to rank documents based on their semantic similarity to the query. This component replaces the legacy TransformersSimilarityRanker component, which may be deprecated in a future release, with removal following a deprecation period. The SentenceTransformersSimilarityRanker also allows choosing different inference backends: PyTorch, ONNX, and OpenVINO. For example, after installing sentence-transformers>=4.1.0, you can run:

from haystack.components.rankers import SentenceTransformersSimilarityRanker
from haystack.utils.device import ComponentDevice

onnx_ranker = SentenceTransformersSimilarityRanker(
    model="sentence-transformers/all-MiniLM-L6-v2",
    token=None,
    device=ComponentDevice.from_str("cpu"),
    backend="onnx",
)
onnx_ranker.warm_up()
docs = [Document(content="Berlin"), Document(content="Sarajevo")]
output = onnx_ranker.run(query="City in Germany", documents=docs)
ranked_docs = output["documents"]

⬆️ Upgrade Notes

• We've added a py.typed file to Haystack to enable type information to be used by downstream projects, in line with PEP 561. This means Haystack's type hints will now be visible to type checkers in projects that depend on it. Haystack is primarily type checked using mypy (not pyright) and, despite our efforts, some type information can be incomplete or unreliable. If you use static type checking in your own project, you may notice some changes: previously, Haystack's types were effectively treated as Any, but now actual type information will be available and enforced. We'll continue improving typing with the next release.
• The deprecated deserialize_tools_inplace utility function has been removed. Use deserialize_tools_or_toolset_inplace instead, importing it as follows: from haystack.tools import deserialize_tools_or_toolset_inplace.

🚀 New Features

• Added run_async method to ToolInvoker class to allow asynchronous tool invocations.

• Agent can now stream tool result with run_async method as well.

• Introduced serialize_value and deserialize_value utility methods for consistent value (de)serialization across modules.

• Moved the State class to the agents.state module and added serialization and deserialization capabilities.

• Add support for multiple outputs in ConditionalRouter

• Implement JSON-safe serialization for OpenAI usage data by converting token counts and details (like CompletionTokensDetails and PromptTokensDetails) into plain dictionaries.

• Added a new SentenceTransformersSimilarityRanker component that uses the Sentence Transformers library to rank documents based on their semantic similarity to the query. This component is a replacement for the legacy TransformersSimilarityRanker component, which may be deprecated in a future release, with removal following after a deprecation period. The SentenceTransformersSimilarityRanker also allows choosing different inference backends: PyTorch, ONNX, and OpenVINO. To use the SentenceTransformersSimilarityRanker, you need to install sentence-transformers>=4.1.0.

• Add a streaming_callback parameter to ToolInvoker to enable streaming of tool results. Note that tool_result is emitted only after the tool execution completes and is not streamed incrementally.

• Update print_streaming_chunk to print ToolCall information if it is present in the chunk's metadata.

• Update Agent to forward the streaming_callback to ToolInvoker to emit tool results during tool invocation.

• Enhance SuperComponent's type compatibility check to return the detected common type between two input types.

⚡️ Enhancement Notes

• When using HuggingFaceAPIChatGenerator with streaming, the returned ChatMessage now contains the number of prompt tokens and completion tokens in its meta data. Internally, the HuggingFaceAPIChatGenerator requests an additional streaming chunk that contains usage data. It then processes the usage streaming chunk to add usage meta data to the returned ChatMessage.

• We now have a Protocol for TextEmbedder. The protocol makes it easier to create custom components or SuperComponents that expect any TextEmbedder as init parameter.

• We added a Component signature validation method that details the mismatches between the run and run_async method signatures. This allows a user to debug custom components easily.

• Enhanced the AnswerBuilder component with two agent-friendly features:

  1. All generated messages are now stored in the meta field of the GeneratedAnswer objects under an all_messages key, improving traceability and debugging capabilities.
  2. Added a new last_message_only parameter that, when set to True, processes only the last message in the replies while still preserving the complete co...

v2.14.0-rc2

v2.14.0-rc2

v2.14.0-rc1

v2.14.0-rc1

v2.13.2

⚡️ Enhancement Notes

• Updated pipeline execution logic to use a new utility method _deepcopy_with_exceptions, which attempts to deep copy an object and safely falls back to the original object if copying fails. Additionally _deepcopy_with_exceptions skips deep-copying of Component, Tool, and Toolset instances when used as runtime parameters. This prevents errors and unintended behavior caused by trying to deepcopy objects that contain non-copyable attributes (e.g. Jinja2 templates, clients). Previously, standard deepcopy was used on inputs and outputs which occasionally lead to errors since certain Python objects cannot be deepcopied.

🐛 Bug Fixes

• Make internal tool conversion in the HuggingFaceAPICompatibleChatGenerator compatible with huggingface_hub>=0.31.0. In the huggingface_hub library, arguments attribute of ChatCompletionInputFunctionDefinition has been renamed to parameters. Our implementation is compatible with both the legacy version and the new one.
• The HuggingFaceAPIChatGenerator now checks the type of the arguments variable in the tool calls returned by the Hugging Face API. If arguments is a JSON string, it is parsed into a dictionary. Previously, the arguments type was not checked, which sometimes led to failures later in the tool workflow.

v2.13.1

Release Notes

v2.13.1

Bug Fixes

• Update the __deepcopy__ of ComponentTool to gracefully handle NotImplementedError when trying to deepcopy attributes.
• Fix an issue where OpenAIChatGenerator and OpenAIGenerator were not properly handling wrapped streaming responses from tools like Weave.
• Move deserialize_tools_inplace back to original import path of from haystack.tools.tool import deserialize_tools_inplace.

v2.13.0

⭐️ Highlights

Enhanced Agent Tracing and Async Support

Haystack's Agent got several improvements!

Agent Tracing
Agent tracing now provides deeper visibility into the agent's execution. For every call, the inputs and outputs of the ChatGenerator and ToolInvoker are captured and logged using dedicated child spans. This makes it easier to debug, monitor, and analyze how an agent operates step-by-step.

Below is an example of what the trace looks like in Langfuse:

# pip install langfuse-haystack
from haystack_integrations.components.connectors.langfuse.langfuse_connector import LangfuseConnector
from haystack.components.agents import Agent 
from haystack.components.generators.chat import OpenAIChatGenerator

tracer = LangfuseConnector("My Haystack Agent")
agent = Agent(
    system_prompt="You help provide the weather for cities" 
    chat_generator=OpenAIChatGenerator(),
    tools=[weather_tool],
) 

Async Support
Additionally, there's a new run_async method to enable built-in async support for Agent. Just use run_async instead of the run method. Here's an example of an async web search agent:

# set `SERPERDEV_API_KEY` and `OPENAI_API_KEY` as env variables
from haystack.components.agents import Agent 
from haystack.components.generators.chat import OpenAIChatGenerator 
from haystack.components.websearch import SerperDevWebSearch 
from haystack.dataclasses import ChatMessage 
from haystack.tools.component_tool import ComponentTool 

web_tool = ComponentTool(component=SerperDevWebSearch()) 

web_search_agent = Agent(     
    chat_generator=OpenAIChatGenerator(),
    tools=[web_tool],
) 

result = await web_search_agent.run_async(
    messages=[ChatMessage.from_user("Find information about Haystack by deepset")]
) 

New Toolset for Enhanced Tool Management

The new Toolset groups multiple Tool instances into a single manageable unit. It simplifies the passing of tools to components like ChatGenerator, ToolInvoker, or Agent, and supports filtering, serialization, and reuse.
Check out the MCPToolset for dynamic tool discovery from an MCP server.

from haystack.tools import Toolset
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator

math_toolset = Toolset([tool_one, tool_two, ...])
agent = Agent(
    chat_generator=OpenAIChatGenerator(model="gpt-4o-mini"),
    tools=math_toolset
)

@super_component decorator and new ready-made SuperComponents

Creating a custom SuperComponents just got even simpler. Now, all you need to do is define a class with a pipeline attribute and decorate it with @super_component. Haystack takes care of the rest!

Here's an example of building a custom HybridRetriever using the @super_component decorator:

# pip install haystack-ai datasets "sentence-transformers>=3.0.0"

from haystack import Document, Pipeline, super_component
from haystack.components.joiners import DocumentJoiner
from haystack.components.embedders import SentenceTransformersTextEmbedder
from haystack.components.retrievers import InMemoryBM25Retriever, InMemoryEmbeddingRetriever
from haystack.document_stores.in_memory import InMemoryDocumentStore
from datasets import load_dataset

@super_component
class HybridRetriever:
    def __init__(self, document_store: InMemoryDocumentStore, embedder_model: str = "BAAI/bge-small-en-v1.5"):
        embedding_retriever = InMemoryEmbeddingRetriever(document_store)
        bm25_retriever = InMemoryBM25Retriever(document_store)
        text_embedder = SentenceTransformersTextEmbedder(embedder_model)
        document_joiner = DocumentJoiner(join_mode="reciprocal_rank_fusion")

        self.pipeline = Pipeline()
        self.pipeline.add_component("text_embedder", text_embedder)
        self.pipeline.add_component("embedding_retriever", embedding_retriever)
        self.pipeline.add_component("bm25_retriever", bm25_retriever)
        self.pipeline.add_component("document_joiner", document_joiner)

        self.pipeline.connect("text_embedder", "embedding_retriever")
        self.pipeline.connect("bm25_retriever", "document_joiner")
        self.pipeline.connect("embedding_retriever", "document_joiner")

dataset = load_dataset("HaystackBot/medrag-pubmed-chunk-with-embeddings", split="train")
docs = [Document(content=doc["contents"], embedding=doc["embedding"]) for doc in dataset]
document_store = InMemoryDocumentStore()
document_store.write_documents(docs)

query = "What treatments are available for chronic bronchitis?"
result = HybridRetriever(document_store).run(text=query, query=query)
print(result)

New ready-made SuperComponents: MultiFileConverter, DocumentPreprocessor
There are also two ready-made SuperComponents, MultiFileConverter and DocumentPreprocessor, that encapsulate widely used common logic for indexing pipelines.

📚 Learn more about SuperComponents and get the full code example in the Tutorial: Creating Custom SuperComponents

⬆️ Upgrade Notes

• The deprecated api, api_key, and api_params parameters for LLMEvaluator, ContextRelevanceEvaluator, and FaithfulnessEvaluator have been removed. By default, these components will continue to use OpenAI in JSON mode. To customize the LLM, use the chat_generator parameter with a ChatGenerator instance configured to return a response in JSON format. For example:

chat_generator=OpenAIChatGenerator(generation_kwargs={"response_format": {"type": "json_object"}})

• The deprecated generator_api and generator_api_params initialization parameters of LLMMetadataExtractor and the LLMProvider enum have been removed. Use chat_generator instead to configure the underlying LLM. In order for the component to work, the LLM should be configured to return a JSON object. For example, if using OpenAI, you should initialize the LLMMetadataExtractor with

chat_generator=OpenAIChatGenerator(generation_kwargs={"response_format": {"type": "json_object"}})

🚀 New Features

• Add run_async for OpenAITextEmbedder.
• Add run_async method to HuggingFaceAPIDocumentEmbedder. This method enriches Documents with embeddings. It supports the same parameters as the run method. It returns a coroutine that can be awaited.
• Support custom HTTP client configuration via http_client_kwargs (proxy, SSL) for:
  • AzureOpenAIGenerator, OpenAIGenerator and DALLEImageGenerator
  • OpenAIDocumentEmbedder and OpenAITextEmbedder
  • RemoteWhisperTranscriber
• OpenAIChatGenerator and AzureOpenAIChatGenerator now support custom HTTP client config via http_client_kwargs, enabling proxy and SSL setup.
• Introduced the Toolset class, allowing for the grouping and management of related tool functionalities. This new abstraction supports dynamic tool loading and registration.
• We have added internal tracing support to Agent. It is now possible to track the internal loops within the agent by viewing the inputs and outputs each time the ChatGenerator and ToolInvoker is called.
• The HuggingFaceAPITextEmbedder now also has support for a run() method in an asynchronous way, i.e., run_async.
• Add a run_async to the Agent, which calls the run_async of the underlying ChatGenerator if available.
• SuperComponents now support mapping nonleaf pipelines outputs to the SuperComponents output when specifying them in output_mapping.
• AzureOpenAITextEmbedder and AzureOpenAIDocumentEmbedder now support custom HTTP client config via http_client_kwargs, enabling proxy and SSL setup.
• The AzureOpenAIDocumentEmbedder component now inherits from the OpenAIDocumentEmbedder component, enabling asynchronous usage.
• The AzureOpenAITextEmbedder component now inherits from the OpenAITextEmbedder component, enabling asynchronous usage.
• Added async support to the OpenAIDocumentEmbedder component.
• Agent now supports a List of Tools or a Toolset as input.

⚡️ Enhancement Notes

• Added component_name and component_type attributes to PipelineRuntimeError.
  • Moved error message creation to within PipelineRuntimeError
  • Created a new subclass of PipelineRuntimeError called PipelineComponentsBlockedError for the specific case where the pipeline cannot run since no components are unblocked.
• The ChatGenerator Protocol no longer requires to_dict and from_dict methods.

⚠️ Deprecation Notes

• The utility function deserialize_tools_inplace has been deprecated and will be removed in Haystack 2.14.0. Use deserialize_tools_or_toolset_inplace instead.

🐛 Bug Fixes

• OpenAITextEmbedder no longer replaces newlines with spaces in the text to embed. This was only required for the discontinued v1 embedding models.
• OpenAIDocumentEmbedder and AzureOpenAIDocumentEmbedder no longer replace newlines with spaces in the text to embed. This was only required for the discontinued v1 embedding models.
• Fix ChatMessage.from_dict to handle cases where optional fields like name and meta are missing.
• Make Document's first-level fields to take precedence ove...

v2.13.0-rc2

v2.13.0-rc2

v2.13.0-rc1

v2.13.0-rc1