TypeError: unsupported operand type(s) for += on dict in _combine_llm_outputs when making multiple LLM calls

[ffoss]

Submission checklist

• This is a bug, not a usage question.
• I added a clear and descriptive title that summarizes this issue.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).
• This is not related to the langchain-community package.
• I posted a self-contained, minimal, reproducible example. A maintainer can copy it and run it AS IS.

Package (Required)

• langchain
• langchain-openai
• langchain-anthropic
• langchain-classic
• langchain-core
• langchain-model-profiles
• langchain-tests
• langchain-text-splitters
• langchain-chroma
• langchain-deepseek
• langchain-exa
• langchain-fireworks
• langchain-groq
• langchain-huggingface
• langchain-mistralai
• langchain-nomic
• langchain-ollama
• langchain-openrouter
• langchain-perplexity
• langchain-qdrant
• langchain-xai
• Other / not sure / general

Related Issues / PRs

No response

Reproduction Steps / Example Code (Python)

import os
from langchain_mistralai import ChatMistralAI
from langchain_core.messages import HumanMessage

llm = ChatMistralAI(
    model="mistral-small-latest",
    api_key=os.environ["MISTRAL_API_KEY"],
)

result = llm.generate([
    [HumanMessage(content="What is 1+1?")],
    [HumanMessage(content="What is 2+2?")],
])
print(result)

Error Message and Stack Trace (if applicable)

TypeError                                 Traceback (most recent call last)
Cell In[2], line 10
      6     model="mistral-small-latest",
      7     api_key=os.environ["MISTRAL_API_KEY"],
      8 )
      9 
---> 10 result = llm.generate([
     11     [HumanMessage(content="What is 1+1?")],
     12     [HumanMessage(content="What is 2+2?")],
     13 ])

File /opt/homebrew/Caskroom/miniconda/base/envs/natural_explanation_for_ids/lib/python3.12/site-packages/langchain_core/language_models/chat_models.py:1677, in BaseChatModel.generate(self, messages, stop, callbacks, tags, metadata, run_name, run_id, **kwargs)
   1672         raise
   1673 flattened_outputs = [
   1674     LLMResult(generations=[res.generations], llm_output=res.llm_output)
   1675     for res in results
   1676 ]
-> 1677 llm_output = self._combine_llm_outputs([res.llm_output for res in results])
   1678 generations = [res.generations for res in results]
   1679 output = LLMResult(generations=generations, llm_output=llm_output)

File /opt/homebrew/Caskroom/miniconda/base/envs/natural_explanation_for_ids/lib/python3.12/site-packages/langchain_mistralai/chat_models.py:651, in ChatMistralAI._combine_llm_outputs(self, llm_outputs)
    649 for k, v in token_usage.items():
    650     if k in overall_token_usage:
--> 651         overall_token_usage[k] += v
    652     else:
    653         overall_token_usage[k] = v

TypeError: unsupported operand type(s) for +=: 'dict' and 'dict'

Description

Calling ChatMistralAI.generate() with multiple message lists raises TypeError: unsupported operand type(s) for +=: 'dict' and 'dict' in _combine_llm_outputs. The method attempts to aggregate token usage across multiple API calls using +=, but Mistral's API returns nested dicts for some token usage fields, making += invalid. Expected behaviour is that the results are combined and returned without error.

As a workaround, using ChatOpenAI from langchain-openai pointed at Mistral's OpenAI-compatible endpoint (base_url="https://api.mistral.ai/v1") works without error, confirming the bug is in langchain-mistralai's _combine_llm_outputs implementation specifically.

System Info

System Information

OS: Darwin
OS Version: Darwin Kernel Version 25.2.0: Tue Nov 18 21:03:25 PST 2025; root:xnu-12377.61.12~1/RELEASE_ARM64_T8122
Python Version: 3.12.13 | packaged by Anaconda, Inc. | (main, Mar 19 2026, 20:12:32) [Clang 20.1.8 ]

Package Information

langchain_core: 1.4.8
langchain: 1.3.2
langchain_community: 0.3.31
langsmith: 0.8.18
langchain_classic: 1.0.8
langchain_mistralai: 1.1.5
langchain_openai: 1.3.2
langchain_protocol: 0.0.18
langchain_text_splitters: 1.1.2
langgraph_sdk: 0.3.6

Optional packages not installed

deepagents
deepagents-cli
...
websockets: 15.0.1
wrapt: 2.2.1
xxhash: 3.7.0
zstandard: 0.25.0

[gitbalaji]

Hi, I'd like to work on this. Could a maintainer please assign it to me?

[OkiriGabriel]

The root cause is in ChatMistralAI._combine_llm_outputs: when Mistral returns nested dicts for some token usage fields (e.g. prompt_tokens_details: {"cached_tokens": 10}), the method does overall_token_usage[k] += v where both sides are dicts — which raises TypeError.

The fix adds a _merge_token_usage_value helper that recursively merges dict values key-by-key and falls back to + for numeric types. This is a single-function addition with no changes to the _combine_llm_outputs call signature.

5 unit tests added covering: flat int fields summed correctly, nested dicts merged recursively, None entries skipped, nested key only in one output, and model name in result. All 73 existing unit tests continue to pass. Could a maintainer assign me to reopen the PR?

[sanmaxdev]

I looked into this and have a fix ready in #38489 (currently closed pending assignment).

The root cause is the += on line 659 of chat_models.py. When Mistral returns nested dict fields like prompt_tokens_details: {"cached_tokens": 10}, both operands are dicts and += raises TypeError.

My approach adds a _merge_token_usage_value helper that:

• Recursively merges dict values key-by-key (so {"cached_tokens": 5} + {"cached_tokens": 3} becomes {"cached_tokens": 8})
• Falls back to + for numeric types (flat fields like prompt_tokens still sum correctly)
• Falls back to the new value for mismatched or non-numeric types

This keeps the _combine_llm_outputs signature unchanged and only changes the merge path. I added 5 unit tests covering flat ints, nested dicts, None entries, asymmetric nested keys, and None token_usage.

Happy to reopen the PR once assigned.