Skip to content

feat(partners): ContextOverflowError for Groq/Mistral + standard test coverage #37533

Description

@Jeel3011

Submission checklist

  • This is a feature request, not a bug report or usage question.
  • I added a clear and descriptive title that summarizes the feature request.
  • I used the GitHub search to find a similar feature request and didn't find it.
  • I checked the LangChain documentation and API reference to see if this feature already exists.
  • This is not related to the langchain-community package.

Package (Required)

  • langchain
  • langchain-openai
  • langchain-anthropic
  • langchain-classic
  • langchain-core
  • langchain-model-profiles
  • langchain-tests
  • langchain-text-splitters
  • langchain-chroma
  • langchain-deepseek
  • langchain-exa
  • langchain-fireworks
  • langchain-groq
  • langchain-huggingface
  • langchain-mistralai
  • langchain-nomic
  • langchain-ollama
  • langchain-openrouter
  • langchain-perplexity
  • langchain-qdrant
  • langchain-xai
  • Other / not sure / general

Feature Description

The Feb 2026 changelog states that SummarizationMiddleware auto-triggers when a chat model raises ContextOverflowError. Today only langchain-anthropic and langchain-openai map provider context-limit errors to that exception.

ChatGroq raises groq.BadRequestError with code context_length_exceeded but never converts it to ContextOverflowError, so summarization does not auto-trigger for Groq users.

ChatMistralAI surfaces context overflow as httpx.HTTPStatusError (HTTP 400) without conversion, so the same gap exists for Mistral.

Additionally, BaseChatModel.get_num_tokens_from_messages() ignores tool schemas (with a warning), which underestimates tokens for tool-heavy agents and can delay summarization.

Use Case

I use SummarizationMiddleware with Groq and Mistral models in agents with many tools. When context limits are hit, I get raw provider errors instead of ContextOverflowError, so automatic summarization never runs. Token estimates also omit tool schemas, so summarization triggers too late.

Proposed Solution

  1. langchain-groq: Add GroqContextOverflowError (subclasses groq.BadRequestError and ContextOverflowError), map overflow in _generate/_agenerate/_stream/_astream. Override get_num_tokens_from_messages to include tool schemas via tiktoken cl100k_base.

  2. langchain-mistralai: Same pattern with MistralContextOverflowError (httpx.HTTPStatusError + ContextOverflowError).

  3. langchain-tests: Add raises_on_context_overflow opt-in property (default False) and integration tests.

Draft PRs (closed pending issue link + assignee):

Alternatives Considered

Manual try/except around every model call to detect context overflow strings — fragile and duplicates logic already in Anthropic/OpenAI integrations.

Additional Context

AI-assisted contribution. Happy to split into separate issues/PRs if maintainers prefer.

Metadata

Metadata

Labels

externalfeature requestRequest for an enhancement / additional functionalitygroq`langchain-groq` package issues & PRsmistralai`langchain-mistralai` package issues & PRsstandard-testsRelated to the package `standard-tests`
No fields configured for Feature.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions