Add reasoning streaming event #1211
Conversation
WalkthroughAdds extraction and streaming emission of LLM "reasoning" content: a new Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant ToolCallingLLM
participant LLM
participant StreamEmitter
participant ToolRunner
Client->>ToolCallingLLM: request call_stream(...)
ToolCallingLLM->>LLM: send prompt / await streaming response
LLM-->>ToolCallingLLM: stream chunks (content, reasoning_content, tool_call hints)
alt reasoning_content present
ToolCallingLLM->>StreamEmitter: emit AI_REASONING(reasoning_content, metadata)
end
alt content present
ToolCallingLLM->>StreamEmitter: emit AI_MESSAGE(content + reasoning, metadata)
end
alt tool call required
ToolCallingLLM->>StreamEmitter: emit TOOL_CALL(...)
ToolCallingLLM->>ToolRunner: invoke tool(...)
ToolRunner-->>ToolCallingLLM: tool result
ToolCallingLLM->>StreamEmitter: emit TOOL_RESPONSE(...)
else no tool call
ToolCallingLLM->>StreamEmitter: emit ANSWER_END(final_message)
end
StreamEmitter-->>Client: forward stream events
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested reviewers
Pre-merge checks✅ Passed checks (3 passed)
Comment |
…-to-ui-as-streaming-ev
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
holmes/core/tool_calling_llm.py (1)
938-946: LGTM: AI_MESSAGE event properly handles reasoning and content together.The event correctly includes both
contentandreasoningfields, allowing the frontend to display them together when tool calls are present. The conditionif reasoning or message:appropriately prevents emitting empty events.Optional: Consider consistency with non-streaming version
The non-streaming version (Line 448) checks
text_response and text_response.strip()to avoid logging whitespace-only content. For consistency, consider:-if reasoning or message: +if reasoning or (message and message.strip()):This prevents emitting AI_MESSAGE events when message contains only whitespace. However, this is a minor edge case and the current implementation is acceptable.
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
holmes/core/tool_calling_llm.py
🧰 Additional context used
📓 Path-based instructions (2)
**/*.py
📄 CodeRabbit inference engine (CLAUDE.md)
**/*.py: Use Ruff for formatting and linting (configured in pyproject.toml)
Type hints required (mypy configuration in pyproject.toml)
ALWAYS place Python imports at the top of the file, not inside functions or methods
Files:
holmes/core/tool_calling_llm.py
holmes/core/tool_calling_llm.py
📄 CodeRabbit inference engine (CLAUDE.md)
holmes/core/tool_calling_llm.py: Use LiteLLM for multi-provider support (OpenAI, Anthropic, Azure, etc.) in LLM integration
Implement structured tool calling with automatic retry and error handling
Files:
holmes/core/tool_calling_llm.py
🧠 Learnings (2)
📚 Learning: 2025-12-21T13:17:48.366Z
Learnt from: CR
Repo: HolmesGPT/holmesgpt PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-12-21T13:17:48.366Z
Learning: Applies to holmes/core/tool_calling_llm.py : Use LiteLLM for multi-provider support (OpenAI, Anthropic, Azure, etc.) in LLM integration
Applied to files:
holmes/core/tool_calling_llm.py
📚 Learning: 2025-12-21T13:17:48.366Z
Learnt from: CR
Repo: HolmesGPT/holmesgpt PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-12-21T13:17:48.366Z
Learning: Applies to holmes/core/tool_calling_llm.py : Implement structured tool calling with automatic retry and error handling
Applied to files:
holmes/core/tool_calling_llm.py
🧬 Code graph analysis (1)
holmes/core/tool_calling_llm.py (1)
holmes/utils/stream.py (2)
StreamMessage(28-30)StreamEvents(16-25)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (5)
- GitHub Check: build (3.12)
- GitHub Check: build (3.11)
- GitHub Check: build (3.10)
- GitHub Check: llm_evals
- GitHub Check: build
🔇 Additional comments (3)
holmes/core/tool_calling_llm.py (3)
915-916: LGTM: Clear variable extraction for reasoning and message content.Extracting
reasoning_contentand message content into local variables improves readability and prepares them for use in multiple stream events. The use ofgetattrwith a default value safely handles cases wherereasoning_contentmay not be present.
927-936: LGTM: Proper use of extracted message in ANSWER_END event.Using the extracted
messagevariable instead of accessingresponse_message.contentdirectly improves code readability and is consistent with the non-streaming implementation.
918-925: LGTM: Correctly implements AI reasoning stream event with cross-provider safety.The
AI_REASONINGevent emission uses defensive attribute access viagetattr(response_message, "reasoning_content", None), ensuring compatibility across all LiteLLM providers. The event is emitted only when reasoning exists, gracefully degrading for providers that don't support reasoning content. This pattern aligns with the non-streaming implementation and requires no additional provider-specific handling.
|
Dev Docker images are ready for this commit: Use this tag to pull the image for testing. gcloud auth configure-docker us-central1-docker.pkg.dev
docker pull us-central1-docker.pkg.dev/robusta-development/temporary-builds/holmes:425c09e
docker tag us-central1-docker.pkg.dev/robusta-development/temporary-builds/holmes:425c09e me-west1-docker.pkg.dev/robusta-development/development/holmes-dev:425c09e
docker push me-west1-docker.pkg.dev/robusta-development/development/holmes-dev:425c09ePatch Helm values in one line (choose the chart you use):
helm upgrade --install holmesgpt ./helm/holmes \
--set registry=me-west1-docker.pkg.dev/robusta-development/development \
--set image=holmes-dev:425c09e
helm upgrade --install robusta robusta/robusta \
--reuse-values \
--set holmes.registry=me-west1-docker.pkg.dev/robusta-development/development \
--set holmes.image=holmes-dev:425c09e |
Summary
Testing
Codex Task
Summary by CodeRabbit
✏️ Tip: You can customize this high-level summary in your review settings.