FROM feat/464-update-input-support-multiple-keys TO development

[ryaneggz]

Closes #464

Summary by CodeRabbit

• New Features

  • Dynamic model discovery with caching and manual reset; models list now includes additional providers.
  • Model selector shows truncated labels with hover tooltips.
  • App version displayed dynamically in chat UI.
  • Messages now show tokens count alongside rate.
  • New input wrapper format for messages used across streaming and regular requests.

• Changes

  • Streaming and final-state handling simplified; streaming now uses the unified input shape.
  • Improved mobile submit behavior to reduce accidental Enter submissions.
  • Anonymization results saved into the unified input payload.

• Chores

  • Docker Compose startup behavior simplified.
  • Schemas and services reorganized for clearer LLM/model handling.

[coderabbitai]

Caution

Review failed

The pull request is closed.

Walkthrough

Replaces top-level message payloads with a structured LLMInput and new LLM schemas; adds LLM discovery service and model-filtering utilities; updates flows, stream pipeline, routes, auth, presidio, and assistant services to use params.input; refactors frontend payload typings, context and UI; adjusts docker-compose and Changelog.

Changes

Cohort / File(s)	Summary
Schemas — new LLM types backend/src/schemas/entities/llm.py	Added Pydantic models and helpers: PresidioRequest, Config, LLMInput (with ChatMessage + to_langchain_messages()), AssistantSearch, Assistant (slug, datetime serializers, to_llm_request()), and LLMRequest.
Schemas — public surface & exports backend/src/schemas/entities/__init__.py, backend/src/schemas/models/assistant.py (deleted)	Reworked public import surface: removed LangChain-specific public imports, re-exported llm entities, and deleted legacy Assistant/AssistantSearch models file.
Flows / Orchestration backend/src/flows/__init__.py	Added ProtectedUser; narrowed subagent init to LLMRequest; new `init_config(params: LLMRequest, user: ProtectedUser
Routes / Stream / Auth backend/src/routes/v0/llm.py, backend/src/utils/stream.py, backend/src/utils/auth.py	Endpoints and streaming use params.input (and params.input.to_langchain_messages()); llm_stream now calls init_config(params, user); stream pipeline and auth helpers now accept LLMInput / LLMRequest shaped inputs; error handling standardized to HTTP status codes.
Services — LLM discovery & assistant store backend/src/services/llm.py, backend/src/services/assistant.py, backend/src/services/presidio.py	New LLMService with caching (_fetch_models, _reset_cache) and model_by_provider; AssistantService gained _get_store_key() and get(key); Presidio reads/writes query at params.input.messages[-1].content.
Constants — dynamic model lists backend/src/constants/llm.py	Removed static Ollama constant; added get_ollama_models() and updated get_all_models() / get_free_models() to build model lists dynamically (include Ollama when available) with error handling.
Utils — model filtering & middleware backend/src/utils/llm.py, backend/src/utils/middleware.py	Added filter_models(...) and filter_tool_call_models(...); introduced add_ai_message_metadata hook and pii_middleware() PIIMiddleware definitions.
Stream pipeline changes backend/src/utils/stream.py, frontend/src/hooks/useChat.ts	stream_generator signature and internals now take LLMInput, attach agent model to last input message, pass input.messages to agents, handle PIIDetectionError and structured SSE errors, and simplify final-state thread updates; frontend sends input wrapper.
Frontend — payload shape, hooks & types frontend/src/lib/services/threadService.ts, frontend/src/hooks/useChat.ts, frontend/src/context/ChatContext.tsx, frontend/src/pages/chat/chat-v2.tsx, frontend/src/hooks/useAgent.ts	Stream payload switched from messages to input: { messages }; introduced typed aliases (Input, Metadata, A2A, MCP, Tools, Subagents, Presidio); context exposes model hooks; components invoke useModelsEffect to load models; agent state synced to selected model.
Frontend UI & Utils frontend/src/components/lists/SelectModel.tsx, frontend/src/pages/chat/ChatPanel.tsx, frontend/src/components/inputs/ChatInput.tsx, frontend/src/hooks/useAppHook.ts, frontend/src/components/lists/ChatMessages.tsx, frontend/src/lib/utils/message.ts	SelectModel shows truncated labels with tooltip; ChatPanel uses dynamic appVersion; ChatInput uses new isLikelyMobile() guard; added isLikelyMobile() helper; ChatMessages falls back to latest human message model and shows token counts; added latestHumanMessage() util.
Prompt optimizer backend/src/services/prompt/optimize.py	PromptOptimizerRequest.model type changed from ChatModels enum to str (default unchanged).
Compose & Changelog docker-compose.yml, Changelog.md	Removed active ollama startup command (container no-op by default; commented alternate command remains); added Changelog entry v0.0.2 (feat/464-update-input-support-multiple-keys).

Sequence Diagram(s)

sequenceDiagram
  actor Client
  participant Frontend
  participant API as FastAPI
  participant Auth
  participant InitConfig as init_config
  participant AgentCtor as construct_agent
  participant StreamGen as stream_generator
  participant Orchestra

  Note right of Frontend `#F0F8FF`: Frontend wraps messages in LLMInput
  Client->>Frontend: send messages
  Frontend->>API: POST /llm (body: input:{ messages })
  API->>Auth: get_optional_user(params: LLMRequest)
  API->>InitConfig: init_config(params, user)
  InitConfig-->>API: RunnableConfig
  API->>AgentCtor: construct_agent(system_prompt, tools, model, subagents, config)
  AgentCtor-->>Orchestra: create agent/orchestra
  API->>StreamGen: stream_generator(input: LLMInput, model, system_prompt, tools, subagents, config, service_context)
  StreamGen->>Orchestra: stream using input.messages
  Orchestra-->>StreamGen: stream chunks & final_state
  StreamGen-->>API: final_state -> thread update (last human message)
  API-->>Frontend: stream SSE + final update
  Frontend-->>Client: render assistant response

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

• Heterogeneous, cross-cutting changes touching schemas, orchestration, streaming, services, utils, and frontend.
• Files needing targeted attention:
  • backend/src/schemas/entities/llm.py — message conversion, defaults, datetime serializers, Assistant.to_llm_request.
  • backend/src/flows/__init__.py — init_config, construct_agent changes, removal of aget_state.
  • backend/src/utils/stream.py — signature change, LLMInput handling, final-state/thread update logic, SSE error handling.
  • backend/src/services/llm.py & backend/src/utils/llm.py — caching, network error handling, provider normalization and filtering.
  • backend/src/routes/v0/llm.py — consistent use of params.input and message conversions.
  • Frontend — threadService payload types and places where input: { messages } is sent/consumed.
  • Presidio and assistant store key changes — ensure read/write align with new request shape.

Poem

🐰
I nibble code beneath the cabinet light,
Messages tucked in an input bed just right,
Models cached and streams that hop,
Middleware guards and tool calls pop,
I stamp the changelog and munch a carrot bright. 🥕

---

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Free

📥 Commits

Reviewing files that changed from the base of the PR and between 8340dd6 and fb3781d.

📒 Files selected for processing (2)

• frontend/src/components/inputs/ChatInput.tsx (2 hunks)
• frontend/src/hooks/useAgent.ts (3 hunks)

---

Note

🎁 Summarized by CodeRabbit Free

Your organization is on the Free plan. CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please upgrade your subscription to CodeRabbit Pro by visiting https://app.coderabbit.ai/login.

_{Comment @coderabbitai help to get the list of available commands and usage tips.}