[WIP] Integrate emotion detection model into message processing

[Copilot]

Emotion-Aware Message Processing Integration

This PR integrates emotion detection into the chat pipeline to provide context-aware responses.

Plan:

• Create robust emotion detection wrapper (robertA_model.py) with fallback handling
• Add environment configuration for emotion detection (EMOTION_ENABLE, EMOTION_THRESHOLD, EMOTION_TOP_K)
• Modify chat service to detect emotions before LLM call
• Augment system prompt with emotion context
• Add comprehensive logging throughout emotion pipeline
• Maintain existing TTS pipeline unchanged
• Add graceful error handling (model failures don't block responses)
• Create test file for emotion integration (if pytest exists)

Key Requirements:

• Load emotion model once at startup (lazy singleton pattern)
• Multi-label emotion detection with confidence scores
• Inject emotion context into system instructions without removing existing persona
• Preserve all existing functionality and API structure
• Comprehensive server-side logging for debugging

Current Status:

• Analyzed existing codebase structure
• Identified chat flow: routes/chat.py → services/chat_service.py → LongCat API
• Reviewed existing emotion prediction code in prediction.py
• Identified TTS service that must remain unchanged

Original prompt

You are GitHub Copilot Coding Agent. Implement emotion-aware message processing without breaking existing functionality.

Goal
Integrate a pretrained RoBERTa-based emotion detection model into the backend so that every incoming user message is first classified for emotions, and the detected emotions (potentially multiple, with confidence scores) are injected into the system instruction for the LLM before generating the reply. After the LLM produces the reply, the existing Text-To-Speech (TTS) pipeline should run exactly as before. No other behavior or APIs should be altered.

Repository Structure (relevant)
backend/
ml_models/
emotion_detector_model/
preTrainedModel/
robertA_model.py # already exists – load model from here (adjust naming if needed)
llm/
orchestrator.py # (assumed) main coordination logic for LLM responses
services/
tts_service.py # existing TTS pipeline (DO NOT MODIFY logic beyond necessary call points)
api/
routes/
chat_routes.py # (assumed) endpoint receiving user messages
utils/
logging.py
config.py

If actual file names differ, adapt accordingly but keep changes minimal and localized.

Requirements

1. Emotion Model Integration

   • Use the existing file backend/ml_models/emotion_detector_model/preTrainedModel/robertA_model.py.
   • If the file only contains weights or a partial wrapper, extend it to expose a function:
     def predict_emotions(text: str, threshold: float = 0.35, top_k: int | None = None) -> list[tuple[str, float]]
     Returns a list of (label, confidence) sorted descending by confidence.
   • Multi-label detection: use sigmoid outputs; keep all labels above threshold. If none pass threshold, include the top-1 label.
   • Provide an internal utility to map raw labels to grouped high-level categories only if needed; otherwise return raw labels unchanged.

2. System Instruction Augmentation

   • Before calling the LLM generation function, construct a system instruction segment:
     Example improved wording (must be parameterized):
     Current user emotional state (multi-label with confidence):
     {emotion_list_formatted}
     Guidelines:
     1. Acknowledge salient emotions naturally (no forced repetition).
     2. Adapt tone: supportive for negative, encouraging for anticipatory/anxious, celebratory for positive.
     3. Maintain factual correctness and do not hallucinate emotional states not detected.
     4. Preserve existing persona and all other system rules.
     • Inject this segment WITHOUT removing existing system instructions or other configuration.
     • If existing orchestrator uses a list of messages (e.g. [{role: system, content: ...}, ...]) then append or merge by adding a new system message after the base persona rather than overwriting.

3. Flow

   • chat_routes.py (or equivalent endpoint):
     1. Receive user text.
     2. Call predict_emotions(text).
     3. Pass both original text and emotion metadata to orchestrator.
   • orchestrator.py:
     1. Build augmented system instruction incorporating emotion output.
     2. Call LLM as before.
     3. Forward resulting text unchanged into TTS stage.

4. TTS Pipeline

   • DO NOT change the API signature or internal logic of the TTS service.
   • Ensure the existing post-LLM hook still triggers exactly once.
   • If current flow is: response = generate_llm_reply(...); audio = tts_service.synthesize(response); return {text, audio}, keep it intact.
   • Only add emotion metadata if the API already supports additional fields; otherwise keep output structure unchanged.

5. Configuration & Extensibility

   • Add optional environment variables:
     EMOTION_THRESHOLD (float, default 0.35)
     EMOTION_TOP_K (int, optional)
     EMOTION_ENABLE (bool, default true; if false, skip model and do not modify system prompt)
   • If EMOTION_ENABLE is false, orchestrator must not add any emotion instruction segment.
   • Log (debug level) the detected emotions and chosen thresholds.
   • Fail gracefully: if model load fails, log warning and proceed without emotion augmentation.

6. Performance

   • Load emotion model once at startup (lazy singleton).
   • Support batching later: design predict_emotions to accept a single text now but be easily extendable.
   • Ensure GPU usage if available (torch.cuda.is_available()) else CPU.

7. Error Handling

   • If prediction raises an exception, catch it, log it, and continue the LLM flow without emotion augmentation.
   • Never allow emotion failure to block a user response.

8. Testing / Validation

   • Add a lightweight unit test file: tests/test_emotion_integration.py
     • Mock a sample text and assert predict_emotions returns list with (label, score).
     • Test orchestrator builds a system message containing “Current user emotional state”.
   • If test infrastructure exists (pytest), integrate; otherwise create the test file ready fo...

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Preview	Comments	Updated (UTC)
isabella	Ready	Preview	Comment	Nov 17, 2025 8:25am