Skip to content

Feat/issue 39 llm personalized insight generation - #84

Open
nattyakos07-web wants to merge 3 commits into
Arena1X:mainfrom
nattyakos07-web:feat/issue-39-llm-personalized-insight-generation
Open

Feat/issue 39 llm personalized insight generation#84
nattyakos07-web wants to merge 3 commits into
Arena1X:mainfrom
nattyakos07-web:feat/issue-39-llm-personalized-insight-generation

Conversation

@nattyakos07-web

Copy link
Copy Markdown
Closes #39
  
  Summary
  
  Implements the LLM-powered personalized coaching insight pipeline for the
  Leaderboard Coach agent role. Trend signals detected from a user's prediction
  history are fed through a structured OpenAI prompt and returned as 1–3 short,
  encouraging coaching messages.
  
  Hard rules are enforced in code after generation — not solely via prompt
  instructions — so no generated message can ever exceed 280 characters or
  contain gambling-encouragement phrasing. If the LLM call fails or every
  insight is rejected by the rules, the service degrades gracefully to
  deterministic template-based fallbacks. The user always receives insights;
  errors never surface.
  
  This PR builds directly on the streak/trend detection engine from #38
  (TrendService.detectSignals()).
  
  Changes Made
  
  - [x] src/coach/interfaces/coaching-insight.interface.ts — New CoachingInsight
   and LlmInsightResponse interfaces defining the validated output shape
  - [x] src/coach/prompts/coaching-insight.prompt.ts — Prompt builder that takes
  UserPerformance + TrendSignal[] and returns { system, user } strings;
  references actual signal numbers for grounding
  - [x] src/coach/coach.constants.ts — INSIGHT_BLOCKLIST of
  gambling-encouragement phrases + containsInsightBlockedPhrase() enforcement
  helper, and MAX_INSIGHT_MESSAGE_LENGTH = 280
  - [x] src/coach/coach.service.ts — CoachService.generateInsights(performance)
   wiring the full pipeline: TrendService.detectSignals() → prompt build →
  LlmService.complete() (JSON mode) → parse → validate schema → enforce hard
  rules → fallback
  - [x] src/coach/coach.module.ts — Updated to register CoachService and
  LlmService
  
  Testing
  
  How was this tested?
  
  Unit tests for CoachService covering the three required scenarios are pending
  (tracked as the remaining open task on this issue). The existing TrendService
   tests (779 lines, 60+ cases) continue to cover the signal-detection layer.
  
  Manual verification of the validation pipeline:
  
  // Blocklist check rejects gambling phrases
  containsInsightBlockedPhrase("You're on a streak, double down!")  // → true
  containsInsightBlockedPhrase("You're on a 4-win streak!")         // → false
  
  // Length check truncates oversized messages
  "x".repeat(281).length > MAX_INSIGHT_MESSAGE_LENGTH  // → true (rejected)
  
  LLM fallback path triggers on:
  
  - OPENAI_API_KEY not set (LlmService.isConfigured() → false, throws)
  - API call failure (network error, timeout, rate limit)
  - LLM returns non-JSON or schema mismatch
  - All generated insights fail the hard-rule check
  
  Example valid output shape:
  
  {
    "insights": [
      {
        "message": "You're on a 6-prediction winning streak! Your recent form is
  excellent — keep trusting your analysis.",
        "signalType": "hot-streak",
        "priority": 3
      },
      {
        "message": "You're 1 correct pick away from beating your personal best
  streak of 6. Stay focused.",
        "signalType": "near-milestone",
        "priority": 3
      }
    ]
  }
  
  API Documentation
  
  - [ ] Swagger documentation updated (if endpoints changed)
  - [x] No new HTTP endpoints — CoachService is a provider consumed internally
  
  Performance Impact
  
  - [x] No performance impact on existing endpoints
  - LLM call adds latency only when coaching insights are explicitly requested;
  fallback is synchronous
  
  Breaking Changes
  
  - [x] No breaking changes
  - CoachModule now imports LlmService directly — no changes to existing module
  consumers
  
  Checklist
  
  - [x] Code follows project conventions (NestJS injectable services, plain
  interfaces, no ORM decorators)
  - [ ] All tests pass — unit tests for CoachService still to be written
  - [ ] Linting passes: pnpm lint
  - [x] No console.log or debug code committed — uses NestJS Logger
  - [x] Commit messages follow Conventional Commits
  - [x] No unrelated changes included
  
  Related Issues / PRs
  
  - Depends on #38 (streak and trend detection engine — TrendService,
  UserPerformance, TrendSignal interfaces)
  - Uses LlmService and prompt-template patterns established in the Creator
  Assistant module
  
  Additional Notes
  
  Known limitation / open task: coach.service.spec.ts (blocklist filtering,
  fallback path, JSON validation tests) was not committed with this PR. That
  work was interrupted and should be completed before merge.
  
  Design decision: LlmService is registered directly in CoachModule rather than
  importing the full AssistantModule, keeping the coach feature self-contained
  and avoiding unnecessary module coupling.

Awointa and others added 3 commits August 21, 2026 19:35
- Add UserPerformance and TrendSignal interfaces (discriminated union,
  priorities 1-5) in src/coach/interfaces/trend.interface.ts
- Implement TrendService.detectSignals() with five signal detectors:
    hot-streak  (≥4 consecutive correct, priority 1)
    cold-streak (≥4 consecutive wrong,   priority 2)
    improving   (recent-10 accuracy ≥15pp above overall, priority 3)
    declining   (recent-10 accuracy ≥15pp below overall, priority 4)
    near-milestone (within 2 of total round-number or PB streak, p. 5)
- All six thresholds configurable via TREND_* env vars with documented
  defaults; invalid values fall back to defaults gracefully
- Results are always priority-ordered; multiple simultaneous signals
  are all returned
- Service is pure and deterministic (no side effects, no mutation)
- Add CoachModule exposing TrendService
- 194 exhaustive unit tests: positive + negative for every signal type,
  simultaneous signals, priority ordering, env overrides, determinism
…rena1X#39)

- Add CoachingInsight and LlmInsightResponse interfaces
- Add coaching-insight prompt template (UserPerformance + TrendSignal[])
- Add INSIGHT_BLOCKLIST with containsInsightBlockedPhrase() enforcement
- Add CoachService.generateInsights() wiring performance → trends → LLM → validated insights
- Hard rules enforced post-generation: max 280 chars, no gambling phrases
- Fallback to deterministic template messages on LLM failure or full rejection
- Update CoachModule to register CoachService and LlmService
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants