Skip to content

feat: add PostHog-gated model switching to chat - #38

Draft
burnedinthesky wants to merge 7 commits into
mainfrom
feat/add-model-switching-support
Draft

feat: add PostHog-gated model switching to chat#38
burnedinthesky wants to merge 7 commits into
mainfrom
feat/add-model-switching-support

Conversation

@burnedinthesky

Copy link
Copy Markdown
Member

Type of changes

  • Feature

Purpose

  • Add the ability to select between two LLM models in the chat UI, gated by a PostHog feature flag so we can force a single model or open up a selector without a redeploy

The two models:

Send as (id) Display name
gpt-oss-120b GPT OSS 120b
gemma-4-31B-it Gemma 4 31b

IDs match the sciedu-llm ALLOWED_MODELS allowlist exactly (note the capital B in gemma-4-31B-it; a mismatch is rejected with 400).

Additional Information

Feature flag: force-model-selection

Read with useFeatureFlagVariantKey; resolves to one of three modes:

  • value is a model id -> force that model, no selector, every request sends that id
  • "none" -> show a selector in the composer bar; user switches freely; choice persisted to localStorage
  • absent / undefined / other -> hide the selector and omit model, so the backend uses its default (preserves todays behavior)

Changes

  • New models.ts registry (single source of truth for ids and display names)
  • New ModelSelectionContext / ModelSelectionProvider that resolves the flag, holds the selection, and persists to localStorage; wraps the chat pages in ChatLayout
  • New ModelSelector (Radix Select) rendered in the composer bar; self-hides unless in "none" mode
  • model threaded through createMessage -> useChat.sendMessage / startChat -> NewChatPage
  • useModelSelection degrades to an "off" default outside its provider, so the shared chat components used by the edu-mode course chat keep working (no selector, no model sent)

Notes and follow-ups

Test

pnpm lint and pnpm build pass. Locally, override the flag in the console:

  • posthog.featureFlags.override({ "force-model-selection": "none" }) -> selector appears; sending shows the chosen model in the POST /api/chat/... body; survives reload
  • posthog.featureFlags.override({ "force-model-selection": "gemma-4-31B-it" }) -> no selector; every send carries "model":"gemma-4-31B-it"
  • clear or false -> no selector; no model field in the body

🤖 Generated with Claude Code

burnedinthesky and others added 7 commits July 13, 2026 12:02
Support selecting between two LLM models (GPT OSS 120b / Gemma 4 31b) and
thread the chosen model id through the chat send path down to the backend's
`model` field.

A PostHog flag `force-model-selection` drives the behavior:
- a model id  -> force that model, no selector
- "none"      -> show a selector in the composer bar, switch freely, persist
                 the choice to localStorage
- absent/other-> hide the selector and omit `model` (backend default)

`useModelSelection` degrades to an "off" default outside its provider, so the
shared chat components (used by the edu-mode course chat) keep working with no
selector and no model sent.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The floating `version: 11` resolved to pnpm 11.12.0, whose self-installer
crashes in pnpm/action-setup ("Cannot use 'in' operator to search for
'integrity' in undefined"), failing every CI job at the Install pnpm step.
Pin to 11.9.0 (matches local dev) to avoid the broken release.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants