Skip to content

feat: extract LLM prompts and settings into external config#14

Merged
msogin merged 24 commits into
mainfrom
settings-extraction
Mar 21, 2026
Merged

feat: extract LLM prompts and settings into external config#14
msogin merged 24 commits into
mainfrom
settings-extraction

Conversation

@PavelLoparev

@PavelLoparev PavelLoparev commented Mar 21, 2026

Copy link
Copy Markdown
Contributor
image

Summary

  • Prompt template system: All hardcoded LLM prompts extracted into prompts/ directory with {{PLACEHOLDER}} syntax, loaded by prompt-loader.ts with in-memory caching
  • Configurable LLM settings: temperature, max_tokens, max_iterations now readable from env vars (ANALYZER_TEMPERATURE, CHAT_AGENT_MAX_TOKENS, etc.) with current values as defaults
  • Settings UI: LLM Settings tab now displays per-service settings grid, prompt templates directory, and masked secrets (last 5 chars visible)
  • Snapshot tests: All 5 LLM services have exact-match prompt and settings assertions that verified zero behavior change during migration

Files

New

  • src/lib/prompt-loader.ts — template file loader with caching and {{VAR}} substitution
  • src/lib/tests/unit/prompt-loader.test.ts — 7 tests
  • src/lib/tests/unit/chat-agent.test.ts — 7 tests (new file, previously untested)
  • prompts/ — 9 template files extracted from source

Modified

  • src/lib/analyzer.ts, chat/agent.ts, report/summary.ts, report-highlights/service.ts, llm-config/service.ts — use loadPrompt + env var settings
  • src/app/settings/page.tsx — new per-service settings grid, secrets display
  • next.config.ts, README.md, CLAUDE.md — config and documentation

Test plan

  • All 264 tests pass (12 new)
  • 8 snapshot tests verify prompts are character-for-character identical after migration
  • npm run build succeeds
  • Manual: verify Settings page displays new sections
  • Manual: run a report to confirm LLM calls work as before

PavelLoparev and others added 20 commits March 21, 2026 10:34
Design for extracting hardcoded LLM prompts into template files (prompts dir)
and LLM settings (temperature, max_tokens) into env configuration.
Includes settings UI update to display all config values with masked secrets.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
21 tasks covering snapshot tests, prompt-loader, template files,
service migration, settings UI update, and documentation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Locks in current system prompt text, user message format, and LLM call
settings (temperature, max_tokens, model, response_format) before migrating
hardcoded prompts to external config files.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Reads prompt templates from PROMPTS_DIR (default: ./prompts), caches
file contents in-memory, and substitutes {{PLACEHOLDER}} variables.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
9 template files with {{PLACEHOLDER}} syntax for dynamic values.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…o env

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…gs to env

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…tings to env

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…o env

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds three new sections to LlmSettingsTab: per-service LLM settings table
(temperature/maxTokens per analyzer, chat agent, summary, highlights, test),
prompt templates directory, and masked secrets display. Also fixes a TS type
error in getLLMConfig() where the partial initial object was annotated as the
full LLMConfig type before required fields were assigned.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
  Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@PavelLoparev PavelLoparev requested a review from msogin March 21, 2026 09:26
PavelLoparev and others added 4 commits March 21, 2026 11:32
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…eads

Eliminates duplicated config reads across 5 services. All now use
getAppConfig() as single source of truth for settings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@msogin msogin merged commit 76f1e6e into main Mar 21, 2026
1 check passed
msogin added a commit that referenced this pull request Mar 21, 2026
This reverts commit 76f1e6e, reversing
changes made to 3f2044a.
PavelLoparev added a commit that referenced this pull request Mar 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants