Skip to content
Merged
Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
c032f71
feat(enricher): add enriched context fields to FieldContext, ParamFie…
spencercjh Mar 30, 2026
8c2a917
feat(enricher): populate enriched context (format, enum, constraints)…
spencercjh Mar 30, 2026
fe4d0ec
feat(enricher): populate enriched context (tags, format, enum, constr…
spencercjh Mar 30, 2026
c0daed4
feat(enricher): rewrite built-in prompts with type-specific system pr…
spencercjh Mar 30, 2026
27f943d
feat(enricher): add custom prompt configuration and wire through enri…
spencercjh Mar 30, 2026
d26aa7d
chore: fix lint issues and add custom prompts integration test
spencercjh Mar 30, 2026
9dfba58
docs: add P5 prompt optimization design and implementation plan
spencercjh Mar 31, 2026
1f8db91
refactor(enricher): extract shared BuildConstraintsString and BuildEn…
spencercjh Mar 31, 2026
288deda
fix(enricher): warn on invalid custom prompt type keys
spencercjh Mar 31, 2026
fdf9c39
feat(enricher): validate custom prompt templates at startup
spencercjh Mar 31, 2026
b25b564
refactor(cmd): deduplicate custom prompt mapping with shared helper
spencercjh Mar 31, 2026
cd16e54
fix(enricher): remove format from FieldType to avoid duplicate in prompt
spencercjh Mar 31, 2026
3b12f39
fix(enricher): populate ExistingDescription for parameter field items
spencercjh Mar 31, 2026
d954451
docs: update implementation plan to reflect shared helper reuse and r…
spencercjh Mar 31, 2026
b5ed3e4
fix(enricher): merge custom prompts with built-in templates and rejec…
spencercjh Mar 31, 2026
556716c
refactor(enricher): extract applyCustomPrompts to reduce Enrich cyclo…
spencercjh Mar 31, 2026
bdc1da2
fix(enricher): remove unsafe quoting of ExistingDescription in templa…
spencercjh Mar 31, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .spec-forge.example.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,16 @@ enrich:
language: zh
# Request timeout
timeout: 60s
# Custom prompt templates (optional)
# Override built-in prompt templates for any type: api, schema, param, response
# Supports Go template syntax (same as built-in templates)
# customPrompts:
# api:
# system: "You are a REST API documentation writer..."
# user: "API: {{.Method}} {{.Path}}\nDescribe this endpoint."
# schema:
# system: "You are a data model documenter..."
Comment thread
spencercjh marked this conversation as resolved.
# user: "Schema: {{.SchemaName}}\nDescribe this data model and its fields."

# Output Settings
output:
Expand Down
4 changes: 4 additions & 0 deletions cmd/enrich.go
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,9 @@ func runEnrich(cmd *cobra.Command, args []string) error {
customAPIKeyEnv = cfg.Enrich.APIKeyEnv
}

// Map custom prompts from config
customPrompts := enricher.CustomPromptsFromMap(cfg.Enrich.CustomPrompts)

enricherCfg := enricher.Config{
Provider: prov,
Model: model,
Expand All @@ -147,6 +150,7 @@ func runEnrich(cmd *cobra.Command, args []string) error {
Timeout: timeoutFlag,
CustomBaseURL: customBaseURL,
CustomAPIKeyEnv: customAPIKeyEnv,
CustomPrompts: customPrompts,
}
enricherCfg = enricherCfg.MergeWithDefaults()

Expand Down
4 changes: 4 additions & 0 deletions cmd/generate.go
Original file line number Diff line number Diff line change
Expand Up @@ -409,13 +409,17 @@ func enrichGeneratedSpec(ctx context.Context, specFilePath string, cfg *config.C
}
}

// Map custom prompts from config
customPrompts := enricher.CustomPromptsFromMap(cfg.Enrich.CustomPrompts)

// Create enricher config
enricherCfg := enricher.Config{
Provider: cfg.Enrich.Provider,
Model: cfg.Enrich.Model,
Language: lang,
Timeout: timeout,
CustomBaseURL: cfg.Enrich.BaseURL,
CustomPrompts: customPrompts,
}
enricherCfg = enricherCfg.MergeWithDefaults()

Expand Down
157 changes: 157 additions & 0 deletions docs/plans/2026-03-31-p5-prompt-optimization-design.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,157 @@
# P5 Prompt Optimization Design

> **Status:** Implemented
> **Date:** 2026-03-31
> **Parent Issue:** #40 (Phase 4 - LangchainGo Features)

## Overview

Improve LLM enrichment output quality by enriching context passing, rewriting built-in prompt templates, and adding custom prompt file support.

## Current State

| Feature | Status | Detail |
|-------------------|---------------|----------------------------------------------------------|
| Context passing | Minimal | Only name, type, required passed to templates |
| System prompts | Generic | All 4 types use "You are an API documentation expert" |
| Few-shot examples | None | Templates contain no input/output examples |
| Constraints/enums | Ignored | OpenAPI format, enum, min/max, pattern not passed to LLM |
| API tags | Ignored | Operation tags not included in context |
| Custom prompts | Not supported | No way to override built-in templates |

## Design

### Optimization 1: Enriched Context Passing

**Problem:** Templates receive only `Name`, `Type`, `Required` for fields/params. The OpenAPI spec contains much richer metadata that would help LLMs generate more precise descriptions.

**Solution:** Pass additional spec metadata to templates:

**FieldContext additions:**
- `Format` — e.g., `"email"`, `"date-time"`, `"uuid"`
- `Enum` — allowed values, e.g., `["active", "inactive"]`
- `Constraints` — human-readable string: `"min: 0, max: 100, pattern: ^[a-z]+$"`
- `ExistingDescription` — existing description from spec (useful in `--force` mode)

**ParamFieldContext additions:** Same as FieldContext.

**TemplateContext additions (API-specific):**
- `Tags` — operation tags from the spec
- `ExistingSummary` / `ExistingDescription` — existing partial documentation

**Example impact on Schema prompt:**

```
Before:
- email (string, required)

After:
- email (string, required, format: email, maxLength: 255)
- role (string, optional, enum: [admin, user, guest])
```

### Optimization 2: Improved Built-in Prompts

**Problem:** Generic system prompts produce generic descriptions. No examples, no quality guidelines, no output constraints.

**Solution:** Type-specific system prompts with:

1. **Role definition** — Different expert roles per type (API writer, data modeler, parameter documenter)
2. **Quality guidelines** — Specific rules per type (e.g., "Summary starts with a verb", "Avoid repeating field name")
3. **Few-shot examples** — Input/output pairs showing expected quality
4. **Explicit output format** — JSON schema with constraints

**API Template (before):**
```
System: You are an API documentation expert. Generate concise, clear descriptions.
Respond in {{.Language}} language.
Output format: JSON with "summary" and "description" fields.

User: API Endpoint: {{.Path}}
HTTP Method: {{.Method}}

Generate the summary (one line) and description (1-3 sentences) for this API.
```

**API Template (after):**
```
System: You are an expert OpenAPI documentation writer specializing in REST API descriptions.
Your task is to write clear, concise, and informative API summaries and descriptions.

Guidelines:
- Summary: A single line (max 80 chars) starting with a verb (e.g., "List", "Create", "Delete")
- Description: 1-3 sentences explaining what the endpoint does, when to use it, and notable behavior
- Be specific: mention resource names, ID formats, and key constraints
- Avoid generic phrases like "This API is used for..."

Respond in {{.Language}} language.
Output MUST be valid JSON: {"summary": "...", "description": "..."}

Example input:
POST /users
Example output:
{"summary": "Create a new user", "description": "Registers a new user account..."}

User: API Endpoint: {{.Method}} {{.Path}}
{{- if .Tags}}
Tags: {{join .Tags ", "}}
{{- end}}
...
```

### Optimization 3: Custom Prompt File Support

**Problem:** Users cannot customize prompts for their domain without modifying source code.

**Solution:** Add `customPrompts` section to `.spec-forge.yaml`:

```yaml
enrich:
customPrompts:
api:
system: "You are a Chinese API documentation writer..."
user: "API: {{.Method}} {{.Path}}\n用中文描述这个接口。"
schema:
system: "You are a data model expert..."
```

**Implementation:** Config loads custom prompts via Viper, passes through `enricher.Config.CustomPrompts`, and applies via `TemplateManager.Set()`.

## Architecture

```
┌─────────────────────┐
│ .spec-forge.yaml │
│ customPrompts: │
│ api/system/user │
└──────────┬──────────┘
┌──────────────┐ ┌─────────────────────┐ ┌──────────────┐
│ OpenAPI │───▶│ Collection Layer │───▶│ Template │
│ Spec │ │ (enricher.go) │ │ Manager │
│ │ │ - format │ │ │
│ - format │ │ - enum │ │ Built-in │
│ - enum │ │ - constraints │ │ + Custom │
│ - min/max │ │ - tags │ │ overrides │
│ - pattern │ │ - existing desc │ │ │
│ - tags │ └─────────────────────┘ └──────┬───────┘
└──────────────┘ │
┌──────────────────┐
│ LLM Provider │
│ (OpenAI/etc.) │
└──────────────────┘
```

## Key Decisions

1. **Backward compatible** — Output format stays the same (`{"summary": "...", "description": "..."}` for API, `{"field": "desc"}` for schema/param). No response parsing changes.

2. **Constraint helper reuse** — `enricher.go` reuses `processor.BuildConstraintsString` and `processor.BuildEnumStrings` to keep constraint/enum formatting logic centralized in the `processor` package, avoiding duplicated helpers.

3. **ExistingDescription in templates** — Only visible in `--force` mode (fields with existing descriptions are skipped otherwise). When force is on, the LLM can improve or translate existing descriptions.

4. **Template FuncMap** — Added `join` function (maps to `strings.Join`) for rendering enum/tag lists. Registered in `renderString` via `template.FuncMap`.

5. **Config key mapping** — Custom prompt keys (`"api"`, `"schema"`, `"param"`, `"response"`) directly match `TemplateType` string constants.
152 changes: 152 additions & 0 deletions docs/plans/2026-03-31-p5-prompt-optimization-implementation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,152 @@
# P5 Prompt Optimization Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Improve LLM enrichment quality by enriching context passing, rewriting built-in prompts with few-shot examples, and adding custom prompt file support.

**Architecture:** Three-layer improvement: (1) pass more OpenAPI spec metadata (format, enum, constraints, tags) to templates, (2) rewrite templates with type-specific system prompts and examples, (3) allow users to override prompts via `.spec-forge.yaml`. Output format stays the same — no response parsing changes needed.

**Tech Stack:** Go 1.26, text/template, Viper config, kin-openapi

---

## File Structure

| File | Responsibility |
|----------------------------------------------|-------------------------------------------------------------|
| `internal/enricher/prompt/templates.go` | Context types, built-in templates, FuncMap, TemplateManager |
| `internal/enricher/prompt/templates_test.go` | Template rendering tests |
| `internal/enricher/processor/schema.go` | Schema field collection with enriched metadata |
| `internal/enricher/processor/processor.go` | FieldElement/ParamFieldItem types, conversion helpers |
| `internal/enricher/enricher.go` | Parameter collection, API context, custom prompt wiring |
| `internal/enricher/config.go` | CustomPrompts field on enricher Config |
| `internal/config/config.go` | CustomPrompts on EnrichConfig |
| `cmd/enrich.go` | Wire custom prompts from config to enricher |
| `cmd/generate.go` | Wire custom prompts in generate pipeline |
| `.spec-forge.example.yaml` | Document new config option |

---

### Task 1: Enrich FieldContext, ParamFieldContext, and TemplateContext types

**Files:**
- Modify: `internal/enricher/prompt/templates.go`
- Modify: `internal/enricher/prompt/templates_test.go`

- [x] Add enriched fields to `FieldContext` (Format, Enum, Constraints, ExistingDescription)
- [x] Add enriched fields to `ParamFieldContext` (Format, Enum, Constraints, ExistingDescription)
- [x] Add Tags, ExistingSummary, ExistingDescription to `TemplateContext`
- [x] Add `join` func to `renderString` FuncMap
- [x] Add tests for enriched field rendering
- [x] Commit: `feat(enricher): add enriched context fields to FieldContext, ParamFieldContext, and TemplateContext`

---

### Task 2: Populate enriched context in schema field collection

**Files:**
- Modify: `internal/enricher/processor/processor.go`
- Modify: `internal/enricher/processor/schema.go`
- Test: `internal/enricher/processor/schema_test.go`

- [x] Add enriched fields to `FieldElement` and `ParamFieldItem`
- [x] Update `convertFieldElements` and `convertParamFieldItems` to propagate enriched fields
- [x] Add `buildConstraintsString` and `buildEnumStrings` helpers to schema.go
- [x] Update `CollectSchemaFields` to populate Format, Enum, Constraints, ExistingDescription
- [x] Add `TestCollectSchemaFields_EnrichedContext`
- [x] Commit: `feat(enricher): populate enriched context (format, enum, constraints) in schema field collection`

---

### Task 3: Populate enriched context in parameter and API collection

**Files:**
- Modify: `internal/enricher/enricher.go`
- Modify: `internal/enricher/enricher_test.go`

- [x] Reuse shared `processor.BuildConstraintsString` and `processor.BuildEnumStrings` helpers for parameter constraint/enum extraction
- [x] Update `collectParameterGroups` to extract format, enum, constraints from param schemas
- [x] Update `collectElements` to pass Tags, ExistingSummary, ExistingDescription for API operations
- [x] Add `TestEnricher_CollectParameters_EnrichedContext` and `TestEnricher_CollectElements_APITags`
- [x] Commit: `feat(enricher): populate enriched context (tags, format, enum, constraints) in parameter and API collection`

---

### Task 4: Rewrite built-in prompt templates

**Files:**
- Modify: `internal/enricher/prompt/templates.go`
- Modify: `internal/enricher/prompt/templates_test.go`

- [x] Replace all 4 templates with type-specific system prompts, few-shot examples, quality guidelines
- [x] API template: verb-led summaries, specificity guidelines, tags/existing desc support
- [x] Schema template: constraint-aware descriptions, enum explanation guidance
- [x] Param template: location context, enum guidance, format hints
- [x] Response template: error cause guidance, success content hints
- [x] Add `TestNewTemplateManager_RendersAllTypesWithEnrichedContext` and `TestNewTemplateManager_APITemplateUsesTags`
- [x] Commit: `feat(enricher): rewrite built-in prompts with type-specific system prompts, few-shot examples, and enriched context`

---

### Task 5: Add custom prompt config

**Files:**
- Modify: `internal/config/config.go`
- Modify: `internal/enricher/config.go`
- Modify: `.spec-forge.example.yaml`

- [x] Add `CustomPrompts map[string]CustomPromptCfg` to `config.EnrichConfig`
- [x] Add `CustomPromptConfig` type and `CustomPrompts` field to `enricher.Config`
- [x] Update `.spec-forge.example.yaml` with commented customPrompts section

---

### Task 6: Wire custom prompts through enricher pipeline

**Files:**
- Modify: `internal/enricher/enricher.go`
- Modify: `cmd/enrich.go`
- Modify: `cmd/generate.go`

- [x] Apply custom prompts via `TemplateManager.Set()` in `Enrich()` method
- [x] Map custom prompts from config in `cmd/enrich.go`
- [x] Map custom prompts from config in `cmd/generate.go`
- [x] Commit: `feat(enricher): add custom prompt configuration and wire through enricher pipeline`

---

### Task 7: Integration test and verification

**Files:**
- Modify: `internal/enricher/enricher_test.go`

- [x] Add `TestEnricher_CustomPrompts` with `trackingMockProvider`
- [x] Fix lint issues (perfsprint, gocritic rangeValCopy)
- [x] `make fmt`, `make lint` (0 issues), `make test` (all pass)
- [x] Commit: `chore: fix lint issues and add custom prompts integration test`

---

## Verification

```bash
# Build
go build -o ./build/spec-forge .

# Test with real LLM
LLM_API_KEY="your-key" ./build/spec-forge enrich \
./integration-tests/maven-springboot-openapi-demo/target/openapi.json \
--provider custom --model deepseek-chat \
--custom-base-url https://api.deepseek.com/v1 \
--language zh -v

# Test custom prompts via config
# Add to .spec-forge.yaml:
# enrich:
# customPrompts:
# api:
# system: "You are a Chinese API writer..."
# user: "Endpoint: {{.Method}} {{.Path}}\nWrite summary+description in JSON."
```

Expected: Descriptions leverage format/enum/constraint context for more specific output.
27 changes: 17 additions & 10 deletions internal/config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,16 +26,23 @@ type ReadMeConfig struct {

// EnrichConfig contains LLM enrichment settings.
type EnrichConfig struct {
Enabled bool `mapstructure:"enabled"`
Provider string `mapstructure:"provider"`
Model string `mapstructure:"model"`
Language string `mapstructure:"language"`
APIKey string `mapstructure:"apiKey"`
Headers map[string]string `mapstructure:"headers"`
BaseURL string `mapstructure:"baseUrl"`
APIKeyEnv string `mapstructure:"apiKeyEnv"`
Timeout string `mapstructure:"timeout"`
SkipEnrich bool `mapstructure:"skipEnrich"`
Enabled bool `mapstructure:"enabled"`
Provider string `mapstructure:"provider"`
Model string `mapstructure:"model"`
Language string `mapstructure:"language"`
APIKey string `mapstructure:"apiKey"`
Headers map[string]string `mapstructure:"headers"`
BaseURL string `mapstructure:"baseUrl"`
APIKeyEnv string `mapstructure:"apiKeyEnv"`
Timeout string `mapstructure:"timeout"`
SkipEnrich bool `mapstructure:"skipEnrich"`
CustomPrompts map[string]CustomPromptCfg `mapstructure:"customPrompts"`
}

// CustomPromptCfg holds custom system/user prompt overrides for a template type.
type CustomPromptCfg struct {
System string `mapstructure:"system"`
User string `mapstructure:"user"`
}

// OutputConfig contains output settings.
Expand Down
Loading
Loading