Skip to content

Commit 9ce7288

Browse files
ScarletCarpetScarletCarpet
committed
feat(llm): add top_p, top_k, temperature sampling parameters
Brief: Add three commonly-used LLM sampling parameters (top_p, top_k, temperature) with support from both CLI flags and provider config. This improves the stability of LLM outputs but requires considering the trade-offs. Test: - test on deepseekv4 with temp=0.0 and top_p=0.8 and review the same code it got 80% similarity of output. - there is no param field on http request to LLM without customlize temp/topp/topk Change Log: - Add fields to ChatRequest, ClientConfig, and both OpenAI/Anthropic clients - Add ApplyDefaults() to fill unset fields from ClientConfig - Add provider config fields for built-in and custom providers - Add CLI flags --top-p, --top-k, --temperature for review and scan commands - Add config set support for providers.<name>.top_p, top_k, temperature - Propagate params through agent.Args → llmloop.Deps → ChatRequest - Update all ChatRequest call sites (plan, main loop, compression, dedup, summary, relocation, llm test) - Update README.md and README.zh-CN.md with new configuration items and CLI flags - Add unit tests for ApplyDefaults, build params, resolver, config set, and flag parsing Discussion: - Suggest deprecating the LLM config, as it duplicates the provider config - One way to set the temperature without this commit is via extra_body, but it is less elegant - Suggest deprecating other languages' READMEs, keeping only English and Chinese to reduce the maintenance workload
1 parent 6439846 commit 9ce7288

20 files changed

Lines changed: 649 additions & 33 deletions

‎README.md‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -211,6 +211,9 @@ Optional settings:
211211
| Key | Description |
212212
|-----|-------------|
213213
| `providers.<name>.auth_header` | Auth header: `x-api-key` or `authorization` (default: `authorization`) |
214+
| `providers.<name>.top_p` | Nucleus sampling parameter (0.0–1.0) |
215+
| `providers.<name>.top_k` | Top-K sampling parameter (non-negative integer) |
216+
| `providers.<name>.temperature` | Temperature parameter (0.0–2.0) |
214217
| `providers.<name>.extra_body` | Custom JSON fields merged into the request body |
215218
| `providers.<name>.extra_headers` | Comma-separated `key=value` pairs of custom HTTP headers added to every request |
216219
| `providers.<name>.models` | Model list for interactive selection |
@@ -440,6 +443,9 @@ See the [`examples/`](./examples/) directory for integration examples:
440443
| `--audience` | — | `human` | `human` (show progress) or `agent` (summary only) |
441444
| `--background` | `-b` | — | Optional requirement/business context for the review; auto-filled from commit message when using `--commit` |
442445
| `--model` | — | — | Select or override the LLM model for this review |
446+
| `--top-p` | — | 0 | Nucleus sampling parameter top_p (0 = use provider config default) |
447+
| `--top-k` | — | 0 | Top-K sampling parameter (0 = use provider config default) |
448+
| `--temperature` | — | 0 | Temperature parameter (0 = use provider config default) |
443449
| `--rule` | — | — | Path to custom JSON review rules |
444450
| `--max-tools` | — | built-in | Max tool call rounds per file; only takes effect when greater than template default |
445451
| `--max-git-procs` | — | built-in | Max concurrent git subprocesses |
@@ -464,6 +470,9 @@ non-git directories too (it falls back to a filesystem walk that honors `.gitign
464470
| `--format` | `-f` | `text` | Output format: `text` or `json` (JSON includes a `project_summary` field) |
465471
| `--concurrency` | — | `8` | Max concurrent file scans |
466472
| `--rule` | — | — | Path to custom JSON review rules |
473+
| `--top-p` | — | 0 | Nucleus sampling parameter top_p (0 = use provider config default) |
474+
| `--top-k` | — | 0 | Top-K sampling parameter (0 = use provider config default) |
475+
| `--temperature` | — | 0 | Temperature parameter (0 = use provider config default) |
467476
| `--repo` | — | current dir | Repository or directory root to scan |
468477

469478
Before each run, `ocr scan` prints a rough token-cost estimate. Use `--preview` to see the
@@ -497,6 +506,10 @@ ocr review --commit abc123 --format json --audience agent
497506
ocr review --model claude-opus-4-6
498507
ocr review --commit abc123 --model claude-sonnet-4-6
499508

509+
# Adjust LLM sampling parameters
510+
ocr review --top-p 0.9 --temperature 0.3
511+
ocr scan --top-p 0.95 --top-k 50 --temperature 0.5
512+
500513
# Provide requirement context for more targeted review
501514
ocr review --background "Adding rate limiting to the login API"
502515

@@ -667,6 +680,9 @@ Config file: `~/.opencodereview/config.json`
667680
| `providers.<name>.model` | string | Model name for the provider |
668681
| `providers.<name>.models` | array | Optional provider model list for interactive selection |
669682
| `providers.<name>.auth_header` | string | `x-api-key` \| `authorization` |
683+
| `providers.<name>.top_p` | float | Nucleus sampling parameter (0.0–1.0) |
684+
| `providers.<name>.top_k` | integer | Top-K sampling parameter (non-negative) |
685+
| `providers.<name>.temperature` | float | Temperature parameter (0.0–2.0) |
670686
| `providers.<name>.extra_body` | object | JSON object merged into every request body |
671687
| `providers.<name>.timeout_sec` | integer | Per-request HTTP timeout in seconds (default: `300`) |
672688
| `providers.<name>.extra_headers` | string | Comma-separated `key=value` HTTP headers |

‎README.zh-CN.md‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -211,6 +211,9 @@ ocr config set custom_providers.my-gateway.model gpt-4o
211211
| 键 | 描述 |
212212
|----|------|
213213
| `providers.<name>.auth_header` | 认证头：`x-api-key` 或 `authorization`（默认 `authorization`） |
214+
| `providers.<name>.top_p` | Nucleus 采样参数 (0.0–1.0) |
215+
| `providers.<name>.top_k` | Top-K 采样参数（非负整数） |
216+
| `providers.<name>.temperature` | 温度参数 (0.0–2.0) |
214217
| `providers.<name>.extra_body` | 合并到请求体的自定义 JSON 字段 |
215218
| `providers.<name>.extra_headers` | 逗号分隔的 `key=value` 键值对，为每个请求添加自定义 HTTP 头 |
216219
| `providers.<name>.models` | 用于交互式选择的模型列表 |
@@ -438,6 +441,9 @@ ocr review \
438441
| `--audience` | — | `human` | `human`（显示进度）或 `agent`（仅输出摘要） |
439442
| `--background` | `-b` | — | 可选的需求/业务背景信息；使用 `--commit` 时如未指定则自动从 commit message 中提取 |
440443
| `--model` | — | — | 为本次审查选择或覆盖 LLM 模型 |
444+
| `--top-p` | — | 0 | Nucleus 采样参数 top_p（0 = 使用供应商配置默认值） |
445+
| `--top-k` | — | 0 | Top-K 采样参数（0 = 使用供应商配置默认值） |
446+
| `--temperature` | — | 0 | 温度参数（0 = 使用供应商配置默认值） |
441447
| `--rule` | — | — | 自定义 JSON 审查规则路径 |
442448
| `--max-tools` | — | 内置默认 | 每个文件的最大工具调用轮次；仅在大于模板默认值时生效 |
443449
| `--max-git-procs` | — | 内置默认 | 最大并发 git 子进程数 |
@@ -459,6 +465,9 @@ ocr review \
459465
| `--batch` | — | `by-language` | 批处理策略：`none`、`by-language` 或 `by-directory` |
460466
| `--format` | `-f` | `text` | 输出格式：`text` 或 `json`（JSON 包含 `project_summary` 字段） |
461467
| `--concurrency` | — | `8` | 最大并发文件扫描数 |
468+
| `--top-p` | — | 0 | Nucleus 采样参数 top_p（0 = 使用供应商配置默认值） |
469+
| `--top-k` | — | 0 | Top-K 采样参数（0 = 使用供应商配置默认值） |
470+
| `--temperature` | — | 0 | 温度参数（0 = 使用供应商配置默认值） |
462471
| `--rule` | — | — | 自定义 JSON 审查规则路径 |
463472
| `--repo` | — | 当前目录 | 要扫描的仓库或目录根路径 |
464473

@@ -492,6 +501,10 @@ ocr review --commit abc123 --format json --audience agent
492501
ocr review --model claude-opus-4-6
493502
ocr review --commit abc123 --model claude-sonnet-4-6
494503

504+
# 修改模型采样参数
505+
ocr review --top-p 0.9 --temperature 0.3
506+
ocr scan --top-p 0.95 --top-k 50 --temperature 0.5
507+
495508
# 提供需求背景以获得更有针对性的审查
496509
ocr review --background "为登录 API 添加限流"
497510

@@ -652,6 +665,9 @@ OCR 通过四层优先级链解析评审规则。每层采用首次匹配原则
652665
| `providers.<name>.model` | string | 供应商模型名称 |
653666
| `providers.<name>.models` | array | 用于交互式选择的可选供应商模型列表 |
654667
| `providers.<name>.auth_header` | string | `x-api-key` \| `authorization` |
668+
| `providers.<name>.top_p` | float | Nucleus 采样参数 (0.0–1.0) |
669+
| `providers.<name>.top_k` | integer | Top-K 采样参数（非负整数） |
670+
| `providers.<name>.temperature` | float | 温度参数 (0.0–2.0) |
655671
| `providers.<name>.extra_body` | object | 合并到每个请求体的 JSON 对象 |
656672
| `providers.<name>.timeout_sec` | integer | 每次请求的 HTTP 超时时间（秒），默认 `300` |
657673
| `providers.<name>.extra_headers` | string | 逗号分隔的 `key=value` HTTP 头 |

‎cmd/opencodereview/config_cmd.go‎

Lines changed: 32 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -194,6 +194,9 @@ type ProviderEntry struct {
194194
Model string `json:"model,omitempty"`
195195
Models []string `json:"models,omitempty"`
196196
AuthHeader string `json:"auth_header,omitempty"`
197+
TopP *float64 `json:"top_p,omitempty"`
198+
TopK *int `json:"top_k,omitempty"`
199+
Temperature *float64 `json:"temperature,omitempty"`
197200
ExtraBody map[string]any `json:"extra_body,omitempty"`
198201
ExtraHeaders map[string]string `json:"extra_headers,omitempty"`
199202
}
@@ -374,7 +377,7 @@ func setConfigValue(cfg *Config, key, value string) error {
374377
}
375378
cfg.Llm.ExtraBody = m
376379
default:
377-
return fmt.Errorf("unknown config key: %s\nSupported keys: provider, model, providers.<name>.<field>, custom_providers.<name>.<field>, mcp_servers.<name>.<field>, llm.url, llm.auth_token, llm.auth_header, llm.model, llm.use_anthropic, llm.extra_body, llm.extra_headers, language, telemetry.enabled, telemetry.exporter, telemetry.otlp_endpoint, telemetry.content_logging\nProvider fields: api_key, url, protocol, model, models, auth_header, extra_body, extra_headers\nMCP server fields: command, args, env, tools, setup", key)
380+
return fmt.Errorf("unknown config key: %s\nSupported keys: provider, model, providers.<name>.<field>, custom_providers.<name>.<field>, mcp_servers.<name>.<field>, llm.url, llm.auth_token, llm.auth_header, llm.model, llm.use_anthropic, llm.extra_body, llm.extra_headers, language, telemetry.enabled, telemetry.exporter, telemetry.otlp_endpoint, telemetry.content_logging\nProvider fields: api_key, url, protocol, model, models, auth_header, top_p, top_k, temperature, extra_body, extra_headers\nMCP server fields: command, args, env, tools, setup", key)
378381
}
379382
return nil
380383
}
@@ -416,8 +419,35 @@ func applyProviderField(entry *ProviderEntry, field, key, value string) error {
416419
return fmt.Errorf("invalid extra headers for %s: %w", key, err)
417420
}
418421
entry.ExtraHeaders = parsed
422+
case "top_p":
423+
f, err := strconv.ParseFloat(value, 64)
424+
if err != nil {
425+
return fmt.Errorf("invalid float for %s: %w", key, err)
426+
}
427+
if f <= 0 || f > 1 {
428+
return fmt.Errorf("top_p must be between 0 (exclusive) and 1, got %f", f)
429+
}
430+
entry.TopP = &f
431+
case "top_k":
432+
i, err := strconv.Atoi(value)
433+
if err != nil {
434+
return fmt.Errorf("invalid integer for %s: %w", key, err)
435+
}
436+
if i < 0 {
437+
return fmt.Errorf("top_k must be non-negative, got %d", i)
438+
}
439+
entry.TopK = &i
440+
case "temperature":
441+
f, err := strconv.ParseFloat(value, 64)
442+
if err != nil {
443+
return fmt.Errorf("invalid float for %s: %w", key, err)
444+
}
445+
if f < 0 || f > 2 {
446+
return fmt.Errorf("temperature must be between 0 and 2, got %f", f)
447+
}
448+
entry.Temperature = &f
419449
default:
420-
return fmt.Errorf("unknown provider field %q: supported fields are api_key, url, protocol, model, models, auth_header, extra_body, extra_headers", field)
450+
return fmt.Errorf("unknown provider field %q: supported fields are api_key, url, protocol, model, models, auth_header, top_p, top_k, temperature, extra_body, extra_headers", field)
421451
}
422452
return nil
423453
}

‎cmd/opencodereview/config_cmd_test.go‎

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -838,6 +838,98 @@ func TestSetConfigValueProviderClearsModel(t *testing.T) {
838838
}
839839
}
840840

841+
func TestSetConfigValueProviderTopP(t *testing.T) {
842+
cfg := &Config{
843+
Provider: "anthropic",
844+
Providers: map[string]ProviderEntry{"anthropic": {}},
845+
}
846+
if err := setConfigValue(cfg, "providers.anthropic.top_p", "0.9"); err != nil {
847+
t.Fatalf("setConfigValue: %v", err)
848+
}
849+
entry := cfg.Providers["anthropic"]
850+
if entry.TopP == nil || *entry.TopP != 0.9 {
851+
t.Errorf("TopP = %v, want 0.9", entry.TopP)
852+
}
853+
}
854+
855+
func TestSetConfigValueProviderTopK(t *testing.T) {
856+
cfg := &Config{
857+
Provider: "anthropic",
858+
Providers: map[string]ProviderEntry{"anthropic": {}},
859+
}
860+
if err := setConfigValue(cfg, "providers.anthropic.top_k", "50"); err != nil {
861+
t.Fatalf("setConfigValue: %v", err)
862+
}
863+
entry := cfg.Providers["anthropic"]
864+
if entry.TopK == nil || *entry.TopK != 50 {
865+
t.Errorf("TopK = %v, want 50", entry.TopK)
866+
}
867+
}
868+
869+
func TestSetConfigValueProviderTemperature(t *testing.T) {
870+
cfg := &Config{
871+
Provider: "anthropic",
872+
Providers: map[string]ProviderEntry{"anthropic": {}},
873+
}
874+
if err := setConfigValue(cfg, "providers.anthropic.temperature", "0.3"); err != nil {
875+
t.Fatalf("setConfigValue: %v", err)
876+
}
877+
entry := cfg.Providers["anthropic"]
878+
if entry.Temperature == nil || *entry.Temperature != 0.3 {
879+
t.Errorf("Temperature = %v, want 0.3", entry.Temperature)
880+
}
881+
}
882+
883+
func TestSetConfigValueProviderTopPOutOfRange(t *testing.T) {
884+
cfg := &Config{
885+
Provider: "anthropic",
886+
Providers: map[string]ProviderEntry{"anthropic": {}},
887+
}
888+
if err := setConfigValue(cfg, "providers.anthropic.top_p", "1.5"); err == nil {
889+
t.Fatal("expected error for top_p > 1")
890+
}
891+
if err := setConfigValue(cfg, "providers.anthropic.top_p", "0"); err == nil {
892+
t.Fatal("expected error for top_p == 0 (must be > 0)")
893+
}
894+
if err := setConfigValue(cfg, "providers.anthropic.top_p", "-0.1"); err == nil {
895+
t.Fatal("expected error for negative top_p")
896+
}
897+
}
898+
899+
func TestSetConfigValueProviderTopKNegative(t *testing.T) {
900+
cfg := &Config{
901+
Provider: "anthropic",
902+
Providers: map[string]ProviderEntry{"anthropic": {}},
903+
}
904+
if err := setConfigValue(cfg, "providers.anthropic.top_k", "-1"); err == nil {
905+
t.Fatal("expected error for negative top_k")
906+
}
907+
}
908+
909+
func TestSetConfigValueProviderTemperatureOutOfRange(t *testing.T) {
910+
cfg := &Config{
911+
Provider: "anthropic",
912+
Providers: map[string]ProviderEntry{"anthropic": {}},
913+
}
914+
if err := setConfigValue(cfg, "providers.anthropic.temperature", "3.0"); err == nil {
915+
t.Fatal("expected error for temperature > 2")
916+
}
917+
}
918+
919+
func TestSetConfigValueCustomProviderTopP(t *testing.T) {
920+
cfg := &Config{
921+
Provider: "my-gateway",
922+
CustomProviders: map[string]ProviderEntry{"my-gateway": {}},
923+
}
924+
if err := setConfigValue(cfg, "custom_providers.my-gateway.top_p", "0.8"); err != nil {
925+
t.Fatalf("setConfigValue: %v", err)
926+
}
927+
entry := cfg.CustomProviders["my-gateway"]
928+
if entry.TopP == nil || *entry.TopP != 0.8 {
929+
t.Errorf("TopP = %v, want 0.8", entry.TopP)
930+
}
931+
}
932+
841933
func TestRunConfigUnset_InvalidKey(t *testing.T) {
842934
if err := runConfigUnset("provider"); err == nil {
843935
t.Fatal("expected error for non custom_providers key")

‎cmd/opencodereview/flags.go‎

Lines changed: 25 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,10 @@ func (a *ocrFlagSet) IntVar(p *int, name string, value int, usage string) {
5454
a.fs.IntVar(p, name, value, usage)
5555
}
5656

57+
func (a *ocrFlagSet) Float64Var(p *float64, name string, value float64, usage string) {
58+
a.fs.Float64Var(p, name, value, usage)
59+
}
60+
5761
func (a *ocrFlagSet) DurationVar(p *time.Duration, name string, value time.Duration, usage string) {
5862
a.fs.DurationVar(p, name, value, usage)
5963
}
@@ -106,6 +110,9 @@ type reviewOptions struct {
106110
audience string // --audience: "human" (default) or "agent"
107111
background string // --background: optional requirement context
108112
model string // --model: override resolved LLM model for this review
113+
topP float64
114+
topK int
115+
temperature float64
109116
concurrency int
110117
perFileTimeout int
111118
maxTools int
@@ -132,6 +139,9 @@ func parseReviewFlags(args []string) (reviewOptions, error) {
132139
a.StringVar(&opts.audience, "audience", "human", "output audience: human (show progress) or agent (summary only)")
133140
a.StringVarP(&opts.background, "background", "b", "", "optional requirement/business context for the review")
134141
a.StringVar(&opts.model, "model", "", "override LLM model for this review (e.g., claude-opus-4-6)")
142+
a.Float64Var(&opts.topP, "top-p", -1, "nucleus sampling parameter top_p (-1 = use provider default)")
143+
a.IntVar(&opts.topK, "top-k", -1, "top-k sampling parameter (-1 = use provider default)")
144+
a.Float64Var(&opts.temperature, "temperature", -1, "temperature parameter (-1 = use provider default)")
135145
a.IntVar(&opts.maxTools, "max-tools", 0, "max tool call rounds per file (0 = template default; min 10)")
136146
a.IntVar(&opts.maxGitProcs, "max-git-procs", 16, "max concurrent git subprocesses")
137147
a.BoolVarP(&opts.preview, "preview", "p", false, "preview which files will be reviewed without running the LLM")
@@ -182,6 +192,16 @@ func parseReviewFlags(args []string) (reviewOptions, error) {
182192
return opts, fmt.Errorf("--max-git-procs must be a non-negative integer (0 means use default 16)")
183193
}
184194

195+
if opts.topP > 1 {
196+
return opts, fmt.Errorf("--top-p must be <= 1, got %f", opts.topP)
197+
}
198+
if opts.temperature > 0 && opts.temperature > 2 {
199+
return opts, fmt.Errorf("--temperature must be <= 2, got %f", opts.temperature)
200+
}
201+
if opts.topK > 0 && opts.topK > 100 {
202+
fmt.Fprintf(os.Stderr, "[ocr] --top-k %d is unusually large, expected <= 100\n", opts.topK)
203+
}
204+
185205
return opts, nil
186206
}
187207

@@ -229,7 +249,10 @@ Flags:
229249
--rule string path to JSON file with system review rules
230250
--timeout int concurrent task timeout in minutes (default 10)
231251
--to string target ref to end diff at (e.g., 'feature-branch')
232-
--tools string path to JSON tools config file (default: embedded)`)
252+
--tools string path to JSON tools config file (default: embedded)
253+
--top-p float nucleus sampling parameter top_p (0 = use provider default)
254+
--top-k int top-k sampling parameter (0 = use provider default)
255+
--temperature float temperature parameter (0 = use provider default)`)
233256
}
234257

235258
// --- config subcommand ---
@@ -320,6 +343,6 @@ Examples:
320343
ocr config set telemetry.enabled true
321344
322345
Supported keys: provider, model, providers.<name>.<field>, custom_providers.<name>.<field>, mcp_servers.<name>.<field>, llm.url, llm.auth_token, llm.auth_header, llm.model, llm.use_anthropic, llm.extra_body, llm.extra_headers, language, telemetry.enabled, telemetry.exporter, telemetry.otlp_endpoint, telemetry.content_logging
323-
Provider fields: api_key, url, protocol, model, models, auth_header, extra_body, extra_headers
346+
Provider fields: api_key, url, protocol, model, models, auth_header, top_p, top_k, temperature, extra_body, extra_headers
324347
MCP server fields: command, args, env, tools, setup`)
325348
}

0 commit comments

Comments
 (0)