Add --sampling-params to response regeneration script (#724)

<!-- markdownlint-disable -->

<!-- PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS
(AT THE BOTTOM) HAVE BEEN CONSIDERED. -->

## Purpose

Response regeneration requests only set max_tokens, so sampling behavior
(temperature, top_p, seed) was stuck on server defaults with no way to
customize. Adds a --sampling-params JSON flag merged into each
chat-completion request, records it in output metadata for
reproducibility, and documents it in the CLI reference. Part of #694.

## Tests

- Sanity-checked arg parsing: --sampling-params '{"temperature": 0.6,
"top_p": 0.95}' parses to a dict, omitted flag yields {}, non-object
JSON errors out

## Checklist

I have filled in:

- [x] The purpose of the PR, such as "Fix some issue (link existing
issues this PR will resolve)".
- [x] The test plan/results, such as providing test command and pasting
the results.
- [ ] (Optional) The necessary documentation update.
- [x] I (a human) have written or reviewed the code in this pr to the
best of my ability.

---------

Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>
Signed-off-by: Guan-Ming Chiu <105915352+guan404ming@users.noreply.github.com>
Co-authored-by: Orestis Zambounis <23146389+orestis-z@users.noreply.github.com>

Author: guan404ming

docs/cli/response_regeneration.md

@@ -89,6 +89,8 @@ python scripts/response_regeneration/script.py --dataset magpie
 
 - **`--max-tokens`** (int, default: `8192`) Max tokens for generation.
 
+- **`--sampling-params`** (str, default: `None`) JSON object merged into each chat-completion request, e.g. `'{"temperature": 0.6, "top_p": 0.95, "seed": 0}'`. Unset keys use the server defaults.
+
 #### Output Arguments
 
 - **`--outfile`** (str, default: auto-generated) Output JSONL path. If not specified, auto-generated as `{dataset}_{model}.jsonl`.
@@ -133,7 +135,8 @@ Rows are pre-tokenized and ready for training: one row per assistant turn, holdi
     "idx": 0,
     "finish_reason": "stop",
     "usage": {...},
-    "endpoint": "http://127.0.0.1:8000/v1/chat/completions"
+    "endpoint": "http://127.0.0.1:8000/v1/chat/completions",
+    "sampling_params": {...}
   }
 }
 ```

scripts/response_regeneration/script.py

@@ -85,6 +85,14 @@ def parse_args():
         default=8192,
         help="max_tokens for generation",
     )
+    parser.add_argument(
+        "--sampling-params",
+        default=None,
+        help=(
+            "JSON object merged into each chat-completion request, "
+            'e.g. \'{"temperature": 0.6, "top_p": 0.95, "seed": 0}\''
+        ),
+    )
     parser.add_argument(
         "--outfile",
         default=None,
@@ -112,6 +120,14 @@ def parse_args():
     args = parser.parse_args()
     if args.max_retries < 0:
         parser.error("--max-retries must be >= 0")
+    try:
+        args.sampling_params = (
+            json.loads(args.sampling_params) if args.sampling_params else {}
+        )
+    except json.JSONDecodeError as e:
+        parser.error(f"--sampling-params is not valid JSON: {e}")
+    if not isinstance(args.sampling_params, dict):
+        parser.error("--sampling-params must be a JSON object")
     return args
 
 
@@ -326,6 +342,7 @@ async def worker(
                 prefix.append({"role": "user", "content": turn["content"]})
 
                 payload = {
+                    **args.sampling_params,
                     "model": args.model,
                     "messages": prefix,
                     "max_tokens": args.max_tokens,
@@ -374,6 +391,7 @@ async def worker(
                             "finish_reason": choice.get("finish_reason"),
                             "usage": data.get("usage") or {},
                             "endpoint": endpoint,
+                            "sampling_params": args.sampling_params,
                         },
                     }
                 )

tests/unit/scripts/test_response_regeneration.py

@@ -11,6 +11,7 @@
 import importlib.util
 import json
 from pathlib import Path
+from typing import Any
 
 import pytest
 
@@ -434,6 +435,7 @@ class _Args:
     model = "m"
     max_tokens = 16
     max_retries = 0
+    sampling_params: dict[str, Any] = {}
 
 
 class _NullProgress: