docs(regen): update response regeneration docs for multi-turn support

Brings docs in line with shipped code from PRs #693 (multi-turn),
--reasoning-parser). Adds gsm8k to supported datasets, documents
--subset arg, and adds a multi-turn output example.

Signed-off-by: Sawyer Bowerman <sbowerma@redhat.com>

Author: soyr-redhat

docs/cli/response_regeneration.md

@@ -1,6 +1,6 @@
 # response_regeneration
 
-Regenerates assistant responses in existing datasets using a vLLM-served model. Given a dataset containing user prompts (e.g., Magpie, UltraChat), this pipeline extracts the prompts, sends them to a vLLM server, and produces a new dataset with the original prompts paired with freshly generated responses from the target model. This is useful for creating training data where you want a specific model's outputs in place of the original assistant responses.
+Regenerates assistant responses in existing datasets using a vLLM-served model. Given a dataset containing conversations (e.g., Magpie, UltraChat, GSM8K), this pipeline extracts conversation turns, regenerates each assistant response turn-by-turn against the model's own prior outputs, and produces pre-tokenized training samples. For multi-turn conversations, each turn conditions on the regenerated history, producing on-policy training data.
 
 The pipeline consists of two scripts:
 
@@ -33,6 +33,10 @@ Orchestrates the entire pipeline: starts a vLLM server (with optional data/tenso
 
 - **`--tp-size`** (int) Tensor parallel size per replica (maps to vLLM's `--tensor-parallel-size`).
 
+- **`--max-model-len`** (int) Maximum model context length (passed to `vllm serve --max-model-len`).
+
+- **`--reasoning-parser`** (str) Reasoning parser for the vLLM server (passed to `vllm serve --reasoning-parser`).
+
 - **`--keep-server`** (flag) Don't stop the vLLM server after processing completes.
 
 All other arguments are passed through to `script.py`.
@@ -51,13 +55,15 @@ All other arguments are passed through to `script.py`.
 
 ## script.py
 
-Extracts user prompts from a dataset, sends them to a vLLM chat completion endpoint, and writes out new prompt-response pairs with the target model's generated responses.
+Extracts conversation turns from a dataset, regenerates each assistant response turn-by-turn via a vLLM chat completion endpoint, and writes out pre-tokenized training samples with generation boundaries marked in the loss mask.
 
 ### Features
 
+- **Multi-turn support** — detects `messages`/`conversations` fields and regenerates each assistant turn against the model's own prior responses
 - **Auto-detects model** from vLLM server (no need to specify `--model`)
-- **Resume capability** to skip already-processed rows
+- **Resume capability** to skip already-processed conversations
 - **Async processing** with configurable concurrency
+- **Automatic retries** with exponential backoff on transient failures
 
 ### Basic Usage
 
@@ -93,6 +99,8 @@ python scripts/response_regeneration/script.py --dataset magpie
 
 - **`--sampling-params`** (str, default: `None`) JSON object merged into each chat-completion request, e.g. `'{"temperature": 0.6, "top_p": 0.95, "seed": 0}'`. Unset keys use the server defaults.
 
+- **`--max-retries`** (int, default: `3`) Max retry attempts per request on transient HTTP failures (408, 409, 425, 429, 5xx) with exponential backoff. Permanent errors (e.g., 400, 404) fail immediately.
+
 #### Output Arguments
 
 - **`--outfile`** (str, default: auto-generated) Output JSONL path. If not specified, auto-generated as `{dataset}_{model}.jsonl`.

docs/user_guide/tutorials/response_regeneration.md

@@ -1,6 +1,6 @@
 # Response Regeneration
 
-This tutorial walks you through regenerating assistant responses in an existing dataset using a target model served by vLLM. The resulting dataset pairs the original user prompts with freshly generated responses, which can be used as training data for speculator models with improved alignment to your target model.
+This tutorial walks you through regenerating assistant responses in an existing dataset using a target model served by vLLM. For multi-turn conversations, the script regenerates each assistant turn sequentially against the model's own prior responses, producing on-policy training data with improved alignment to your target model.
 
 ## Overview
 
@@ -26,8 +26,8 @@ The simplest way to regenerate responses is using the `run_all.sh` script, which
 This will:
 
 1. Start a vLLM server with the specified model
-2. Extract prompts from the dataset and generate new responses
-3. Save results to a JSONL file (e.g., `magpie_Llama-3.3-70B-Instruct.jsonl`)
+2. Extract conversation turns from the dataset and regenerate assistant responses turn-by-turn
+3. Save pre-tokenized results to a JSONL file (e.g., `magpie_Llama-3.3-70B-Instruct.jsonl`)
 4. Stop the server
 
 ### Multi-GPU Configurations
@@ -71,6 +71,29 @@ The output is a JSONL file with one pre-tokenized row per assistant turn. `loss_
 }
 ```
 
+For multi-turn datasets, later turns include the regenerated history as context. For example, the second turn of the same conversation would be:
+
+```json
+{
+  "id": "conv-abc_turn1",
+  "primary_id": "conv-abc",
+  "input_ids": [151644, 872, ...],
+  "loss_mask": [0, 0, ..., 1, 1],
+  "conversations": [
+    {"role": "user", "content": "What is the capital of France?"},
+    {"role": "assistant", "content": "The capital of France is Paris."},
+    {"role": "user", "content": "What about Germany?"},
+    {"role": "assistant", "content": "The capital of Germany is Berlin."}
+  ],
+  "metadata": {
+    "idx": 0,
+    "finish_reason": "stop",
+    "usage": {...},
+    "endpoint": "http://127.0.0.1:8000/v1/chat/completions"
+  }
+}
+```
+
 A conversation with N assistant turns produces N rows, so expect more lines than input conversations. `conversations` is a review-only twin of `input_ids`; training drops it.
 
 Check that the output looks correct: