Skip to content

Commit b0be6d8

Browse files
committed
docs(regen): dataset choices come from the shared registry
The CLI doc's hand-copied dataset list had already drifted (gsm8k was never added); point at the shared registry instead of maintaining a third copy, and document the prepare-data-parity ingestion. Signed-off-by: Ranran Haoran Zhang <ranzhang@redhat.com>
1 parent 9db7928 commit b0be6d8

1 file changed

Lines changed: 15 additions & 6 deletions

File tree

docs/cli/response_regeneration.md

Lines changed: 15 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -69,9 +69,9 @@ python scripts/response_regeneration/script.py --dataset magpie
6969

7070
#### Data Arguments
7171

72-
- **`--dataset`** (str, default: `ultrachat`, choices: `magpie`, `ultrachat`) Dataset to process.
72+
- **`--dataset`** (str, default: `ultrachat`) Dataset preset to process (see [Supported Datasets](#supported-datasets)).
7373

74-
- **`--split`** (str, default: dataset-specific) Dataset split. Defaults to `train` for magpie and `train_sft` for ultrachat.
74+
- **`--split`** (str, default: preset-specific) Dataset split. Defaults to the preset's split.
7575

7676
- **`--limit`** (int, default: `None`) Stop after N rows.
7777

@@ -110,10 +110,19 @@ python scripts/response_regeneration/script.py \
110110

111111
## Supported Datasets
112112

113-
| Dataset | HuggingFace ID | Prompt Field | Default Split |
114-
| --------- | ------------------------------------------------- | ------------- | ------------- |
115-
| Magpie | `Magpie-Align/Magpie-Llama-3.1-Pro-300K-Filtered` | `instruction` | `train` |
116-
| UltraChat | `HuggingFaceH4/ultrachat_200k` | `prompt` | `train_sft` |
113+
All presets from the shared dataset registry (`DATASET_CONFIGS` in `speculators/data_generation/configs.py`) — the same ones `prepare-data` accepts:
114+
115+
| Dataset | HuggingFace ID | Default Split |
116+
| ------------------- | ------------------------------------------------- | ------------- |
117+
| `sharegpt` | `Aeala/ShareGPT_Vicuna_unfiltered` | `train` |
118+
| `ultrachat` | `HuggingFaceH4/ultrachat_200k` | `train_sft` |
119+
| `gsm8k` | `openai/gsm8k` | `train` |
120+
| `magpie` | `Magpie-Align/Magpie-Llama-3.1-Pro-300K-Filtered` | `train` |
121+
| `nemotron` | `nvidia/Llama-Nemotron-Post-Training-Dataset` | `chat` |
122+
| `sharegpt4v_coco` | `Lin-Chen/ShareGPT4V` | `train` |
123+
| `open-perfectblend` | `mlabonne/open-perfectblend` | `train` |
124+
125+
`sharegpt4v_coco` additionally needs an image-capable endpoint and local COCO files.
117126

118127
## Output Format
119128

0 commit comments

Comments
 (0)