Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/cli/response_regeneration.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,7 @@ The text presets from the shared dataset registry (`DATASET_CONFIGS` in `specula
| `magpie` | `Magpie-Align/Magpie-Llama-3.1-Pro-300K-Filtered` | `train` |
| `nemotron` | `nvidia/Llama-Nemotron-Post-Training-Dataset` | `chat` |
| `open-perfectblend` | `mlabonne/open-perfectblend` | `train` |
| `when2call` | `nvidia/When2Call` | `train` |

The registry's multimodal preset, `sharegpt4v_coco`, is **off-policy only** and `--dataset` rejects it. Its turns carry image content parts, which the Chat Completions API rejects, and the pre-tokenized output row has nowhere to keep pixel data. Use it with `prepare-data`.

Expand Down
13 changes: 13 additions & 0 deletions src/speculators/data_generation/configs.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,12 @@ def _normalize_nemotron(example: dict) -> dict:
}


def _normalize_when2call(example: dict) -> dict:
# When2Call already stores the dialogue as role/content ``messages``;
# the parallel ``tools`` column is preserved by the map for tool-aware runs.
return {"conversations": example["messages"]}


def get_coco_dir():
return os.getenv("COCO_DIR") or "coco/"

Expand Down Expand Up @@ -151,4 +157,11 @@ def _normalize_sharegpt4v_coco(example: dict) -> dict:
hf_path="mlabonne/open-perfectblend",
split="train",
),
"when2call": DatasetConfig(
name="when2call",
hf_path="nvidia/When2Call",
subset="train_sft",
split="train",
normalize_fn=_normalize_when2call,
),
}