[Dataset] [Tool call] Add Hermes function-calling dataset (#771)

## Purpose

Add the **Hermes function-calling** dataset
(`NousResearch/hermes-function-calling-v1`, `func_calling` config) as a
predefined preset — a function-calling SFT set (Apache-2.0) originally
for post-training.

why:

- Tool-use training fixture for the tool-call speculator work, alongside
When2Call (#769).
- Hermes `func_calling` is the canonical function-calling SFT dataset
(used to train Hermes 2 Pro) — genuine tool-call → tool-result → answer
trajectories.
- License **Apache-2.0**. Verified **no overlap** with the
`nvidia/SPEED-Bench` eval set (SPEED-Bench contains no function-calling
data; Hermes is not among its 24 sources).

Rebased onto `main` after #777, the followup that collapsed the two
dataset registries into one. This is now a **single** entry in
`src/speculators/data_generation/configs.py` — since #777 the
regeneration script consumes that same registry, so the preset reaches
both pipelines (`prepare-data` and `--dataset`) by construction, with no
change to `scripts/response_regeneration/script.py`.
`docs/cli/response_regeneration.md` gains the matching row.

### Notes for reviewers

- The dataset is already in `conversations` / ShareGPT `from,value` form
(same shape as the existing `sharegpt` preset), so it needs **no
`normalize_fn`** — the offline `_normalize_conversation` maps
`from/value` → canonical roles natively (`system → user → assistant →
tool → assistant`, verified).
- **Shape**, measured over 1299 rows parsed from `func-calling.json`:
each row is a *single* user request followed by a multi-step agentic
trajectory. Raw `from` sequences: `system → human → gpt → tool → gpt`
(927), `system → human → gpt` (199), `system → human → gpt → tool`
(163), `system → human → tool → gpt` (10). 1100/1299 rows carry a
tool-result turn; **no** row has more than one `human` turn.
"Multi-turn" here means the agentic exchange, not multiple user turns.
- Tool calls/results are **text-encoded** (`<tool_call>` /
`<tool_response>`) inside the turn value, not structured `tool_calls`
fields. Offline training on this is self-consistent (the system turn
embeds `<tools>`; the assistant emits `<tool_call>` text). The
structured tool-aware path (#750) will additionally need `<tool_call>`
parsing, and must avoid double-injecting the tool schema (the system
turn already contains `<tools>`) — out of scope here.
- The `tools` column is a single JSON string, which `_parse_conv_tools`
consumes via `json.loads`.
- **On-policy note.** #777 makes every registry preset a valid
`--dataset`, so this entry also enables `--dataset hermes-fc`. That is
coherent here: `prepare_row()` yields `[system, user]` and the system
turn still carries the `<tools>` schemas, so the model *does* see the
tools and can emit a `<tool_call>`. The tool-result rounds are dropped,
so only the first assistant turn is regenerated; full tool-call regen
semantics are #750. This is the difference from #769, which is held as
draft: When2Call keeps its tool schemas *outside* the conversation (a
parallel `tools` column), so on-policy regen there would see no tools at
all.

## Tests

- `make quality` (ruff check, ruff format, mdformat, mypy): clean.
- `pytest tests/unit/scripts tests/unit/data_generation`: **49 passed**.
- Schema verified against the real data file — 1299 rows parsed from
`func-calling.json`: `id` / `conversations` (from/value) / `tools` (JSON
string) / `category` / `subcategory` / `task`, with `<tool_call>` and
`<tool_response>` markup present inline. Config/split resolve confirmed
against the Hub API (`func_calling` config → `train` split, Apache-2.0).
- Registry sanity: `hermes-fc` entry is bare (`normalize_fn is None`);
the offline `_normalize_conversation` maps a real Hermes tool
conversation to `system/user/assistant/tool/assistant` with the
`<tool_call>` text preserved; `prepare_row()` on a real row yields
`[system, user]` with the `<tools>` schemas intact; `hermes-fc` appears
in the `--dataset` CLI choices with `script.py` byte-identical to
`main`.

## Checklist

I have filled in:

- [x] The purpose of the PR, such as "Fix some issue (link existing
issues this PR will resolve)".
- [x] The test plan/results, such as providing test command and pasting
the results.
- [x] (Optional) The necessary documentation update.
- [x] I (a human) have written or reviewed the code in this pr to the
best of my ability.

Signed-off-by: Ranran Haoran Zhang <ranzhang@redhat.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: shanjiaz <zsjwpianpian@gmail.com>

Author: 3 people

docs/cli/response_regeneration.md

@@ -124,6 +124,7 @@ The text presets from the shared dataset registry (`DATASET_CONFIGS` in `specula
 | `magpie`            | `Magpie-Align/Magpie-Llama-3.1-Pro-300K-Filtered` | `train`       |
 | `nemotron`          | `nvidia/Llama-Nemotron-Post-Training-Dataset`     | `chat`        |
 | `open-perfectblend` | `mlabonne/open-perfectblend`                      | `train`       |
+| `hermes-fc`         | `NousResearch/hermes-function-calling-v1`         | `train`       |
 
 The registry's multimodal preset, `sharegpt4v_coco`, is **off-policy only** and `--dataset` rejects it. Its turns carry image content parts, which the Chat Completions API rejects, and the pre-tokenized output row has nowhere to keep pixel data. Use it with `prepare-data`.
 

src/speculators/data_generation/configs.py

@@ -151,4 +151,11 @@ def _normalize_sharegpt4v_coco(example: dict) -> dict:
         hf_path="mlabonne/open-perfectblend",
         split="train",
     ),
+    # Multi-turn function-calling SFT
+    "hermes-fc": DatasetConfig(
+        name="hermes-fc",
+        hf_path="NousResearch/hermes-function-calling-v1",
+        subset="func_calling",
+        split="train",
+    ),
 }