Skip to content

Commit ab043d7

Browse files
orestis-zclaude
andcommitted
style: apply mdformat to docs
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Orestis Zambounis <orestis.zambounis@gmail.com>
1 parent e384825 commit ab043d7

14 files changed

Lines changed: 20 additions & 96 deletions

File tree

.claude/skills/pr-review/SKILL.md

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,6 @@
1-
---
2-
name: pr-review
3-
description: Review a GitHub PR with design-first analysis, posted as a GitHub review.
4-
---
1+
______________________________________________________________________
2+
3+
## name: pr-review description: Review a GitHub PR with design-first analysis, posted as a GitHub review.
54

65
# PR Review
76

CONTRIBUTING.md

Lines changed: 1 addition & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -21,12 +21,7 @@ To make the contribution process as smooth as possible, we ask that you coordina
2121

2222
### When should you check in?
2323

24-
| **Let's chat first if...** | **Feel free to dive right in for...** |
25-
| :---------------------------------------------- | :-------------------------------------- |
26-
| Adding new training algorithms or model support | Fixing typos or improving documentation |
27-
| Modifying the data pipeline or CLI/API | Small bug fixes (generally < 20 lines) |
28-
| Large refactors or performance optimizations | Adding missing type annotations |
29-
| Changes affecting 3+ files | Minor dependency updates |
24+
| **Let's chat first if...** | **Feel free to dive right in for...** | | :---------------------------------------------- | :-------------------------------------- | | Adding new training algorithms or model support | Fixing typos or improving documentation | | Modifying the data pipeline or CLI/API | Small bug fixes (generally < 20 lines) | | Large refactors or performance optimizations | Adding missing type annotations | | Changes affecting 3+ files | Minor dependency updates |
3025

3126
*Maintainers typically respond in 2-3 business days. If you haven’t heard from us after a week, please feel free to give the thread a nudge!*
3227

README.md

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -59,8 +59,7 @@ ______________________________________________________________________
5959
- **Standardized, Extensible Format:** Provides a Hugging Face-compatible format for defining speculative models, with tools to convert from external research repositories into a standard speculators format for easy adoption.
6060
- **Seamless vLLM Integration:** Built for direct deployment into vLLM, enabling low-latency, production-grade inference with minimal overhead.
6161

62-
> [!TIP]
63-
> Read more about Speculators features in this [vLLM blog post](https://blog.vllm.ai/2025/12/13/speculators-v030.html).
62+
> [!TIP] Read more about Speculators features in this [vLLM blog post](https://blog.vllm.ai/2025/12/13/speculators-v030.html).
6463
6564
## Supported Models
6665

docs/api/index.md

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,4 @@
22

33
This section contains the auto-generated Python API documentation for Speculators.
44

5-
!!! warning
6-
Using the Python API directly is **not officially supported** at this time and there are **no guarantees of backward compatibility** between releases. We recommend using the [CLI commands](../cli/index.md) as the primary entrypoints for interacting with Speculators.
5+
!!! warning Using the Python API directly is **not officially supported** at this time and there are **no guarantees of backward compatibility** between releases. We recommend using the [CLI commands](../cli/index.md) as the primary entrypoints for interacting with Speculators.

docs/cli/index.md

Lines changed: 1 addition & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -6,13 +6,7 @@ This page provides a comprehensive reference for all command-line interface (CLI
66

77
Speculators provides the following CLI scripts for different stages of the speculative decoding workflow:
88

9-
| Script | Purpose | Reference |
10-
| ---------------------------- | ------------------------------------------------------------ | --------------------------------------- |
11-
| `prepare_data.py` | Preprocess and tokenize datasets for training | [→ Details](prepare_data.md) |
12-
| `data_generation_offline.py` | Generate hidden states offline using vLLM | [→ Details](data_generation_offline.md) |
13-
| `launch_vllm.py` | Launch vLLM server configured for hidden states extraction | [→ Details](launch_vllm.md) |
14-
| `train.py` | Train speculator models with online or offline hidden states | [→ Details](train.md) |
15-
| `response_regeneration/` | Regenerate dataset responses using a vLLM-served model | [→ Details](response_regeneration.md) |
9+
| Script | Purpose | Reference | | ---------------------------- | ------------------------------------------------------------ | --------------------------------------- | | `prepare_data.py` | Preprocess and tokenize datasets for training | [→ Details](prepare_data.md) | | `data_generation_offline.py` | Generate hidden states offline using vLLM | [→ Details](data_generation_offline.md) | | `launch_vllm.py` | Launch vLLM server configured for hidden states extraction | [→ Details](launch_vllm.md) | | `train.py` | Train speculator models with online or offline hidden states | [→ Details](train.md) | | `response_regeneration/` | Regenerate dataset responses using a vLLM-served model | [→ Details](response_regeneration.md) |
1610

1711
## Common Workflows
1812

docs/cli/response_regeneration.md

Lines changed: 2 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,7 @@ Regenerates assistant responses in existing datasets using a vLLM-served model.
44

55
The pipeline consists of two scripts:
66

7-
| Script | Purpose |
8-
| ------------ | -------------------------------------------------------------- |
9-
| `run_all.sh` | End-to-end pipeline: starts vLLM, regenerates responses, stops |
10-
| `script.py` | Standalone response regeneration against a running vLLM server |
7+
| Script | Purpose | | ------------ | -------------------------------------------------------------- | | `run_all.sh` | End-to-end pipeline: starts vLLM, regenerates responses, stops | | `script.py` | Standalone response regeneration against a running vLLM server |
118

129
## run_all.sh
1310

@@ -110,10 +107,7 @@ python scripts/response_regeneration/script.py \
110107

111108
## Supported Datasets
112109

113-
| Dataset | HuggingFace ID | Prompt Field | Default Split |
114-
| --------- | ------------------------------------------------- | ------------- | ------------- |
115-
| Magpie | `Magpie-Align/Magpie-Llama-3.1-Pro-300K-Filtered` | `instruction` | `train` |
116-
| UltraChat | `HuggingFaceH4/ultrachat_200k` | `prompt` | `train_sft` |
110+
| Dataset | HuggingFace ID | Prompt Field | Default Split | | --------- | ------------------------------------------------- | ------------- | ------------- | | Magpie | `Magpie-Align/Magpie-Llama-3.1-Pro-300K-Filtered` | `instruction` | `train` | | UltraChat | `HuggingFaceH4/ultrachat_200k` | `prompt` | `train_sft` |
117111

118112
## Output Format
119113

docs/developer/branding.md

Lines changed: 2 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -37,21 +37,7 @@ Speculators is an open, unified library for creating and storing speculative dec
3737

3838
All files reside in `docs/assets/branding`.
3939

40-
| Category | Variant / Name | SVG | PNG | Notes |
41-
| ---------- | ----------------- | ------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------- | ------------------------------- |
42-
| Logo | Black | [`speculators-logo-black.svg`](../assets/branding/speculators-logo-black.svg) | [`speculators-logo-black.png`](../assets/branding/speculators-logo-black.png) | Default on light backgrounds |
43-
| Logo | Blue | [`speculators-logo-blue.svg`](../assets/branding/speculators-logo-blue.svg) | [`speculators-logo-blue.png`](../assets/branding/speculators-logo-blue.png) | Accent / section headers |
44-
| Logo | White | [`speculators-logo-white.svg`](../assets/branding/speculators-logo-white.svg) | [`speculators-logo-white.png`](../assets/branding/speculators-logo-white.png) | Use on dark imagery/backgrounds |
45-
| Icon | Black | [`speculators-icon-black.svg`](../assets/branding/speculators-icon-black.svg) | [`speculators-icon-black.png`](../assets/branding/speculators-icon-black.png) | App tile / light UI |
46-
| Icon | Blue | [`speculators-icon-blue.svg`](../assets/branding/speculators-icon-blue.svg) | [`speculators-icon-blue.png`](../assets/branding/speculators-icon-blue.png) | Accent |
47-
| Icon | White | [`speculators-icon-white.svg`](../assets/branding/speculators-icon-white.svg) | [`speculators-icon-white.png`](../assets/branding/speculators-icon-white.png) | Dark backgrounds |
48-
| Icon | White-Orange | [`speculators-icon-white-orange.svg`](../assets/branding/speculators-icon-white-orange.svg) | [`speculators-icon-white-orange.png`](../assets/branding/speculators-icon-white-orange.png) | Limited highlight |
49-
| Model Icon | Black | [`speculators-model-icon-black.svg`](../assets/branding/speculators-model-icon-black.svg) | [`speculators-model-icon-black.png`](../assets/branding/speculators-model-icon-black.png) | Architecture visuals |
50-
| Model Icon | Blue | [`speculators-model-icon-blue.svg`](../assets/branding/speculators-model-icon-blue.svg) | [`speculators-model-icon-blue.png`](../assets/branding/speculators-model-icon-blue.png) | Architecture visuals |
51-
| Model Icon | White | [`speculators-model-icon-white.svg`](../assets/branding/speculators-model-icon-white.svg) | [`speculators-model-icon-white.png`](../assets/branding/speculators-model-icon-white.png) | Dark backgrounds |
52-
| Model Icon | White-Orange | [`speculators-model-icon-white-orange.svg`](../assets/branding/speculators-model-icon-white-orange.svg) | [`speculators-model-icon-white-orange.png`](../assets/branding/speculators-model-icon-white-orange.png) | Highlight sparingly |
53-
| Diagram | User Flow (Light) | [`speculators-user-flow-light.svg`](../assets/branding/speculators-user-flow-light.svg) | [`speculators-user-flow-light.png`](../assets/branding/speculators-user-flow-light.png) | Light docs / print |
54-
| Diagram | User Flow (Dark) | [`speculators-user-flow-dark.svg`](../assets/branding/speculators-user-flow-dark.svg) | [`speculators-user-flow-dark.png`](../assets/branding/speculators-user-flow-dark.png) | Dark slides / sites |
40+
| Category | Variant / Name | SVG | PNG | Notes | | ---------- | ----------------- | ------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------- | ------------------------------- | | Logo | Black | [`speculators-logo-black.svg`](../assets/branding/speculators-logo-black.svg) | [`speculators-logo-black.png`](../assets/branding/speculators-logo-black.png) | Default on light backgrounds | | Logo | Blue | [`speculators-logo-blue.svg`](../assets/branding/speculators-logo-blue.svg) | [`speculators-logo-blue.png`](../assets/branding/speculators-logo-blue.png) | Accent / section headers | | Logo | White | [`speculators-logo-white.svg`](../assets/branding/speculators-logo-white.svg) | [`speculators-logo-white.png`](../assets/branding/speculators-logo-white.png) | Use on dark imagery/backgrounds | | Icon | Black | [`speculators-icon-black.svg`](../assets/branding/speculators-icon-black.svg) | [`speculators-icon-black.png`](../assets/branding/speculators-icon-black.png) | App tile / light UI | | Icon | Blue | [`speculators-icon-blue.svg`](../assets/branding/speculators-icon-blue.svg) | [`speculators-icon-blue.png`](../assets/branding/speculators-icon-blue.png) | Accent | | Icon | White | [`speculators-icon-white.svg`](../assets/branding/speculators-icon-white.svg) | [`speculators-icon-white.png`](../assets/branding/speculators-icon-white.png) | Dark backgrounds | | Icon | White-Orange | [`speculators-icon-white-orange.svg`](../assets/branding/speculators-icon-white-orange.svg) | [`speculators-icon-white-orange.png`](../assets/branding/speculators-icon-white-orange.png) | Limited highlight | | Model Icon | Black | [`speculators-model-icon-black.svg`](../assets/branding/speculators-model-icon-black.svg) | [`speculators-model-icon-black.png`](../assets/branding/speculators-model-icon-black.png) | Architecture visuals | | Model Icon | Blue | [`speculators-model-icon-blue.svg`](../assets/branding/speculators-model-icon-blue.svg) | [`speculators-model-icon-blue.png`](../assets/branding/speculators-model-icon-blue.png) | Architecture visuals | | Model Icon | White | [`speculators-model-icon-white.svg`](../assets/branding/speculators-model-icon-white.svg) | [`speculators-model-icon-white.png`](../assets/branding/speculators-model-icon-white.png) | Dark backgrounds | | Model Icon | White-Orange | [`speculators-model-icon-white-orange.svg`](../assets/branding/speculators-model-icon-white-orange.svg) | [`speculators-model-icon-white-orange.png`](../assets/branding/speculators-model-icon-white-orange.png) | Highlight sparingly | | Diagram | User Flow (Light) | [`speculators-user-flow-light.svg`](../assets/branding/speculators-user-flow-light.svg) | [`speculators-user-flow-light.png`](../assets/branding/speculators-user-flow-light.png) | Light docs / print | | Diagram | User Flow (Dark) | [`speculators-user-flow-dark.svg`](../assets/branding/speculators-user-flow-dark.svg) | [`speculators-user-flow-dark.png`](../assets/branding/speculators-user-flow-dark.png) | Dark slides / sites |
5541

5642
## Usage Guidelines
5743

@@ -81,13 +67,7 @@ All files reside in `docs/assets/branding`.
8167

8268
## Do and Avoid
8369

84-
| Do | Avoid |
85-
| ------------------------------------- | ---------------------------------- |
86-
| Use supplied SVG for scalable quality | Recreating marks from scratch |
87-
| Maintain clear space and aspect ratio | Distorting, squeezing, or rotating |
88-
| Choose color for sufficient contrast | Recoloring to arbitrary hues |
89-
| Use a single primary mark per surface | Layering multiple logos |
90-
| Attribute the project when citing | Using assets to imply endorsement |
70+
| Do | Avoid | | ------------------------------------- | ---------------------------------- | | Use supplied SVG for scalable quality | Recreating marks from scratch | | Maintain clear space and aspect ratio | Distorting, squeezing, or rotating | | Choose color for sufficient contrast | Recoloring to arbitrary hues | | Use a single primary mark per surface | Layering multiple logos | | Attribute the project when citing | Using assets to imply endorsement |
9171

9272
## License & Attribution
9373

docs/index.md

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -31,8 +31,7 @@ Speculators standardizes this process by providing a productionized end-to-end f
3131
- **Standardized, Extensible Format:** Provides a Hugging Face-compatible format for defining speculative models, with tools to convert from external research repositories into a standard speculators format for easy adoption.
3232
- **Seamless vLLM Integration:** Built for direct deployment into vLLM, enabling low-latency, production-grade inference with minimal overhead.
3333

34-
!!! tip
35-
Read more about Speculators features in this [vLLM blog post](https://blog.vllm.ai/2025/12/13/speculators-v030.html).
34+
!!! tip Read more about Speculators features in this [vLLM blog post](https://blog.vllm.ai/2025/12/13/speculators-v030.html).
3635

3736
## Quick Start
3837

docs/user_guide/algorithms/decision_guide.md

Lines changed: 1 addition & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -16,13 +16,7 @@ Eagle-3, P-EAGLE, and DFlash can be paired with any supported verifier model (in
1616

1717
## Current Support
1818

19-
| | Eagle-3 | P-EAGLE | DFlash | MTP |
20-
| ------------------- | ------------- | ------------------- | ------------------- | --------------------------- |
21-
| **Draft layers** | Llama-style | Llama-style | Qwen3-style | Native MTP layers |
22-
| **Verifier models** | Any supported | Any supported | Any supported | Models with native MTP only |
23-
| **Training mode** | From scratch | From scratch | From scratch | Finetune existing MTP head |
24-
| **Speculators** | Mature | Newer, growing fast | Newer, growing fast | Newer, growing fast |
25-
| **vLLM** | Mature | Newer, growing fast | Newer, growing fast | Newer, growing fast |
19+
| | Eagle-3 | P-EAGLE | DFlash | MTP | | ------------------- | ------------- | ------------------- | ------------------- | --------------------------- | | **Draft layers** | Llama-style | Llama-style | Qwen3-style | Native MTP layers | | **Verifier models** | Any supported | Any supported | Any supported | Models with native MTP only | | **Training mode** | From scratch | From scratch | From scratch | Finetune existing MTP head | | **Speculators** | Mature | Newer, growing fast | Newer, growing fast | Newer, growing fast | | **vLLM** | Mature | Newer, growing fast | Newer, growing fast | Newer, growing fast |
2620

2721
Eagle-3 has been available longer and has broader support in both Speculators and vLLM. P-EAGLE, DFlash, and MTP were added more recently and support is improving rapidly.
2822

docs/user_guide/algorithms/dflash.md

Lines changed: 2 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -21,9 +21,7 @@ The target model produces hidden states (fused target context features) and deco
2121

2222
Pretrained DFlash speculator models are available on HuggingFace from the [RedHatAI speculator models collection](https://huggingface.co/collections/RedHatAI/speculator-models):
2323

24-
| Verifier | Speculator |
25-
| ----------------------- | --------------------------------------------------------------------------------------------------------------- |
26-
| `google/gemma-4-31B-it` | [`RedHatAI/gemma-4-31B-it-speculator.dflash`](https://huggingface.co/RedHatAI/gemma-4-31B-it-speculator.dflash) |
24+
| Verifier | Speculator | | ----------------------- | --------------------------------------------------------------------------------------------------------------- | | `google/gemma-4-31B-it` | [`RedHatAI/gemma-4-31B-it-speculator.dflash`](https://huggingface.co/RedHatAI/gemma-4-31B-it-speculator.dflash) |
2725

2826
> **Note:** DFlash is under active development. Not all hardware configurations have been validated yet — refer to individual model cards for details.
2927
@@ -33,11 +31,7 @@ Domino is a lightweight causal correction head that can be added on top of the D
3331

3432
The overhead is negligible: a single GRU layer and a two-layer MLP operate only on the suffix tokens within each block, adding a tiny fraction of the backbone transformer's FLOPs and requiring no additional sequential passes.
3533

36-
| | DFlash | DFlash + Domino |
37-
| ------------------------- | -------------------- | --------------------------------------------- |
38-
| **Draft path** | Base logits → argmax | Base logits → GRU correction → refined logits |
39-
| **Per-position accuracy** | Good | Better (GRU conditions on prior tokens) |
40-
| **Training cost** | ~| Negligible overhead |
34+
| | DFlash | DFlash + Domino | | ------------------------- | -------------------- | --------------------------------------------- | | **Draft path** | Base logits → argmax | Base logits → GRU correction → refined logits | | **Per-position accuracy** | Good | Better (GRU conditions on prior tokens) | | **Training cost** | ~| Negligible overhead |
4135

4236
Select Domino via `--projector-type domino` when training (see the [Train DFlash tutorial](../tutorials/train_dflash_online.md#domino-variant)).
4337

0 commit comments

Comments
 (0)