You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
| Adding new training algorithms or model support | Fixing typos or improving documentation |
27
-
| Modifying the data pipeline or CLI/API | Small bug fixes (generally < 20 lines) |
28
-
| Large refactors or performance optimizations | Adding missing type annotations |
29
-
| Changes affecting 3+ files | Minor dependency updates |
24
+
|**Let's chat first if...**|**Feel free to dive right in for...**|| :---------------------------------------------- | :-------------------------------------- || Adding new training algorithms or model support | Fixing typos or improving documentation || Modifying the data pipeline or CLI/API | Small bug fixes (generally < 20 lines) || Large refactors or performance optimizations | Adding missing type annotations || Changes affecting 3+ files | Minor dependency updates |
30
25
31
26
*Maintainers typically respond in 2-3 business days. If you haven’t heard from us after a week, please feel free to give the thread a nudge!*
-**Standardized, Extensible Format:** Provides a Hugging Face-compatible format for defining speculative models, with tools to convert from external research repositories into a standard speculators format for easy adoption.
60
60
-**Seamless vLLM Integration:** Built for direct deployment into vLLM, enabling low-latency, production-grade inference with minimal overhead.
61
61
62
-
> [!TIP]
63
-
> Read more about Speculators features in this [vLLM blog post](https://blog.vllm.ai/2025/12/13/speculators-v030.html).
62
+
> [!TIP] Read more about Speculators features in this [vLLM blog post](https://blog.vllm.ai/2025/12/13/speculators-v030.html).
Copy file name to clipboardExpand all lines: docs/api/index.md
+1-2Lines changed: 1 addition & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,5 +2,4 @@
2
2
3
3
This section contains the auto-generated Python API documentation for Speculators.
4
4
5
-
!!! warning
6
-
Using the Python API directly is **not officially supported** at this time and there are **no guarantees of backward compatibility** between releases. We recommend using the [CLI commands](../cli/index.md) as the primary entrypoints for interacting with Speculators.
5
+
!!! warning Using the Python API directly is **not officially supported** at this time and there are **no guarantees of backward compatibility** between releases. We recommend using the [CLI commands](../cli/index.md) as the primary entrypoints for interacting with Speculators.
|`prepare_data.py`| Preprocess and tokenize datasets for training |[→ Details](prepare_data.md)|
12
-
|`data_generation_offline.py`| Generate hidden states offline using vLLM |[→ Details](data_generation_offline.md)|
13
-
|`launch_vllm.py`| Launch vLLM server configured for hidden states extraction |[→ Details](launch_vllm.md)|
14
-
|`train.py`| Train speculator models with online or offline hidden states |[→ Details](train.md)|
15
-
|`response_regeneration/`| Regenerate dataset responses using a vLLM-served model |[→ Details](response_regeneration.md)|
9
+
| Script | Purpose | Reference || ---------------------------- | ------------------------------------------------------------ | --------------------------------------- ||`prepare_data.py`| Preprocess and tokenize datasets for training |[→ Details](prepare_data.md)||`data_generation_offline.py`| Generate hidden states offline using vLLM |[→ Details](data_generation_offline.md)||`launch_vllm.py`| Launch vLLM server configured for hidden states extraction |[→ Details](launch_vllm.md)||`train.py`| Train speculator models with online or offline hidden states |[→ Details](train.md)||`response_regeneration/`| Regenerate dataset responses using a vLLM-served model |[→ Details](response_regeneration.md)|
| Logo | Black |[`speculators-logo-black.svg`](../assets/branding/speculators-logo-black.svg)|[`speculators-logo-black.png`](../assets/branding/speculators-logo-black.png)| Default on light backgrounds |
43
-
| Logo | Blue |[`speculators-logo-blue.svg`](../assets/branding/speculators-logo-blue.svg)|[`speculators-logo-blue.png`](../assets/branding/speculators-logo-blue.png)| Accent / section headers |
44
-
| Logo | White |[`speculators-logo-white.svg`](../assets/branding/speculators-logo-white.svg)|[`speculators-logo-white.png`](../assets/branding/speculators-logo-white.png)| Use on dark imagery/backgrounds |
| Icon | Blue |[`speculators-icon-blue.svg`](../assets/branding/speculators-icon-blue.svg)|[`speculators-icon-blue.png`](../assets/branding/speculators-icon-blue.png)| Accent |
47
-
| Icon | White |[`speculators-icon-white.svg`](../assets/branding/speculators-icon-white.svg)|[`speculators-icon-white.png`](../assets/branding/speculators-icon-white.png)| Dark backgrounds |
| Model Icon | Black |[`speculators-model-icon-black.svg`](../assets/branding/speculators-model-icon-black.svg)|[`speculators-model-icon-black.png`](../assets/branding/speculators-model-icon-black.png)| Architecture visuals |
50
-
| Model Icon | Blue |[`speculators-model-icon-blue.svg`](../assets/branding/speculators-model-icon-blue.svg)|[`speculators-model-icon-blue.png`](../assets/branding/speculators-model-icon-blue.png)| Architecture visuals |
51
-
| Model Icon | White |[`speculators-model-icon-white.svg`](../assets/branding/speculators-model-icon-white.svg)|[`speculators-model-icon-white.png`](../assets/branding/speculators-model-icon-white.png)| Dark backgrounds |
52
-
| Model Icon | White-Orange |[`speculators-model-icon-white-orange.svg`](../assets/branding/speculators-model-icon-white-orange.svg)|[`speculators-model-icon-white-orange.png`](../assets/branding/speculators-model-icon-white-orange.png)| Highlight sparingly |
| Use supplied SVG for scalable quality | Recreating marks from scratch |
87
-
| Maintain clear space and aspect ratio | Distorting, squeezing, or rotating |
88
-
| Choose color for sufficient contrast | Recoloring to arbitrary hues |
89
-
| Use a single primary mark per surface | Layering multiple logos |
90
-
| Attribute the project when citing | Using assets to imply endorsement |
70
+
| Do | Avoid || ------------------------------------- | ---------------------------------- || Use supplied SVG for scalable quality | Recreating marks from scratch || Maintain clear space and aspect ratio | Distorting, squeezing, or rotating || Choose color for sufficient contrast | Recoloring to arbitrary hues || Use a single primary mark per surface | Layering multiple logos || Attribute the project when citing | Using assets to imply endorsement |
Copy file name to clipboardExpand all lines: docs/index.md
+1-2Lines changed: 1 addition & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -31,8 +31,7 @@ Speculators standardizes this process by providing a productionized end-to-end f
31
31
-**Standardized, Extensible Format:** Provides a Hugging Face-compatible format for defining speculative models, with tools to convert from external research repositories into a standard speculators format for easy adoption.
32
32
-**Seamless vLLM Integration:** Built for direct deployment into vLLM, enabling low-latency, production-grade inference with minimal overhead.
33
33
34
-
!!! tip
35
-
Read more about Speculators features in this [vLLM blog post](https://blog.vllm.ai/2025/12/13/speculators-v030.html).
34
+
!!! tip Read more about Speculators features in this [vLLM blog post](https://blog.vllm.ai/2025/12/13/speculators-v030.html).
|**Verifier models**| Any supported | Any supported | Any supported | Models with native MTP only |
23
-
|**Training mode**| From scratch | From scratch | From scratch | Finetune existing MTP head |
24
-
|**Speculators**| Mature | Newer, growing fast | Newer, growing fast | Newer, growing fast |
25
-
|**vLLM**| Mature | Newer, growing fast | Newer, growing fast | Newer, growing fast |
19
+
|| Eagle-3 | P-EAGLE | DFlash | MTP || ------------------- | ------------- | ------------------- | ------------------- | --------------------------- ||**Draft layers**| Llama-style | Llama-style | Qwen3-style | Native MTP layers ||**Verifier models**| Any supported | Any supported | Any supported | Models with native MTP only ||**Training mode**| From scratch | From scratch | From scratch | Finetune existing MTP head ||**Speculators**| Mature | Newer, growing fast | Newer, growing fast | Newer, growing fast ||**vLLM**| Mature | Newer, growing fast | Newer, growing fast | Newer, growing fast |
26
20
27
21
Eagle-3 has been available longer and has broader support in both Speculators and vLLM. P-EAGLE, DFlash, and MTP were added more recently and support is improving rapidly.
Copy file name to clipboardExpand all lines: docs/user_guide/algorithms/dflash.md
+2-8Lines changed: 2 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -21,9 +21,7 @@ The target model produces hidden states (fused target context features) and deco
21
21
22
22
Pretrained DFlash speculator models are available on HuggingFace from the [RedHatAI speculator models collection](https://huggingface.co/collections/RedHatAI/speculator-models):
> **Note:** DFlash is under active development. Not all hardware configurations have been validated yet — refer to individual model cards for details.
29
27
@@ -33,11 +31,7 @@ Domino is a lightweight causal correction head that can be added on top of the D
33
31
34
32
The overhead is negligible: a single GRU layer and a two-layer MLP operate only on the suffix tokens within each block, adding a tiny fraction of the backbone transformer's FLOPs and requiring no additional sequential passes.
0 commit comments