Skip to content

Speculative Decoding path #21

@andthattoo

Description

@andthattoo

Either by adding multiple LM heads (medusa) or using a drafter model.
Alternative: https://github.com/SafeAILab/EAGLE
Alternative: https://arxiv.org/html/2502.09419v1

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions