Skip to content

Add support for additional draft checkpoint#2317

Open
bitgamma wants to merge 1 commit into
mainfrom
bitgamma/draft-model
Open

Add support for additional draft checkpoint#2317
bitgamma wants to merge 1 commit into
mainfrom
bitgamma/draft-model

Conversation

@bitgamma

@bitgamma bitgamma commented Jun 19, 2026

Copy link
Copy Markdown
Member
  • Added support for draft checkpoint in llamacpp recipe. This allows downloading a drafter model (MTP, EAGLE3, etc) alongside the main model. Works well with the existing mtp label and with manually specifying MTP/drafting options
  • Added Gemma4 MTP models that use this mechanism.

@github-actions github-actions Bot added the enhancement New feature or request label Jun 19, 2026
@bitgamma bitgamma self-assigned this Jun 19, 2026

@fl0rianr fl0rianr left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice addition, great! One concern: --model-draft is reserved unconditionally, even when draft_path is empty. Since validate_custom_args() rejects reserved flags, this prevents users from manually passing --model-draft / -md / --spec-draft-model through llamacpp_args for models that do not have a registered draft checkpoint. Could we move the reservation into the if (!draft_path.empty()) block, e.g. pass the aliases directly to push_arg() when Lemonade actually adds the managed --model-draft argument?

@bitgamma

Copy link
Copy Markdown
Member Author

@fl0rianr, this is intentional and the same is done with mmproj file and all other reserved argumens really. Allowing specifying arbitrary paths is a potential security risk

@bitgamma bitgamma requested a review from fl0rianr June 20, 2026 04:25

@fl0rianr fl0rianr left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your right - I remember that topic brought up by Mario. In that case you have my approval.

@fl0rianr fl0rianr added this pull request to the merge queue Jun 20, 2026
@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jun 20, 2026
@fl0rianr

Copy link
Copy Markdown
Collaborator

not possible to merge until the whisper issue is resolved. https://discord.com/channels/1392562559122407535/1394414791593820252/1517829183206854736

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants