Add support for additional draft checkpoint#2317
Conversation
fl0rianr
left a comment
There was a problem hiding this comment.
Nice addition, great! One concern: --model-draft is reserved unconditionally, even when draft_path is empty. Since validate_custom_args() rejects reserved flags, this prevents users from manually passing --model-draft / -md / --spec-draft-model through llamacpp_args for models that do not have a registered draft checkpoint. Could we move the reservation into the if (!draft_path.empty()) block, e.g. pass the aliases directly to push_arg() when Lemonade actually adds the managed --model-draft argument?
|
@fl0rianr, this is intentional and the same is done with mmproj file and all other reserved argumens really. Allowing specifying arbitrary paths is a potential security risk |
fl0rianr
left a comment
There was a problem hiding this comment.
Your right - I remember that topic brought up by Mario. In that case you have my approval.
|
not possible to merge until the whisper issue is resolved. https://discord.com/channels/1392562559122407535/1394414791593820252/1517829183206854736 |
draftcheckpoint in llamacpp recipe. This allows downloading a drafter model (MTP, EAGLE3, etc) alongside the main model. Works well with the existingmtplabel and with manually specifying MTP/drafting options