Skip to content

Draft-only (MTP) repos indexed as loadable models, then crash at generation: 'Qwen3_5MTPDraftModel' object has no attribute 'get_input_embeddings' #371

Description

@kafasam

Environment

  • LM Studio's bundled MLX engine (observed on app-mlx-generate @34, macOS)
  • Model: mlx-community/Qwen3.8-27B-MTP-4bit (draft-only / MTP repo)

Observed behavior

The draft-only MTP repo is indexed as if it were a normal runnable model:

  1. It appears in the model catalog.
  2. It loads successfully.
  3. Every generation request then crashes deterministically with:
AttributeError: 'Qwen3_5MTPDraftModel' object has no attribute 'get_input_embeddings'

Expected behavior

Either support MTP drafting, or refuse to index/load draft-only repos with a
clear error at load time instead of a generation-time traceback.

Impact

Downstream users select these repos believing they are runnable base models;
the failure surfaces only mid-session, after load "succeeded."

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions