LocalAI version:
Latest brew install of 3.8.0
Environment, CPU architecture, OS, and Version:
Mac M4:
Darwin macmini.local 24.5.0 Darwin Kernel Version 24.5.0: Tue Apr 22 19:54:43 PDT 2025; root:xnu-11417.121.6~2/RELEASE_ARM64_T8132 arm64
Describe the bug
After importing an MLX model downloaded from HuggingFace manually or with LM Studio, trying to "Chat" with the LLM fails due to repo_type. There is no documentation on repo_type anywhere (that I can find).
To Reproduce
- Download
mlx-community/Qwen3-Coder-30B-A3B-Instruct-4bit from HuggingFace using curl or LM Studio
- Import the model from the location, in my case,
file:///Users/user/.lmstudio/models/mlx-community/Qwen3-Coder-30B-A3B-Instruct-4bit
- Import is successful
- Go to chat
- Type "Test" and send
- Chat responds with:
Internal error: failed to load model with internal loader: could not load model (no success): Error loading MLX model: Repo id must be in the form 'repo_name' or 'namespace/repo_name': 'file:///Users/user/.lmstudio/models/mlx-community/Qwen3-Coder-30B-A3B-Instruct-4bit'. Use repo_type argument if needed.
Expected behavior
A response.
Logs
11:10AM INF HTTP request method=GET path=/static/logo_horizontal.png status=200
11:10AM INF BackendLoader starting backend=mlx modelID=Qwen3-Coder-30B-A3B-Instruct-4bit o.model=file:///Users/user/.lmstudio/models/mlx-community/Qwen3-Coder-30B-A3B-Instruct-4bit
11:10AM ERR Failed to load model Qwen3-Coder-30B-A3B-Instruct-4bit with backend mlx error="failed to load model with internal loader: could not load model (no success): Error loading MLX model: Repo id must be in the form 'repo_name' or 'namespace/repo_name': 'file:///Users/user/.lmstudio/models/mlx-community/Qwen3-Coder-30B-A3B-Instruct-4bit'. Use `repo_type` argument if needed." modelID=Qwen3-Coder-30B-A3B-Instruct-4bit
11:10AM ERR Stream ended with error: failed to load model with internal loader: could not load model (no success): Error loading MLX model: Repo id must be in the form 'repo_name' or 'namespace/repo_name': 'file:///Users/user/.lmstudio/models/mlx-community/Qwen3-Coder-30B-A3B-Instruct-4bit'. Use `repo_type` argument if needed.
11:10AM INF HTTP request method=POST path=/v1/chat/completions status=200
LocalAI version:
Latest
brewinstall of 3.8.0Environment, CPU architecture, OS, and Version:
Mac M4:
Describe the bug
After importing an MLX model downloaded from HuggingFace manually or with LM Studio, trying to "Chat" with the LLM fails due to
repo_type. There is no documentation onrepo_typeanywhere (that I can find).To Reproduce
mlx-community/Qwen3-Coder-30B-A3B-Instruct-4bitfrom HuggingFace using curl or LM Studiofile:///Users/user/.lmstudio/models/mlx-community/Qwen3-Coder-30B-A3B-Instruct-4bitExpected behavior
A response.
Logs