Skip to content

Bug: listing models needs to filter out GGUF extension #1077

@jamesbraza

Description

@jamesbraza

Starting with an empty models/ and a fresh restart of the LocalAI server:

> ls models
> curl http://localhost:8080/models/apply -H "Content-Type: application/json" -d '{
     "url": "github:go-skynet/model-gallery/base.yaml",
     "name": "llama2-chat",
     "files": [
        {
            "uri": "https://huggingface.co/TheBloke/Llama-2-13B-chat-GGUF/resolve/main/llama-2-13b-chat.Q4_K_S.gguf",
            "sha256": "106d3b9c0a8e24217f588f2af44fce95ec8906c1ea92ca9391147ba29cc4d2a4",
            "filename": "llama-2-13b-chat.Q4_K_S.gguf"
        }
     ]
   }'
# ...
> curl http://localhost:8080/models
{"object":"list","data":[{"id":"llama2-chat","object":"model"},{"id":"llama-2-13b-chat.Q4_K_S.gguf","object":"model"}]}
> ls models
chat.tmpl                    completion.tmpl              llama-2-13b-chat.Q4_K_S.gguf llama2-chat.yaml

It seems the GGUF extension is not being filtered out by the listing models endpoint. It should be filtered out, I think only the files ending in .yaml should show up.

Metadata

Metadata

Assignees

Labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions