Use non registry models

[JasonW25]

To save anyone else the huge annoyance of trying to figure out how to run different models when using the docker container:

Fork the registry-tabby repo.

When you launch docker tabby container and want to use your chat model or something:

--chat-model github_user_name/model_name

Obviously replace github_user_name with your username.

You will have to add your model to the models.json in the repo.

Hopefully I saved you hours of bashing your head against the wall.

[mg1986jp]

For models not in the Tabby registry, you need to set up the model manually in the config. In ~/.tabby/config.toml (or wherever your config lives), you can point to a local GGUF file or a custom model path:

[model.completion.local]
model_id = "/path/to/your/model.gguf"

Or if you're running through Docker, mount the model file and reference it in the container:

docker run -v /path/to/models:/models tabbyml/tabby serve --model /models/your-model.gguf

The exact config format depends on your Tabby version — newer versions might use a different section name. Check tabby serve --help for the available model-related flags.

If you're trying to use a HuggingFace model that's not in the registry, you can usually specify the HF repo ID directly and Tabby will download it, though not all architectures are supported.