Would the tabby API work with services such as LibreChat/Open WebUI

[SoCuul]

If I'm understanding it correctly it seems like tabby is hosting the model and providing an API endpoint which can be accessed to interact with the model. Can this be integrated into a web ui such as LibreChat/Open Web UI or would I have to host the model more than once.

[OleMussmann]

Have you tested that, @Cfomodz ? It does not work for me, at least for Open WebUI. It fails with a "network error".

This is the API that tabby provides:

# v1
POST /v1/chat_completions
POST /v1/completions
POST /v1/events
GET /v1/health

# v1beta
POST /v1beta/ingestion
DELETE  /v1beta/ingestion/{source}
DELETE /v1beta/ingestion/{source}/{id}
GET /v1beta/server_setting

Open WebUI says in Admin Settings → Connections → General → OpenAPI → Edit Connection:

Leave empty to include all models from "http://[URL]/v1/models" endpoint

Open WebUI is probing tabby at an endpoint that does not exist. Running the same model that tabby uses directly with llama.cpp's llama-server does connect and does work.

---

Tested versions:

• open-webui 0.7.2
• tabby 0.31.2
• llama.cpp 6981