Open WebUI + LiteLLM Stack – Production-Ready* with Observability (LGTM), Qdrant, Traefik

[mhajder]

Hey everyone!

I wanted to share a project I have been working on to make deploying Open Web UI + LiteLLM easier, more secure, and fully observable for production or serious homelab environments. I am using it as demo to testing AI pods.

I created openwebui-stack, a robust Docker Compose setup that orchestrates Open Web UI alongside a full suite of enterprise-grade dependencies.

What is it?

Unlike simple "all-in-one" deployments, this stack breaks services down for scalability and performance. It includes a pre-configured LGTM stack (Loki, Grafana, Tempo, Mimir) for full observability, Traefik for secure reverse proxying, and shared databases for persistent storage.

Technical Breakdown

The stack automatically configures and connects the following services:

Core & AI

• Open WebUI: The main chat interface, fully configured for RAG and OTel.
• LiteLLM: A unified gateway proxy to manage multiple LLM providers (OpenAI, Anthropic, Ollama, etc.) with centralized key management.
• Qdrant: A production-ready Vector Database for high-performance RAG operations.

Infrastructure & Data

• Traefik: Acts as the reverse proxy, handling automatic HTTPS, routing, and basic auth for dashboards.
• PostgreSQL: A shared relational database replacing default SQLite files for better concurrency (serves both Open WebUI and LiteLLM).
• Valkey: A high-performance open-source Redis alternative used for caching and session management.

Observability (LGTM Stack)

• Grafana: Pre-loaded with dashboards for Open WebUI, LiteLLM, System Metrics, and GPU monitoring.
• OTel Collector: Collects distributed traces and metrics from all services.
• Prometheus/Node Exporter: For system-level resource monitoring (CPU, RAM, Disk).
• NVIDIA DCGM Exporter: (Optional profile) For monitoring GPU usage and temperature.

Key Features

• Security First: Services run on isolated networks (frontend vs internal backend). The Traefik dashboard is protected by authentication, and strict security headers are applied.
• Full Observability: Includes pre-configured Grafana dashboards to track token usage, latency, error rates, and hardware performance out of the box.
• Production Ready: Uses PostgreSQL instead of SQLite and Valkey for caching, making it suitable for multi-user environments.
• One-Command Setup: Includes a setup.sh script to generate secure secrets/passwords automatically.

How to use it

Clone the repo:

git clone https://github.com/mhajder/openwebui-stack.git
cd openwebui-stack

Run the setup script (generates .env secrets):

chmod +x scripts/*.sh
./scripts/setup.sh

Spin up the stack:

docker compose --profile monitoring --profile gpu up -d

You can check out the repository here: https://github.com/mhajder/openwebui-stack

I would love to get your feedback on the architecture, especially regarding the OTel integration! PRs are welcome as always!

[mubashir1osmani]

great project @mhajder , ive been building something similar using nix. thanks for sharing!

[mhajder]

Thanks! I was also thinking about using Nix but mostly I am using this stack as demo for clients as testing space on new installed AI Pods. On production I am using similar looking k8s stack.
You can fork my repo, and just make a PR with Nix configuration files etc. PRs are always welcome! If you don't have time to prepare Nix configuration just use https://github.com/aksiksi/compose2nix

[darfaz]

Nice stack — the isolated network architecture (frontend vs internal backend) is the right approach.

One area I'd suggest adding to the security layer: LLM-level input/output guardrails. The stack handles infrastructure security well (TLS, auth, network isolation), but there's a gap at the application layer — what's actually being sent to and received from the LLMs.

In production multi-user environments, you want to catch:

• PII leaking into LLM requests (especially with RAG pulling from shared knowledge bases)
• Prompt injection attempts via user inputs
• Sensitive data in model responses that shouldn't be exposed to certain users

Since you're already running LiteLLM as a proxy, you could add a guardrail layer there. LiteLLM has built-in guardrail hooks, and for more structured scanning, ClawMoat can plug in as middleware to validate inputs/outputs against configurable policies.

The observability piece (Grafana dashboards for token usage/latency) is great — would be useful to also log guardrail violations so you can track attempted misuse alongside performance metrics.

Thanks for sharing this — bookmarking it as a reference architecture.