Skip to content

Commit 4ba4a37

Browse files
yasinBursaliclaude
yasinBursali
and
claude
authored
docs: add READMEs for all undocumented extension services (#64)
14 of 17 extension services had no README. This adds documentation for every undocumented service following the existing privacy-shield README template: overview, features, configuration tables, API endpoints, architecture diagrams, troubleshooting, and file listings. All content sourced from manifest.yaml, compose files, .env.example, and source code — cross-referenced and audited for accuracy. Services documented: llama-server, open-webui, dashboard-api, whisper, tts, n8n, litellm, qdrant, embeddings, searxng, perplexica, comfyui, openclaw, opencode. Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
1 parent daddd64 commit 4ba4a37

14 files changed

Lines changed: 1667 additions & 0 deletions

File tree

‎dream-server/extensions/services/comfyui/README.md‎

Lines changed: 140 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,140 @@
1+
# ComfyUI
2+
3+
Node-based image generation UI and backend for Dream Server
4+
5+
## Overview
6+
7+
ComfyUI provides a powerful, node-based interface for running Stable Diffusion and FLUX.1 image generation models locally. It exposes both a visual workflow editor in the browser and a REST API, enabling programmatic image generation from other services. ComfyUI requires a GPU (NVIDIA or AMD) and is not available on CPU-only systems.
8+
9+
## Features
10+
11+
- **Node-based workflow editor**: Build and share custom generation pipelines visually
12+
- **FLUX.1 support**: Configured for FLUX.1 image generation out of the box
13+
- **Multiple model types**: Supports checkpoints, LoRAs, VAEs, text encoders, and diffusion models
14+
- **Persistent model storage**: Models stored in `./data/comfyui/models` and survive container rebuilds
15+
- **Workflow templates**: Pre-loaded workflow JSON files from `./data/comfyui/workflows`
16+
- **REST API**: Programmatic image generation via HTTP
17+
- **NVIDIA and AMD GPU support**: Separate optimized images for each GPU vendor
18+
19+
## GPU Requirements
20+
21+
ComfyUI is GPU-only. The service definition is split by GPU vendor:
22+
23+
| Backend | Compose file | Notes |
24+
|---------|-------------|-------|
25+
| NVIDIA (CUDA) | `compose.nvidia.yaml` | Requires NVIDIA Container Toolkit |
26+
| AMD (ROCm) | `compose.amd.yaml` | Targets gfx1151 (RX 9000 series); uses ROCm with flash attention |
27+
28+
> **Apple Silicon:** ComfyUI is not currently configured for Apple Silicon (macOS ARM). Use the native ComfyUI application instead.
29+
30+
## Configuration
31+
32+
Environment variables (set in `.env`):
33+
34+
| Variable | Default | Description |
35+
|----------|---------|-------------|
36+
| `COMFYUI_PORT` | 8188 | External port for the ComfyUI web UI and API |
37+
38+
## Volume Mounts
39+
40+
### NVIDIA
41+
42+
| Host Path | Container Path | Purpose |
43+
|-----------|---------------|---------|
44+
| `./data/comfyui/models` | `/models` | AI model files (checkpoints, LoRAs, VAEs, etc.) |
45+
| `./data/comfyui/output` | `/output` | Generated images output directory |
46+
| `./data/comfyui/input` | `/input` | Input images for img2img and inpainting |
47+
| `./data/comfyui/workflows` | `/workflows` | Workflow JSON templates (read-only) |
48+
49+
### AMD
50+
51+
| Host Path | Container Path | Purpose |
52+
|-----------|---------------|---------|
53+
| `./data/comfyui/ComfyUI` | `/opt/ComfyUI` | Full ComfyUI installation (models, outputs, custom nodes) |
54+
55+
> **Note:** The AMD image uses a single volume containing the entire ComfyUI directory. Models go inside `./data/comfyui/ComfyUI/models/` rather than a separate `./data/comfyui/models/` mount.
56+
57+
### Model Subdirectories (NVIDIA)
58+
59+
Place model files in the appropriate subdirectory under `./data/comfyui/models/`:
60+
61+
| Subdirectory | Model type |
62+
|-------------|------------|
63+
| `checkpoints/` | Full Stable Diffusion / FLUX checkpoints |
64+
| `diffusion_models/` | Standalone diffusion model weights |
65+
| `text_encoders/` | CLIP and T5 text encoders |
66+
| `vae/` | Variational Autoencoders |
67+
| `loras/` | LoRA fine-tuned weights |
68+
| `latent_upscale_models/` | Latent upscale models |
69+
70+
## Architecture
71+
72+
```
73+
┌──────────┐ HTTP :8188 ┌──────────────┐
74+
│ Browser │───────────────▶│ ComfyUI │
75+
│ (Node UI)│◀───────────────│ (PyTorch) │
76+
└──────────┘ └──────┬───────┘
77+
│
78+
┌──────────────┼──────────────┐
79+
▼ ▼ ▼
80+
/models/ /output/ /input/
81+
(checkpoints, (generated (source
82+
LoRAs, VAEs) images) images)
83+
```
84+
85+
## API Endpoints
86+
87+
| Endpoint | Method | Description |
88+
|----------|--------|-------------|
89+
| `GET /` | GET | Web UI / health check |
90+
| `POST /prompt` | POST | Queue a generation workflow |
91+
| `GET /queue` | GET | View current generation queue |
92+
| `GET /history` | GET | View completed generation history |
93+
| `GET /view` | GET | Retrieve a generated image by filename |
94+
| `GET /system_stats` | GET | GPU memory and system resource stats |
95+
96+
## Files
97+
98+
- `manifest.yaml` — Service metadata (port, health endpoint, GPU backends, features)
99+
- `compose.yaml` — Base stub (actual definition is in GPU overlays)
100+
- `compose.nvidia.yaml` — NVIDIA CUDA service definition
101+
- `compose.amd.yaml` — AMD ROCm service definition (gfx1151)
102+
- `startup.sh` — Entrypoint script: sets up model symlinks and launches ComfyUI server
103+
- `Dockerfile` — Container build definition (used by NVIDIA overlay)
104+
105+
## Troubleshooting
106+
107+
**ComfyUI not starting (long start period):**
108+
109+
The container has a 120-second start period to allow model loading. Wait for it to elapse, then check:
110+
```bash
111+
docker compose ps dream-comfyui
112+
docker compose logs dream-comfyui --follow
113+
```
114+
115+
**GPU not detected:**
116+
117+
For NVIDIA:
118+
```bash
119+
# Verify NVIDIA Container Toolkit is installed
120+
docker run --rm --gpus all nvidia/cuda:12.0-base nvidia-smi
121+
```
122+
123+
For AMD:
124+
```bash
125+
# Verify GPU device access
126+
ls /dev/dri /dev/kfd
127+
```
128+
129+
**Models not appearing in the UI:**
130+
- Ensure model files are placed in the correct subdirectory under `./data/comfyui/models/`
131+
- Restart ComfyUI or click **Refresh** in the model loader node
132+
133+
**Out of VRAM errors:**
134+
- Use smaller or quantized model variants
135+
- Close other GPU-intensive services before running ComfyUI
136+
- Check VRAM usage: `nvidia-smi` (NVIDIA) or `rocm-smi` (AMD)
137+
138+
**Generated images not saving:**
139+
- Verify `./data/comfyui/output` exists and is writable
140+
- Check container logs for permission errors

‎dream-server/extensions/services/dashboard-api/README.md‎

Lines changed: 190 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,190 @@
1+
# dashboard-api
2+
3+
FastAPI backend providing system status, metrics, and management for Dream Server
4+
5+
## Overview
6+
7+
The Dashboard API is a Python FastAPI service that powers the Dream Server Dashboard UI. It exposes endpoints for GPU metrics, service health monitoring, LLM inference stats, workflow management, agent monitoring, setup wizard, version checking, and Privacy Shield control.
8+
9+
It runs at `http://localhost:3002` and is the single backend used by the React dashboard frontend.
10+
11+
## Features
12+
13+
- **GPU monitoring**: Real-time VRAM usage, temperature, utilization, and power draw (NVIDIA + AMD)
14+
- **Service health**: Health checks for all Dream Server services via Docker network
15+
- **LLM metrics**: Tokens/second, lifetime tokens, loaded model, context size
16+
- **System metrics**: CPU usage, RAM usage, uptime, disk space
17+
- **Workflow management**: n8n workflow catalog — install, enable, disable, track executions
18+
- **Feature discovery**: Hardware-aware feature recommendations with VRAM tier detection
19+
- **Setup wizard**: First-run setup, persona selection, diagnostic tests
20+
- **Agent monitoring**: Session counts, throughput, cluster status, per-model token usage
21+
- **Privacy Shield control**: Enable/disable container, fetch PII scrubbing statistics
22+
- **Version checking**: GitHub releases integration for update notifications
23+
- **Storage reporting**: Breakdown of disk usage by models, vector DB, and total data
24+
25+
## Configuration
26+
27+
Environment variables (set in `.env`):
28+
29+
| Variable | Default | Description |
30+
|----------|---------|-------------|
31+
| `DASHBOARD_API_PORT` | `3002` | External + internal port |
32+
| `DASHBOARD_API_KEY` | *(auto-generated)* | API key for all protected endpoints. If unset, a random key is generated and written to `/data/dashboard-api-key.txt` |
33+
| `GPU_BACKEND` | `nvidia` | GPU backend: `nvidia` or `amd` |
34+
| `OLLAMA_URL` | `http://llama-server:8080` | LLM backend URL |
35+
| `LLM_MODEL` | `qwen3:30b-a3b` | Active model name shown in dashboard |
36+
| `KOKORO_URL` | `http://tts:8880` | Kokoro TTS URL |
37+
| `N8N_URL` | `http://n8n:5678` | n8n workflow URL |
38+
| `OPENCLAW_TOKEN` | *(empty)* | OpenClaw agent auth token |
39+
40+
## API Endpoints
41+
42+
### Core
43+
44+
| Method | Path | Auth | Description |
45+
|--------|------|------|-------------|
46+
| `GET` | `/health` | No | Health check |
47+
| `GET` | `/gpu` | Yes | GPU metrics (VRAM, temp, utilization) |
48+
| `GET` | `/services` | Yes | All service health statuses |
49+
| `GET` | `/disk` | Yes | Disk usage |
50+
| `GET` | `/model` | Yes | Current model info |
51+
| `GET` | `/bootstrap` | Yes | Model bootstrap/download status |
52+
| `GET` | `/status` | Yes | Full system status (all above combined) |
53+
| `GET` | `/api/status` | Yes | Dashboard-formatted status with inference metrics |
54+
55+
### Preflight
56+
57+
| Method | Path | Auth | Description |
58+
|--------|------|------|-------------|
59+
| `GET` | `/api/preflight/docker` | Yes | Check Docker availability |
60+
| `GET` | `/api/preflight/gpu` | Yes | Check GPU availability |
61+
| `GET` | `/api/preflight/required-ports` | No | List service ports |
62+
| `POST` | `/api/preflight/ports` | Yes | Check port availability conflicts |
63+
| `GET` | `/api/preflight/disk` | Yes | Check available disk space |
64+
65+
### Settings & Storage
66+
67+
| Method | Path | Auth | Description |
68+
|--------|------|------|-------------|
69+
| `GET` | `/api/service-tokens` | Yes | Service auth tokens (e.g. OpenClaw) |
70+
| `GET` | `/api/external-links` | Yes | Sidebar links from service manifests |
71+
| `GET` | `/api/storage` | Yes | Storage breakdown (models, vector DB, total) |
72+
73+
### Workflows (n8n integration)
74+
75+
| Method | Path | Auth | Description |
76+
|--------|------|------|-------------|
77+
| `GET` | `/api/workflows` | Yes | Workflow catalog with install status |
78+
| `POST` | `/api/workflows/{id}/enable` | Yes | Import and activate a workflow in n8n |
79+
| `DELETE` | `/api/workflows/{id}` | Yes | Remove a workflow from n8n |
80+
| `GET` | `/api/workflows/{id}/executions` | Yes | Recent execution history |
81+
82+
### Features
83+
84+
| Method | Path | Auth | Description |
85+
|--------|------|------|-------------|
86+
| `GET` | `/api/features` | Yes | Feature status with hardware recommendations |
87+
| `GET` | `/api/features/{id}/enable` | Yes | Enable instructions for a feature |
88+
89+
### Setup Wizard
90+
91+
| Method | Path | Auth | Description |
92+
|--------|------|------|-------------|
93+
| `GET` | `/api/setup/status` | Yes | First-run check and current step |
94+
| `GET` | `/api/setup/personas` | Yes | List available personas |
95+
| `GET` | `/api/setup/persona/{id}` | Yes | Get persona details |
96+
| `POST` | `/api/setup/persona` | Yes | Select a persona |
97+
| `POST` | `/api/setup/complete` | Yes | Mark setup complete |
98+
| `POST` | `/api/setup/test` | Yes | Run diagnostic tests (streaming) |
99+
| `POST` | `/api/chat` | Yes | Quick chat for setup wizard |
100+
101+
### Agent Monitoring
102+
103+
| Method | Path | Auth | Description |
104+
|--------|------|------|-------------|
105+
| `GET` | `/api/agents/metrics` | Yes | Full agent metrics (sessions, tokens, cost) |
106+
| `GET` | `/api/agents/metrics.html` | Yes | Agent metrics as HTML fragment (htmx) |
107+
| `GET` | `/api/agents/cluster` | Yes | Cluster health and GPU node status |
108+
| `GET` | `/api/agents/throughput` | Yes | Throughput stats (tokens/sec) |
109+
110+
### Privacy Shield
111+
112+
| Method | Path | Auth | Description |
113+
|--------|------|------|-------------|
114+
| `GET` | `/api/privacy-shield/status` | Yes | Privacy Shield container status |
115+
| `POST` | `/api/privacy-shield/toggle` | Yes | Start or stop Privacy Shield |
116+
| `GET` | `/api/privacy-shield/stats` | Yes | PII scrubbing statistics |
117+
118+
### Updates
119+
120+
| Method | Path | Auth | Description |
121+
|--------|------|------|-------------|
122+
| `GET` | `/api/version` | Yes | Current version + GitHub update check |
123+
| `GET` | `/api/releases/manifest` | No | Recent release history from GitHub |
124+
| `POST` | `/api/update` | Yes | Trigger update actions (`check`, `backup`, `update`) |
125+
126+
## Authentication
127+
128+
When `DASHBOARD_API_KEY` is set in `.env`, all authenticated endpoints require the key:
129+
130+
```bash
131+
curl http://localhost:3002/api/status \
132+
-H "Authorization: Bearer YOUR_KEY"
133+
```
134+
135+
When `DASHBOARD_API_KEY` is empty (default), all endpoints are accessible without authentication.
136+
137+
## Architecture
138+
139+
```
140+
Dashboard UI (:3001)
141+
│
142+
▼
143+
Dashboard API (:3002)
144+
├── gpu.py ──────────────── nvidia-smi / sysfs AMD
145+
├── helpers.py ──────────── Docker-network health checks
146+
├── agent_monitor.py ─────── Background metrics collection
147+
└── routers/
148+
├── workflows.py ────── n8n API integration
149+
├── features.py ─────── Hardware-aware feature discovery
150+
├── setup.py ─────────── Setup wizard + persona system
151+
├── updates.py ──────── GitHub releases + dream-update.sh
152+
├── agents.py ───────── Agent session + throughput metrics
153+
└── privacy.py ──────── Privacy Shield container control
154+
```
155+
156+
## Files
157+
158+
- `main.py` — FastAPI application, core endpoints, startup
159+
- `config.py` — Shared configuration and manifest loading
160+
- `models.py` — Pydantic response schemas
161+
- `security.py` — API key authentication
162+
- `gpu.py` — GPU detection for NVIDIA and AMD
163+
- `helpers.py` — Service health checks, LLM metrics, system metrics
164+
- `agent_monitor.py` — Background agent metrics collection
165+
- `routers/` — Endpoint modules (workflows, features, setup, updates, agents, privacy)
166+
- `Dockerfile` — Container definition
167+
- `requirements.txt` — Python dependencies
168+
169+
## Troubleshooting
170+
171+
**API not responding:**
172+
```bash
173+
docker compose ps dashboard-api
174+
docker compose logs dashboard-api
175+
```
176+
177+
**GPU metrics missing:**
178+
- NVIDIA: confirm `nvidia-smi` works on the host
179+
- AMD: the AMD overlay mounts `/sys/class/drm` — confirm `GPU_BACKEND=amd` in `.env`
180+
181+
**Workflow operations failing:**
182+
- Verify n8n is running: `curl http://localhost:5678/healthz`
183+
- Check `N8N_URL` environment variable
184+
185+
**Storage endpoint returning zeros:**
186+
- The container mounts `./data` at `/data` — verify the path exists
187+
188+
## License
189+
190+
Part of Dream Server — Local AI Infrastructure

0 commit comments

Comments
 (0)