yokai

One binary to deploy, monitor, and manage LLM inference across all your GPUs.

yokai is a terminal-based fleet manager for running vLLM, SGLang, llama.cpp, and ComfyUI on any number of GPU machines. Connect your devices, deploy curated Best-Known-Configs (or roll your own through a guided wizard), and watch everything on a btop-style dashboard -- all from a single binary with zero dependencies on the target machines.

 ┌───────────┐ ╔══════════════════╗ ┌──────────────┐ ┌─────────────┐ ┌───────────────┐
 │  G. Home  │ ║  ▸ 1. Dashboard  ║ │  2. Devices  │ │  3. Deploy  │ │  4. Settings  │
 └───────────┘ ╚══════════════════╝ └──────────────┘ └─────────────┘ └───────────────┘

 ╭────────────────────────────────────────────────────────────────────────────────────╮
 │ Dashboard                                                                          │
 │ Fleet services, inspector details, and contextual live logs.                       │
 │                                                                                    │
 │ ┌─ AI Fleet ───────────────────────────────────────────────────────────────────┐   │
 │ │ GPU  73%  ▁▂▃▅▆▇█▇▆▅▆▇█▇▆▅▆▇█▇▆▅▆▇█▇▆▅▆▇█▇▆▅▆▇█▇▆▅▆▇█▇▆▅▆▇█▇▆▅▆▇█▇▆▅▆▇█▇▆ │   │
 │ │ VRAM 84%  20.1 / 24.0 GB                                                     │   │
 │ │ GPUs 2 active / 2 total · Svc 3 total · 0 alert(s)                           │   │
 │ └──────────────────────────────────────────────────────────────────────────────┘   │
 │                                                                                    │
 │ ╔═ AI Services [2] ════════════════════════════════════════════════════════════╗   │
 │ ║ ▌ vllm-llama-3.1-8b      · Gaming Rig   · live                               ║   │
 │ ║ ●  llamacpp-mistral-7b   · Gaming Rig   · live                               ║   │
 │ ╚══════════════════════════════════════════════════════════════════════════════╝   │
 │                                                                                    │
 │ ┌─ Monitoring Services [1] ────────────────────────────────────────────────────┐   │
 │ │ ●  grafana                · Gaming Rig   · live                              │   │
 │ └──────────────────────────────────────────────────────────────────────────────┘   │
 ╰────────────────────────────────────────────────────────────────────────────────────╯

 ╭────────────────────────────────────────────────────────────────────────────────────╮
 │ G home  |  Tab next section  |  Shift+Tab previous section  |  1-4 jump            │
 ╰────────────────────────────────────────────────────────────────────────────────────╯

Why yokai?

If you're running local LLMs across multiple machines, you already know the pain:

SSH into each box just to check if your GPU is melting or idle
Copy-paste 20-flag docker run commands every time you want to swap a model
Juggle separate monitoring dashboards for each machine
Manually edit tool configs every time an endpoint changes

yokai solves all of this with a single binary. Install it, point it at your machines, and you're running models in minutes -- not hours.

Features

Fleet Management

Onboarding wizard -- connect devices via LAN scan, Tailscale peer discovery, or manual IP entry
GPU-aware Tailscale discovery -- surface peer tags and highlight dedicated compute nodes carrying Yokai's recommended tag:ai-gpu identity
SSH bootstrap -- pre-flight checks (Docker, GPU, disk space), agent deployment, and systemd service installation in one step
Device manager -- add, edit, remove, and test connectivity for all devices from the TUI
Secure by default -- auto-generated bearer tokens for agent authentication, SSH key resolution with agent/key/password fallback

Workload Deployment

Best Known Configs (BKC) -- pick from a built-in catalog of pre-validated vLLM, SGLang, and llama.cpp deploys grouped by vendor (NVIDIA, OpenAI, Meta/Llama, Google, Mistral, Qwen, DeepSeek, GLM, Moonshot, Microsoft, and more), filtered to the GPUs each device actually has, with sibling recipes selectable when a model has multiple validated runtimes
Guided deploy wizard -- pick workload type, target device, Docker image, model, and runtime config when you want to deviate from the catalog
HuggingFace integration -- search models directly, browse GGUF quantizations, auto-download during deployment
VRAM estimator -- hf-mem-backed memory estimate for vLLM weights + KV cache before you commit to a deploy
Docker image catalog -- browse official tags from Docker Hub and GHCR, including nightly builds
Plugin system -- model-specific add-ons (e.g. Nemotron Super V3 reasoning parser) that fetch assets, mount them into the container, and append the right runtime flags
GPU-aware deployment -- automatic --gpus flag configuration, multi-GPU tensor parallelism support

Live Monitoring

btop-style dashboard -- live GPU utilization, VRAM, temperature, power draw, and fan speed per GPU
System metrics -- CPU/RAM sparklines with 60-sample rolling history, disk usage, swap
Container metrics -- per-container CPU, memory, GPU memory, and uptime
Service lifecycle -- stop, restart, and remove containers directly from the dashboard

Monitoring Stack

Auto-provisioned -- Prometheus + Grafana + node_exporter + dcgm-exporter seeded onto each device during bootstrap, with the agent bearer token wired in for authenticated scrapes
Pre-built dashboards -- Grafana dashboard with GPU utilization, temperature, power, and system metrics panels
Live in the dashboard -- the monitoring stack shows up as its own services panel alongside your AI workloads

AI Coding Tool Integration

One-shot config -- the Settings route auto-configures every supported AI coding tool at once
VS Code Copilot -- appends OpenAI-compatible models to chat.models[] in the VS Code user settings.json
OpenCode -- registers a per-host provider in ~/.config/opencode/opencode.json via @ai-sdk/openai-compatible
OpenClaw -- adds a yokai provider under models.providers in ~/.openclaw/openclaw.json
Claude Code -- writes ANTHROPIC_BASE_URL, ANTHROPIC_MODEL, and ANTHROPIC_CUSTOM_MODEL_OPTION into ~/.claude/settings.json
Codex -- adds a [model_providers.yokai] section to ~/.codex/config.toml
Backup-safe -- writes a .yokai.bak of every config before modifying it
Multi-endpoint -- registers every running inference service as an available endpoint

Self-Updating

yokai upgrade -- checks GitHub Releases, downloads the correct binary for your OS/arch, and replaces itself in place
Cross-platform -- builds for Linux, macOS, and Windows (amd64 + arm64)
One-line install -- curl | sh installer that detects your platform automatically

Quick Start

Install

One-line installer (Linux/macOS):

curl -fsSL https://raw.githubusercontent.com/spencerbull/Yokai/main/install.sh | sh

From GitHub Releases:

Download the latest binary for your platform from the Releases page, extract it, and move it to your PATH.

Build from source:

git clone https://github.com/spencerbull/yokai.git
cd yokai
make build
./bin/yokai

First Run

# 1. Launch yokai -- OpenTUI starts and auto-starts the local daemon if needed
yokai

# 2. Select how to connect (LAN / Tailscale / Manual IP)
# 3. Enter SSH credentials -- yokai tests the connection
# 4. Bootstrap runs: pre-flight checks → agent deploy → monitoring stack
# 5. Set up your HuggingFace token (optional, for gated models)
# 6. Tab over to "Deploy" and pick a Best-Known-Config to run your first model

Tag GPU compute nodes in Tailscale

Yokai can import online peers from tailscale status --json. Dedicated GPU servers tagged with tag:ai-gpu receive an AI GPU badge and are easier to identify in the device picker. The tag is recommended, not required.

Define tag:ai-gpu and its owner in your tailnet policy, then apply it from Tailscale Admin Console -> Machines -> device -> Edit tags or authenticate the dedicated server with the tag:

sudo tailscale login --advertise-tags=tag:ai-gpu
sudo tailscale up --advertise-tags=tag:ai-gpu --force-reauth

Only tag dedicated, non-human server nodes: Tailscale tags replace user-based device identity and do not grant network or SSH access. See the full Tailscale GPU-node guide for tagOwners, access-policy examples, Yokai verification, and troubleshooting.

Running the Daemon

The daemon is a background process on your local machine that maintains SSH tunnels to your devices and polls agents for live metrics. yokai now auto-starts it on first launch, but you can still run it manually:

# Start the daemon in the background
yokai daemon &

# Launch OpenTUI (auto-starts daemon if needed)
yokai

Usage

Commands

Core

Command	Description
`yokai`	Launch OpenTUI (default; auto-starts the daemon)
`yokai agent [port]`	Run the agent on a target device (default port: 7474)
`yokai daemon`	Start the local background daemon (default `127.0.0.1:7473`)
`yokai upgrade`	Update to the latest release
`yokai version`	Print version and build info

Device management

Command	Description
`yokai devices list`	List all configured devices
`yokai devices add --host <host> [flags]`	Add a device
`yokai devices remove <device-id>`	Remove a device
`yokai devices test <device-id>`	Test SSH + agent connectivity
`yokai devices bootstrap <device-id>`	Install or upgrade the agent on a device

Service management

Command	Description
`yokai services list [--device <id>]`	List containers across the fleet (or one device)
`yokai services deploy --device <id> [flags]`	Deploy a service
`yokai services stop <device-id> <cid>`	Stop a container
`yokai services restart <device-id> <cid>`	Restart a container
`yokai services logs [--follow] <did> <cid>`	Stream container logs

Coordinated deployments

Command	Description
`yokai deployments create [flags]`	Create an atomic multi-device deployment
`yokai deployments list`	List deployment groups, including stopped groups
`yokai deployments status <deployment-id>`	Show one deployment group
`yokai deployments test --api-key-env NAME <deployment-id>`	Test the rank-0 API/model/metrics with a request-time key
`yokai deployments start --api-key-env NAME <deployment-id>`	Start a stopped group head-first using its original launch key and repeat readiness
`yokai deployments stop <deployment-id>`	Stop every managed member in the group
`yokai deployments rollback <deployment-id>`	Remove candidates and restart explicitly selected prior containers

Fleet status & config

Command	Description
`yokai status`	Fleet overview (JSON)
`yokai metrics [--device <id>]`	Detailed device metrics (JSON)
`yokai config show`	Dump config (tokens redacted)
`yokai config set <key> <value>`	Set a config value
`yokai config path`	Print the config file path

All non-TUI commands emit JSON on stdout and JSON-formatted errors on stderr, so they're scriptable from shell or another tool.

Keybinds

Global

Key	Action
`Tab` / `Shift+Tab`	Cycle between Dashboard / Devices / Deploy / Settings
`1` … `4`	Jump to a top-level route by number
`g`	Return to the landing screen
`Esc` (twice) or `Ctrl+C` (twice)	Quit

Dashboard

Key	Action
`j` / `k` (or `↑` / `↓`)	Move selection within the service list
`Tab`	Switch between AI services and the monitoring stack
`Enter` or `l`	Open the selected service's detail view
`Shift+L`	Open logs for the selected service
`s` / `r` / `t` / `x`	Stop / restart / test / delete (in service detail)
`y` / `n`	Confirm or cancel a destructive action

How It Works

yokai uses a three-tier architecture: OpenTUI (what you see, a TypeScript app rendered in your terminal), the Daemon (runs locally and brokers everything), and the Agent (runs on each GPU device).

Your Machine                              GPU Device(s)
┌──────────────────────┐                  ┌──────────────────────┐
│  OpenTUI (Bun/Node)  │                  │  yokai agent         │
│  ├── Dashboard       │   HTTP :7473     │  ├── REST API :7474  │
│  ├── Deploy Wizard   │◄───────────────► │  ├── nvidia-smi      │
│  ├── Device Manager  │                  │  ├── Docker engine   │
│  └── Log Viewer      │                  │  └── System metrics  │
│         ▲            │                  │                      │
│         │ launches   │                  │  Docker containers   │
│         ▼            │   SSH tunnel     │  ├── vLLM :8000      │
│  yokai daemon        │◄═══════════════► │  ├── SGLang :30000   │
│  ├── SSH Tunnels     │                  │  ├── llama.cpp :8080 │
│  ├── Metrics Agg.    │                  │  └── ComfyUI :8188   │
│  ├── BKC Catalog     │                  │  Monitoring stack    │
│  ├── HF + hf-mem     │                  │  ├── Prometheus      │
│  └── Tool Configs    │                  │  ├── Grafana         │
│                      │                  │  ├── node_exporter   │
│  ~/.config/yokai/    │                  │  └── dcgm-exporter   │
│  └── config.json     │                  │                      │
└──────────────────────┘                  └──────────────────────┘

Launch sequence

yokai checks whether the daemon is healthy; if not, it spawns yokai daemon in the background and waits for /health
It then runs the bundled yokai-tui binary (preferred) or, if you're on a dev checkout with bun installed, bun run src/index.tsx from ui/tui/ -- the daemon URL is passed via YOKAI_DAEMON_URL
The TUI talks to the daemon over HTTP on 127.0.0.1:7473

Data flow

The daemon opens SSH tunnels to each device and forwards a local port to each agent's :7474 REST API
Every few seconds it polls each agent's /metrics endpoint and caches the results in memory
The TUI reads aggregated metrics from the daemon and renders the dashboard
Deploy and lifecycle commands (deploy/stop/restart/remove/logs) flow TUI -> daemon -> agent -> Docker

Configuration

Legacy device and service state lives in ~/.config/yokai/config.json. Coordinated deployment state is stored separately in the versioned ~/.config/yokai/deployments.json; older config writers therefore cannot silently discard deployment groups.

{
  "version": 1,
  "hf_token": "hf_...",
  "daemon": {
    "listen": "127.0.0.1:7473",
    "metrics_poll_interval_s": 2,
    "reconnect_interval_s": 30
  },
  "devices": [
    {
      "id": "gaming-rig",
      "label": "Gaming Rig",
      "host": "100.64.0.2",
      "ssh_user": "user",
      "connection_type": "tailscale",
      "agent_port": 7474,
      "agent_token": "a1b2c3...",
      "gpu_type": "nvidia",
      "tags": ["rtx-4090"],
      "monitoring_installed": true
    }
  ],
  "services": [
    {
      "id": "yokai-vllm-llama3",
      "device_id": "gaming-rig",
      "type": "vllm",
      "image": "vllm/vllm-openai:latest",
      "model": "meta-llama/Llama-3.1-8B-Instruct",
      "port": 8000,
      "plugins": [],
      "runtime": { "ipc_mode": "host", "shm_size": "16g" }
    }
  ],
  "preferences": {
    "theme": "tokyonight",
    "default_vllm_image": "vllm/vllm-openai:latest",
    "default_sglang_image": "lmsysorg/sglang:latest",
    "default_llama_image": "ghcr.io/ggml-org/llama.cpp:server-cuda",
    "default_comfyui_image": "yanwk/comfyui-boot:latest"
  }
}

Best Known Configurations (BKC)

The BKC catalog is a library of pre-validated deploy recipes. Each entry pins the Docker image, tensor-parallel size, quantization flags, GPU memory utilization, chat template, tool-call parser, and any runtime options (--ipc=host, --shm-size, ulimits) needed for a given model on a given GPU. The deploy wizard matches your model against the catalog, filters recipes by the target device's VRAM and GPU count, lets you cycle through sibling recipes, and offers one-click apply.

95 serving configs (91 vLLM plus four SGLang) across 89 models from 26 publishers (as of this commit). Entries are grouped by publisher and live in internal/bkc/catalog_*.go:

Publisher	Unique models	Serving configs	Catalog file
`Qwen`	20	20	`catalog_qwen.go`
`RadixArk`	2	3 SGLang	`catalog_qwen.go`
`nvidia`	14	17	`catalog_nvidia.go`, `catalog_google.go`, `catalog_llama.go`, `catalog_moonshotai.go`, `catalog_qwen.go`
`zai-org` (GLM)	7	7	`catalog_glm.go`
`LibertAIDAI`	1	1 SGLang	`catalog_glm.go`
`deepseek-ai`	6	6	`catalog_deepseek.go`
`google`	5	5	`catalog_google.go`
`mistralai`	5	5	`catalog_mistral.go`
`baidu` (ERNIE)	4	4	`catalog_others.go`
`moonshotai`	4	4	`catalog_moonshotai.go`
`openai`	2	4	`catalog_openai.go`
`meta-llama`	2	2	`catalog_llama.go`
`internlm`	2	2	`catalog_others.go`
`tencent`	2	2	`catalog_others.go`
`microsoft` (Phi)	1	1	`catalog_microsoft.go`
`amd`	1	1	`catalog_openai.go`
`ByteDance-Seed`	1	1	`catalog_others.go`
`MiniMaxAI`	1	1	`catalog_others.go`
`OpenGVLab` (InternVL)	1	1	`catalog_others.go`
`PaddlePaddle`	1	1	`catalog_others.go`
`XiaomiMiMo`	1	1	`catalog_others.go`
`arcee-ai`	1	1	`catalog_others.go`
`inclusionAI`	1	1	`catalog_others.go`
`jinaai`	1	1	`catalog_others.go`
`sakamakismile`	2	2	`catalog_others.go`
`stepfun-ai`	1	1	`catalog_others.go`
Total	89	95

Notable recipes

Frontier reasoning / MoE — DeepSeek R1 / V3.1 / V3.2 (FP8, 8× H200/B200), Qwen3-Coder-480B-A35B, Qwen3.5-397B-A17B-FP8, Qwen3-Next-80B-A3B (BF16 and FP8 variants), Moonshot Kimi K2.
Gated Meta / NVIDIA — Llama 3.3 70B (BF16 on Meta base, FP8 on Hopper, FP4 on Blackwell), nvidia/Llama-3.1-Nemotron-Ultra / Super, NGC images validated on DGX Spark GB10 and Jetson Thor (aarch64).
Vision-language — Qwen2.5-VL-7B/72B, Qwen3-VL-235B (BF16 + FP8), InternVL3.5-8B, ERNIE-4.5-VL-28B / VL-424B, PaddleOCR-VL, DeepSeek-OCR.
Small / edge-friendly — Qwen3-0.6B / 1.7B / 4B / 8B, Qwen3Guard-Gen-0.6B, Phi-4, Gemma 3 2B/4B/12B, validated for RTX 4090, RTX 5090, L40S, GB10, and Jetson Thor.
AMD CDNA4 — amd/gpt-oss-120b-w-mxfp4-a-fp8 tuned for MI355X with the ROCm vLLM image.
SGLang speculative decoding — sibling Qwen3.8 27B NVFP4 recipes for DFlash2 and DSpark on RTX PRO 6000, with pinned target/drafter/image revisions, 262K context, FlashInfer, FP8 KV cache, and native metrics. Finn's validated DFlash2 profile is the default and supports up to eight requests sharing its KV pool; DSpark remains available as the three-request rollback recipe.
Dual DGX Spark — glm-5-3-flash-nvfp4-dual-gb10 pins GLM-5.3-Flash NVFP4, its model revision, and its SGLang image digest for torch-distributed TP=2 across two one-GPU GB10 nodes. Rank 0 is the API/head on an explicitly supplied client/monitor address at port 8000, rank 1 is the worker, metrics are enabled, and both the SGLang watchdog and torch-distributed timeout are pinned to one hour. The pinned image does not contain the required CUDA GB10 tile override, so Yokai carries two ordered, exact-hash runtime patches: the 480-to-3600-second loader guard (f0193bfaab96053919e3a260f9ccb10e2137ad108cfae03816c867628f611e1f to fc3ae35cce5f712fd3681dc4bcef05157df6d232ac991ce60b78dae492784ff5) and sglang-dsa-gb10-tile-tp2-v1 (526988aa5fd8fa61529f2d3cf245d1061e30982d2bd0d6e64ea2e51e31f30f7f to 3150ec691843c84bb5db9ed5bf763c4da03842fde4666489e107bf7a9fcddd7b). The latter selects the 32/1/128 TileLang tile only for the no-tail DSA path and is correctness-bound to TP=2: its unaliased shared-memory request is 100352 bytes before pipeline overhead. The bootstrap preflights both complete files and line counts before writing either, verifies both final files, and requires the GPU's opt-in per-block shared-memory limit to be at least 100352 bytes. A live two-GB10 canary validated this exact combination: both ranks reported the 101376-byte GB10 limit, crossed the old 480-second barrier, compiled and warmed without the stock tile's 169984-byte failure, and passed chat, Responses, tool, image, 59.8K retrieval, two-request concurrency, metrics, log, OOM/restart, and thermal gates. The SGLang log level is warning to keep request-time API keys out of the affected info-level server-argument projection, and no MTP is enabled. No host-specific IP is stored in the catalog.

Device-aware selection

BKC entries carry explicit hardware tags so the deploy wizard can hand you the right recipe for the device you selected:

NVIDIA: gb10 (DGX Spark), jetson-thor, rtx-pro-6000, rtx-5090, rtx-4090, l40s, a100-80, h100-80, h100-94, h200, h20, b200, gb200
AMD: mi300x, mi325x, mi355x, radeon-r9700

When multiple BKCs target the same model (for example, Qwen3.8 27B has DFlash2 and DSpark speculative decoders), the daemon puts the most specialised recipe whose TargetDevices include your device profile first. The deploy wizard exposes the remaining sibling recipes with Previous/Next controls, then falls back to the first recipe whose MinVRAMGBPerGPU and MinGPUCount the device can satisfy.

Adding a BKC

Drop a new entry into the appropriate catalog_<vendor>.go file (create one if your vendor doesn't have one yet) and register it in the catalog slice in catalog_data.go. Every entry needs a stable ID, a human-readable Name, the Docker image, port, exact runtime flags, and the hardware gates (TargetDevices, MinVRAMGBPerGPU, MinGPUCount, Quantization, Arch). See catalog_llama.go for a multi-variant example.

Coordinated two-device deployments

Multi-device recipes carry typed cluster metadata and cannot be submitted to the legacy POST /deploy API. Submit explicit head and worker bindings to POST /deployments instead. Both device IDs must be distinct, both fabric addresses must be valid non-loopback IPs, and the head binding must include a separate explicit client/monitor IP (Tailnet or another routable IP) and recipe port 8000; rendezvous remains on the private head fabric IP at port 25000. Unspecified and loopback service binds are rejected. Both online agents must advertise the required capabilities and expose exactly one GB10 GPU before Yokai writes a journal or mutates Docker.

For glm-5-3-flash-nvfp4-dual-gb10, Yokai stages both pinned images, stops only explicitly selected observed old container IDs, launches head then worker, waits for both members, and promotes only after the rank-0 API/model test passes. Before removing candidates after a failure, rollback best-effort captures and persists each rank's sanitized final 2,000 lines, bounded to 256 KiB; drained partial output is retained as truncated, while API keys, credential-shaped arguments, unsafe control bytes, and a potentially partial raw-boundary line are removed. It then removes candidates in reverse order, restarts selected old IDs, and persists the rollback result. Yokai never auto-adopts external containers; unselected external containers remain visible and testable but immutable. Existing managed-only container and metrics views are unchanged, and stopped services and deployment groups remain visible.

The API key is required and request-scoped in Yokai. With the CLI, pass the name of an environment variable via --api-key-env on create, test, and start; OpenTUI uses a masked transient field. Starting a stopped group restarts its existing containers, so the operator must supply the original launch key; start cannot rotate it, and a new key requires a new deployment. Yokai sends a single structured --api-key=<value> token only to rank 0 and omits it from the BKC, deployments.json, deployment responses, inventory, and Yokai logs. SGLang has no documented secret-fd input, so Docker Config.Cmd and rank 0's process argv necessarily retain the key while that container exists. An optional --local-model-path must be an explicit clean absolute path that is an existing readable directory at the same location on both devices. Yokai mounts it read-only at /models/yokai-deployment and never changes legacy host-path behavior.

Example (the private fabric must permit TCP rendezvous on port 25000; clients and monitors keep using the configured head Tailnet address on port 8000):

export YOKAI_CANARY_API_KEY='replace-me'
yokai deployments create \
  --bkc glm-5-3-flash-nvfp4-dual-gb10 \
  --idempotency-key glm53-canary-001 \
  --head-device spark-a --head-fabric 10.20.0.10 \
  --head-service-address 100.100.100.100 --head-service-port 8000 \
  --worker-device spark-b --worker-fabric 10.20.0.11 \
  --api-key-env YOKAI_CANARY_API_KEY \
  --local-model-path /srv/models/glm-5.3-flash-nvfp4

The addresses above are illustrative, not catalog defaults. The local snapshot must be pre-staged and readable by Docker on both hosts. Yokai does not bootstrap the fabric or copy model data. Its rollback log tails are bounded and sanitized as described above, but the live canary should still independently confirm that the key is absent from complete server logs.

Supported Platforms

User Machine (where you run `yokai`)

OS	Architecture	Status
Linux	amd64	Supported
Linux	arm64	Supported
macOS	amd64	Supported
macOS	arm64 (Apple Silicon)	Supported
Windows	amd64	Supported
Windows	arm64	Supported

Target Devices (where models run)

Requirement	Details
OS	Linux (Ubuntu 20.04+, Debian 11+, or similar)
Docker	20.10+ with the Docker CLI available
GPU	NVIDIA with `nvidia-container-toolkit` installed
Network	Reachable via SSH (LAN, Tailscale, or public IP)
Disk	20GB+ free recommended (vLLM images are 10GB+)

Architecture Documentation

Detailed multi-level architecture docs are available in the architecture/ directory:

Level	Document	What It Covers
L1	System Overview	Bird's-eye view of the system
L2	Component Architecture	Go package structure and dependencies
L3	Data Flow	Metrics, deployment, logging, and config data paths
L4	Network Topology	SSH tunnels, ports, authentication, Tailscale
L5	TUI Screen Map	View hierarchy and navigation state machine
L6	Agent API	Full REST API contract with JSON schemas
L7	Daemon UI API	Daemon REST API consumed by OpenTUI

Project Structure

yokai/
├── cmd/yokai/             # Binary entry point and subcommand dispatch
├── internal/
│   ├── agent/             # Remote agent: REST API, Docker ops, system metrics (port :7474)
│   ├── bkc/               # Best Known Configs catalog (per-vendor catalog_*.go files)
│   ├── claudecode/        # Claude Code (~/.claude/settings.json) endpoint registration
│   ├── cli/               # Non-TUI subcommand handlers (devices/services/status/metrics/config)
│   ├── codex/             # Codex (~/.codex/config.toml) endpoint registration
│   ├── config/            # Config load/save/migrate, deploy history (~/.config/yokai/)
│   ├── daemon/            # Local daemon: REST API (:7473), SSH tunnels, metrics aggregation
│   ├── deployments/       # Versioned atomic multi-device state and transaction engine
│   ├── docker/            # Docker Hub/GHCR tag catalog and image helpers
│   ├── hf/                # HuggingFace API: model search, GGUF listing
│   ├── hfmem/             # `hf-mem` wrapper for vLLM weight + KV-cache memory estimation
│   ├── monitoring/        # Provisions Prometheus/Grafana/exporters onto remote devices
│   ├── openclaw/          # OpenClaw openclaw.json provider registration
│   ├── opencode/          # OpenCode opencode.json provider registration
│   ├── opentui/           # OpenTUI launcher (daemon health check + bundled/Bun spawn)
│   ├── platform/          # Cross-platform shims (e.g. chmod no-op on Windows)
│   ├── plugins/           # Plugin catalog: assets, mounts, runtime-flag overrides per model
│   ├── ssh/               # SSH client, SCP upload, bootstrap/deploy
│   ├── tailscale/         # Tailscale CLI wrapper for peer discovery
│   ├── upgrade/           # Self-update from GitHub Releases
│   └── vscode/            # VS Code settings.json (chat.models[]) manipulation
├── ui/tui/                # OpenTUI: TypeScript/React TUI run by Bun
│   ├── src/
│   │   ├── app/           # App shell and routing
│   │   ├── contracts/     # Daemon API types
│   │   ├── features/      # Dashboard, deploy, devices, logs, integrations, etc.
│   │   ├── services/      # Daemon HTTP client
│   │   └── theme/         # Color palette and styles
│   └── package.json
├── assets/
│   ├── grafana/           # Pre-built dashboard JSON and provisioning
│   ├── prometheus/        # Prometheus scrape configuration
│   └── systemd/           # Agent systemd service template
├── architecture/          # Multi-level architecture documentation
├── docker/                # Reference Dockerfiles (e.g. comfyui/)
├── .github/workflows/     # CI (build/test/lint) and release (GoReleaser)
├── .goreleaser.yml        # Cross-compilation and release config
├── install.sh             # curl-pipe-sh installer
├── Makefile               # Build, test, lint, cross-compile targets
└── go.mod

Contributing

Contributions are welcome. See CONTRIBUTING.md for development setup, coding guidelines, and the pull request process.

License

MIT

Navigation Menu

Repository files navigation

yokai

Why yokai?

Features

Fleet Management

Workload Deployment

Live Monitoring

Monitoring Stack

AI Coding Tool Integration

Self-Updating

Quick Start

Install

First Run

Tag GPU compute nodes in Tailscale

Running the Daemon

Usage

Commands

Keybinds

How It Works

Configuration

Best Known Configurations (BKC)

Notable recipes

Device-aware selection

Adding a BKC

Coordinated two-device deployments

Supported Platforms

User Machine (where you run `yokai`)

Target Devices (where models run)

Architecture Documentation

Project Structure

Contributing

License

About

Releases

Packages

Contributors

Languages

Name		Name	Last commit message	Last commit date
.github		.github
architecture		architecture
assets		assets
cmd/yokai		cmd/yokai
docker		docker
internal		internal
ui/tui		ui/tui
.gitignore		.gitignore
.goreleaser.yml		.goreleaser.yml
.tool-versions		.tool-versions
CONTRIBUTING.md		CONTRIBUTING.md
LICENSE		LICENSE
Makefile		Makefile
README.md		README.md
TODO.md		TODO.md
go.mod		go.mod
go.sum		go.sum
install.sh		install.sh

Latest commit

History

Folders and files

Repository files navigation

yokai

Why yokai?

Features

Fleet Management

Workload Deployment

Live Monitoring

Monitoring Stack

AI Coding Tool Integration

Self-Updating

Quick Start

Install

First Run

Tag GPU compute nodes in Tailscale

Running the Daemon

Usage

Commands

Keybinds

How It Works

Configuration

Best Known Configurations (BKC)

Notable recipes

Device-aware selection

Adding a BKC

Coordinated two-device deployments

Supported Platforms

User Machine (where you run yokai)

Target Devices (where models run)

Architecture Documentation

Project Structure

Contributing

License

About

Topics

Resources

Contributing

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages

User Machine (where you run `yokai`)