feat: slo goodput customization (#23)

* feat: customizable per-model SLO goodput thresholds

The SLO Goodput sub-panel now has a gear icon that opens a small editor
for TTFT/ITL/E2E thresholds. Settings persist in localStorage keyed by
engine+model so each served model keeps its own SLOs across reloads.

Backend ships raw histogram buckets alongside the existing pre-computed
goodput percentages so the frontend can recompute goodput at the user's
custom thresholds without a roundtrip. Pre-computed percentages remain
as the warmup/no-data fallback.

* fix(dev): build and sync frontend/dist so embedded :3000 isn't stale

dev.sh now runs npm run build before the initial sync and stops excluding
frontend/dist from rsync, so the backend cargo build embeds the current
bundle. Adds an optional --watch-frontend flag that rebuilds dist + syncs +
rebuilds backend on frontend changes (off by default — Vite at :5173 stays
the fast path; the flag is for keeping the embedded :3000 build live too).

* feat(frontend): recognize LiquidAI as a model provider

Add LiquidAI to the provider logo registry so models from the
HuggingFace org `LiquidAI/...` and bare LFM/LFM2 model ids resolve
to the new `liquid-ai` SVG asset.

Author: niklasfrick

dev/README.md

@@ -12,15 +12,40 @@ For **production installs**, use `cargo install spark-dashboard` or
 
 Runs the full dev environment:
 
-1. rsyncs the project to `${DEPLOY_USER}@${DEPLOY_HOST}:${DEPLOY_DIR}`
-2. builds and starts the Rust backend on the remote host (`cargo build --release`)
-3. starts the Vite dev server locally on port 5173 with a proxy to the backend
-4. streams remote backend logs from `/tmp/spark-dashboard.log`
-5. watches `src/` and `Cargo.toml` — on change, re-syncs and rebuilds the backend
-
-Frontend edits hot-reload in the browser via Vite. Backend edits trigger a
-remote rebuild (takes about as long as `cargo build --release` does on your
-remote host).
+1. builds the frontend bundle locally (`npm run build` in `frontend/`) so the
+   embedded assets shipped to the remote backend are current
+2. rsyncs the project (including `frontend/dist/`) to `${DEPLOY_USER}@${DEPLOY_HOST}:${DEPLOY_DIR}`
+3. builds and starts the Rust backend on the remote host (`cargo build --release`,
+   which embeds the freshly-built `frontend/dist/`)
+4. starts the Vite dev server locally on port 5173 with a proxy to the backend
+5. streams remote backend logs from `/tmp/spark-dashboard.log`
+6. watches `src/` and `Cargo.toml` — on change, re-syncs and rebuilds the backend
+
+Two URLs, two behaviors:
+
+- `http://localhost:5173` — Vite dev server. Frontend edits hot-reload in the
+  browser, API/WS calls proxy to the remote backend.
+- `http://${DEPLOY_HOST}:3000` — the remote backend serving the **embedded**
+  bundle that was built when `dev.sh` started. To refresh it during a session,
+  re-run `npm run build` locally and trigger any backend file change (or just
+  restart `dev.sh`) so the next sync + `cargo build --release` re-embeds the
+  fresh `frontend/dist/`.
+
+Backend edits trigger a remote rebuild (takes about as long as
+`cargo build --release` does on your remote host).
+
+#### `--watch-frontend` (optional)
+
+Pass `./dev/dev.sh --watch-frontend` to also watch `frontend/src/`,
+`frontend/public/`, `frontend/index.html`, `vite.config.ts`, and
+`package.json`. On change the script rebuilds `frontend/dist/`, re-syncs to the
+remote, and rebuilds the backend — so direct hits on
+`http://${DEPLOY_HOST}:3000` refresh too.
+
+Off by default because each save triggers a full `npm run build` plus
+`cargo build --release` (~10–30s on a typical remote). For normal frontend dev,
+use `:5173` (Vite, instant HMR) and only enable this flag when you specifically
+need the embedded bundle to stay current.
 
 ## Required environment variables
 

dev/dev.sh

@@ -3,6 +3,33 @@ set -euo pipefail
 
 PROJECT_ROOT="$(cd "$(dirname "$0")/.." && pwd)"
 
+# --- CLI flags ---------------------------------------------------------------
+WATCH_FRONTEND=false
+for arg in "$@"; do
+    case "$arg" in
+        --watch-frontend)
+            WATCH_FRONTEND=true
+            ;;
+        -h|--help)
+            cat <<EOF
+Usage: dev.sh [--watch-frontend]
+
+  --watch-frontend  Also watch frontend/ — on change, rebuild frontend/dist,
+                    re-sync, and rebuild the backend so the embedded bundle on
+                    :3000 stays current. Off by default (Vite at :5173 is the
+                    fast path for frontend dev; this flag is for live-updating
+                    the embedded build too, at the cost of a cargo rebuild per
+                    frontend change).
+EOF
+            exit 0
+            ;;
+        *)
+            echo "unknown flag: $arg (try --help)" >&2
+            exit 2
+            ;;
+    esac
+done
+
 # --- Load .env if present ----------------------------------------------------
 if [ -f "$PROJECT_ROOT/.env" ]; then
     set -a
@@ -49,6 +76,18 @@ trap cleanup EXIT INT TERM
 # --- Remote shell prefix: ensure cargo is in PATH for non-interactive SSH ---
 REMOTE_ENV="source ~/.cargo/env 2>/dev/null;"
 
+# --- Build the frontend bundle locally so rust-embed picks up a fresh dist ---
+# Direct hits to the backend on :3000 serve the embedded bundle, so this needs
+# to run before we sync + rebuild the backend.
+build_frontend() {
+    if [ ! -d "${PROJECT_ROOT}/frontend/node_modules" ]; then
+        echo "==> Installing frontend dependencies..."
+        (cd "${PROJECT_ROOT}/frontend" && npm install --silent)
+    fi
+    echo "==> Building frontend bundle (frontend/dist)..."
+    (cd "${PROJECT_ROOT}/frontend" && npm run build)
+}
+
 # --- Sync backend source to remote host ---
 sync_backend() {
     rsync -az --delete \
@@ -58,7 +97,6 @@ sync_backend() {
         --exclude .env \
         --exclude .planning \
         --exclude .claude \
-        --exclude 'frontend/dist' \
         "${PROJECT_ROOT}/" "${REMOTE}:${DEPLOY_DIR}/"
 }
 
@@ -75,6 +113,54 @@ rebuild_backend() {
     fi
 }
 
+# --- Frontend watcher: rebuild dist + sync + rebuild backend on change ---
+# Only launched when --watch-frontend is passed. Heavy: each frontend save
+# triggers `npm run build` plus a remote `cargo build --release`.
+watch_frontend() {
+    local trigger="$1"  # message printed when a change fires
+    local watch_paths=(
+        "${PROJECT_ROOT}/frontend/src"
+        "${PROJECT_ROOT}/frontend/public"
+        "${PROJECT_ROOT}/frontend/index.html"
+        "${PROJECT_ROOT}/frontend/vite.config.ts"
+        "${PROJECT_ROOT}/frontend/package.json"
+    )
+
+    rebuild_embedded() {
+        echo ""
+        echo "==> ${trigger}"
+        build_frontend
+        sync_backend
+        rebuild_backend
+    }
+
+    if command -v fswatch &>/dev/null; then
+        fswatch -0 -r -l 2 \
+            --exclude '.*node_modules.*' \
+            --exclude '.*frontend/dist.*' \
+            "${watch_paths[@]}" \
+        | while IFS= read -r -d '' _; do
+            while read -r -d '' -t 0.5 _ 2>/dev/null; do :; done
+            rebuild_embedded
+        done
+    else
+        local last_hash=""
+        while true; do
+            local current_hash
+            current_hash=$(find "${watch_paths[@]}" \
+                -type f \
+                ! -path '*/node_modules/*' \
+                ! -path '*/dist/*' \
+                -exec stat -f '%m %N' {} + 2>/dev/null | sort | md5)
+            if [ -n "$last_hash" ] && [ "$current_hash" != "$last_hash" ]; then
+                rebuild_embedded
+            fi
+            last_hash="$current_hash"
+            sleep 2
+        done
+    fi
+}
+
 # --- File watcher: fswatch if available, else polling fallback ---
 watch_backend() {
     if command -v fswatch &>/dev/null; then
@@ -110,7 +196,8 @@ watch_backend() {
     fi
 }
 
-# 1. Sync and build backend
+# 1. Build embedded frontend bundle, then sync and build backend
+build_frontend
 echo "==> Syncing to ${REMOTE}:${DEPLOY_DIR}..."
 sync_backend
 rebuild_backend
@@ -119,21 +206,15 @@ rebuild_backend
 ssh "${REMOTE}" "tail -n0 -f /tmp/spark-dashboard.log" 2>/dev/null &
 PIDS+=($!)
 
-# 3. Install frontend deps if needed
-if [ ! -d "${PROJECT_ROOT}/frontend/node_modules" ]; then
-    echo "==> Installing frontend dependencies..."
-    (cd "${PROJECT_ROOT}/frontend" && npm install --silent)
-fi
-
-# 4. Start Vite dev server
+# 3. Start Vite dev server
 BACKEND_URL="${VITE_BACKEND_URL:-http://localhost:3000}"
 echo "==> Starting Vite dev server (proxy -> ${BACKEND_URL})..."
 cd "${PROJECT_ROOT}/frontend"
 VITE_BACKEND_URL="${BACKEND_URL}" npx vite --host &
 PIDS+=($!)
 cd "${PROJECT_ROOT}"
 
-# 5. Watch for backend changes
+# 4. Watch for backend changes
 if command -v fswatch &>/dev/null; then
     echo "==> Watching backend changes (fswatch)..."
 else
@@ -142,6 +223,14 @@ fi
 watch_backend &
 PIDS+=($!)
 
+# 5. Optionally watch frontend changes (off by default — Vite at :5173 is the
+#    fast path; this only matters if you want :3000 to stay current too).
+if [ "$WATCH_FRONTEND" = true ]; then
+    echo "==> Watching frontend changes (--watch-frontend) — embedded :3000 will refresh on save"
+    watch_frontend "Frontend change detected" &
+    PIDS+=($!)
+fi
+
 echo ""
 echo "================================================"
 echo "  Frontend (Vite):   http://localhost:5173"

frontend/public/icons/providers/liquid-ai.svg

@@ -0,0 +1,141 @@
+import { describe, expect, it } from 'vitest'
+import {
+  combinedGoodput,
+  DEFAULT_SLO,
+  fractionLe,
+  recomputeGoodputPct,
+  SLO,
+} from '@/lib/slo'
+import type { HistogramBucket } from '@/types/metrics'
+
+const OVERFLOW = Number.MAX_VALUE
+
+function buckets(...pairs: [number, number][]): HistogramBucket[] {
+  return pairs.map(([le, cum]) => ({ le_seconds: le, cumulative_count: cum }))
+}
+
+const APPROX = 1e-9
+
+describe('DEFAULT_SLO mirrors backend constants', () => {
+  it('matches the Rust thresholds (TTFT 500ms, ITL 50ms, E2E 5000ms)', () => {
+    expect(DEFAULT_SLO).toEqual({ ttftMs: 500, itlMs: 50, e2eMs: 5000 })
+  })
+
+  it('keeps SLO as a backwards-compatible alias of DEFAULT_SLO', () => {
+    expect(SLO).toBe(DEFAULT_SLO)
+  })
+})
+
+// The fractionLe tests below are direct ports of the Rust fraction_le tests
+// in src/engines/histogram.rs. Keeping the assertions identical guarantees
+// the TS port stays in lockstep with the backend computation.
+describe('fractionLe (TS port of Rust fraction_le)', () => {
+  it('returns null for empty buckets', () => {
+    expect(fractionLe([], 0.5)).toBeNull()
+    expect(fractionLe(null, 0.5)).toBeNull()
+  })
+
+  it('returns null when total count is zero', () => {
+    const b = buckets([0.1, 0], [1.0, 0], [OVERFLOW, 0])
+    expect(fractionLe(b, 0.5)).toBeNull()
+  })
+
+  it('matches at the bucket boundary', () => {
+    // 50 obs at <= 0.1, 100 total. Threshold == 0.1 → 50/100 = 0.5.
+    const b = buckets([0.1, 50], [1.0, 100])
+    expect(fractionLe(b, 0.1)).toBeCloseTo(0.5, 9)
+  })
+
+  it('linearly interpolates within a bucket', () => {
+    // 25 by 0.1s, 100 by 1.0s. Threshold 0.4 sits inside bucket 1.
+    // bucket has 75 obs over (0.1, 1.0]; (0.4 - 0.1) / 0.9 = 1/3
+    // → cum = 25 + (1/3)*75 = 50 → 50/100 = 0.5.
+    const b = buckets([0.1, 25], [1.0, 100])
+    expect(fractionLe(b, 0.4)).toBeCloseTo(0.5, 9)
+  })
+
+  it('linearly interpolates from zero when threshold is below the first bucket', () => {
+    // 100 obs in [0, 0.1]. Threshold 0.05 → assume uniform within bucket
+    // → 50/100 = 0.5.
+    const b = buckets([0.1, 100], [1.0, 100])
+    expect(fractionLe(b, 0.05)).toBeCloseTo(0.5, 9)
+  })
+
+  it('caps at the last finite cumulative count above finite bounds', () => {
+    // 99 in (0, 1.0], 1 in overflow. Threshold 5.0 → 99/100 = 0.99.
+    const b = buckets([1.0, 99], [OVERFLOW, 100])
+    expect(fractionLe(b, 5.0)).toBeCloseTo(0.99, 9)
+  })
+
+  it('saturates at 1 when threshold exceeds all buckets and no overflow exists', () => {
+    const b = buckets([0.1, 10], [1.0, 100])
+    expect(fractionLe(b, 5.0)).toBeCloseTo(1.0, 9)
+  })
+
+  it('returns 0 when first bucket le is zero or negative', () => {
+    const b = buckets([0.0, 0], [1.0, 100])
+    expect(fractionLe(b, -1.0)).toBeCloseTo(0.0, 9)
+  })
+
+  it('returns null for NaN threshold', () => {
+    const b = buckets([0.1, 50], [1.0, 100])
+    expect(fractionLe(b, Number.NaN)).toBeNull()
+  })
+})
+
+describe('recomputeGoodputPct', () => {
+  it('returns the percentage 0–100 from millisecond thresholds', () => {
+    // 50/100 obs at <= 100ms → 50% goodput at TTFT ≤ 100ms.
+    const b = buckets([0.1, 50], [1.0, 100])
+    const got = recomputeGoodputPct(b, 100)
+    expect(got).not.toBeNull()
+    expect(got!).toBeCloseTo(50, 9)
+  })
+
+  it('returns null when buckets are missing', () => {
+    expect(recomputeGoodputPct(null, 500)).toBeNull()
+    expect(recomputeGoodputPct([], 500)).toBeNull()
+  })
+
+  it('agrees with the backend default goodput within rounding', () => {
+    // Backend with TTFT_SLO_MS=500: vllm test data
+    //   50 at <=0.05, 80 at <=0.1, 95 at <=0.5, 99 at <=1.0, 100 at +Inf.
+    // fractionLe at threshold 0.5s → exactly 95/100 = 0.95 → 95.0%.
+    const b = buckets(
+      [0.05, 50],
+      [0.1, 80],
+      [0.5, 95],
+      [1.0, 99],
+      [OVERFLOW, 100],
+    )
+    const got = recomputeGoodputPct(b, 500)!
+    expect(Math.abs(got - 95)).toBeLessThan(APPROX)
+  })
+
+  it('drops sharply when the user tightens the threshold', () => {
+    const b = buckets(
+      [0.05, 50],
+      [0.1, 80],
+      [0.5, 95],
+      [1.0, 99],
+      [OVERFLOW, 100],
+    )
+    // At a 50ms threshold only the [0, 0.05] bucket fully qualifies → 50%.
+    expect(recomputeGoodputPct(b, 50)!).toBeCloseTo(50, 9)
+  })
+})
+
+describe('combinedGoodput', () => {
+  it('returns the minimum of the present values', () => {
+    expect(combinedGoodput(99, 95, 88)).toBe(88)
+  })
+
+  it('ignores nulls and non-finite values', () => {
+    expect(combinedGoodput(99, null, 88)).toBe(88)
+    expect(combinedGoodput(99, Number.NaN, 50)).toBe(50)
+  })
+
+  it('returns null when all inputs are null', () => {
+    expect(combinedGoodput(null, null, null)).toBeNull()
+  })
+})