- Status: Accepted
- Date: 2026-06-02
- Deciders: Maintainer + Claude Code
- Context tags: architecture, ai, performance, privacy
WorldScript Studio is offline-first and privacy-first: AI must be able to run with no cloud call. The local-inference stack grew across several sprints (WebLLM, ONNX Runtime Web, Transformers.js, WGSL compute shaders, an adaptive engine, a device profiler, benchmark/telemetry). Audits flagged it as "deep and high-maintenance." This ADR records the intended layering and the degradation contract so the depth is legible and each layer's responsibility is fixed.
- WebLLM (
services/localAiFacade.tsβpackages/ai-core) β WebGPU, largest local models (Llama 3.2 1B/3B, Phi-3.5 Mini, Gemma 2 2B). Tab-leader election via BroadcastChannel prevents multi-tab GPU contention. - ONNX Runtime Web (
packages/ai-core/onnxRuntimeEngine.ts) β WebGPU or WASM, smaller models. - Transformers.js (
@huggingface/transformersv3,workers/inference.worker.ts) β off-main-thread pipelines (text-generation, feature-extraction, sentiment, summarization). - Heuristic / energy-based engines β always-available last resort (e.g.
WebRtcVadEngine,EnergyThresholdWakeWordEngine) so a feature never hard-fails when no model is loadable.
Selection is not hardcoded. services/ai/adaptiveAiEngine.ts + localAiDeviceProfiler.ts
(30 s TTL hardware cache) choose a backend per task; recommendBackend() falls to onnx-wasm when
WebGPU is absent. Cloud providers (Gemini/OpenAI/β¦) are a separate axis, always gated by
assertCloudAiAllowed (services/ai/aiPolicy.ts) β never reached when the user is local-only.
- Pipeline residency:
services/ai/pipelineLruCache.tsβ one LRU per worker, bounded (MAX_PIPELINE_CACHE = 8), dispose-on-evict (releases GPU/WASM memory), in-flight load dedup. Replaces the previously-duplicated cache loops in the two inference workers. - Result cache:
services/ai/aiInferenceCacheService.tsβ two-layer (in-mem LRU 64 + IDB 256, 7-day TTL, skips prompts > 512 chars). - Transient resilience:
services/ai/aiRetry.tsβ exponential backoff + full jitter, with a serverRetry-Aftertaking precedence (clamped against hostile values). - Orchestration:
@domain/worker-bus(priority queue, circuit breakers, dead-letter queue);workerBusManagerroutes toworkers/v2/inference.worker.ts. - Observability:
benchmarkService+telemetryService(local DuckDB / localStorage, no cloud), gated byenableDuckDbAnalytics.
A layer that cannot run must report unavailable (isAvailable() === false) so the next layer is
selected β it must never return a plausible-but-fake result. Example: SileroVadEngine is wired but
ONNX integration is deferred, so it reports false and the energy-based VAD is used. The ProForge
pipeline mirrors this with isFallback: true + 0 scores so the supervisor can trigger retries.
- Positive: Each layer has a single responsibility; capability gating means the app runs on a no-GPU low-end device down to the heuristic layer; memory is bounded by the shared LRU; the degradation contract prevents silent quality cliffs.
- Negative: Many moving parts. Mitigated by: shared infra (cache/retry/bus) instead of per-layer re-implementation, and this ADR as the map.
- Maintenance rule: new heavy optional chunks must be added to both
manualChunksandglobIgnoresinvite.config.ts(never service-worker-precached).
.cursor/rules/100-ai-providers-and-storage.mdcfor current AI/storage routing;vite.config.tsfor code-splitting configuration.AUDIT.mdEdge-AI Perfection Cycle (2026-05-31), Perf Hardening (2026-06-02)- [[0001-state-management-boundaries]]