feat(assistant): native on-device LLM backend for iOS and Android (llama.cpp bridge)

[pliablepixels]

Problem

On-device assistant (WebLLM/WebGPU) is unusable on mobile: WKWebView's jetsam memory limit (~2GB) kills the web process while loading Qwen3-4B weights on iOS, and Android webviews have equivalent caps. On-device is gated off on mobile; only Ollama remote works.

Proposal

Native Capacitor bridge running llama.cpp outside the webview memory limit, one shared JS surface (NativeLlm plugin, NativeLlmProvider reusing the WebLLM prompt/parse stack so all prompt/tool optimizations carry over unchanged):

iOS (done)

• LlamaKit local Swift package wrapping the official prebuilt llama.xcframework (SPM binaryTarget, llama.cpp b10087, Metal)
• LlamaPlugin.swift + LlamaEngine.swift: model download/delete, lazy load, non-streaming chat, cancel, coded error rejections
• com.apple.developer.kernel.increased-memory-limit; runtime gate >= 5.5GB physical memory

Android (this phase)

• llama.cpp built via NDK/CMake pinned to the same b10087 tag, arm64-v8a, CPU backend (i8mm); Vulkan later if benchmarks justify
• Kotlin NativeLlmPlugin + JNI engine mirroring the iOS plugin contract exactly (same method names, error codes, downloadProgress event)
• Runtime gate: total RAM >= 5.5GB and not isLowRamDevice
• Target class: Pixel 8 (8GB) and up; CPU decode ~10 tok/s expected, Ollama remains the recommended Android route

Model: Qwen3-4B-Instruct-2507 Q4_K_M GGUF (~2.5GB), downloaded at first use.

Out of scope

• Token streaming UI (existing backends are non-streaming; bridge API leaves room)
• GPU acceleration on Android (Mali Vulkan gains unproven; revisit after llama-bench numbers on device)

Posted by Claude, assisting @pliablepixels.