Skip to content

Commit 465cabf

Browse files
pliablepixelsclaude
andcommitted
docs(assistant): backend order and accuracy ranking in user guide refs #270
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent ca43a40 commit 465cabf

1 file changed

Lines changed: 3 additions & 1 deletion

File tree

docs/user-guide/assistant.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,10 +8,12 @@ Go to **Settings > Ninjii** and turn on **Enable Ninjii**.
88

99
Underneath, **Backend** picks where the model runs:
1010

11+
- **Ollama**: the model runs on an [Ollama](https://ollama.com) server (or anything else speaking the OpenAI-compatible chat API) that you point the app at.
1112
- **On-device**: the model runs inside the app on your computer's GPU, using WebGPU. Nothing leaves your device. **Desktop and web only** (see below).
1213
- **On-device (native)**: on a supported iPhone, iPad, or Android phone, the model runs inside the app itself instead of a browser engine, using the device's GPU (Metal) on iPhone and iPad, or the CPU on Android. Nothing leaves your device. Only on a device with enough memory (see below); Android has no GPU path yet, so replies there are slower than on an iPhone.
1314
- **On-device (Apple Intelligence)**: on an iPhone 15 Pro or newer running iOS 26 with Apple Intelligence turned on, the assistant uses Apple's own on-device system model. There is nothing to download: Apple ships and runs the model as part of iOS, so it uses none of the app's memory for model weights. Nothing leaves your device (see below).
14-
- **Ollama**: the model runs on an [Ollama](https://ollama.com) server (or anything else speaking the OpenAI-compatible chat API) that you point the app at.
15+
16+
The backends differ in how accurate their answers are, and a note under the picker says so: Ollama is the most accurate, the on-device backends (native on a phone, WebGPU on a desktop) come next, and system models such as Apple Intelligence are the least accurate. When the assistant is running on a system model, the chat window keeps a note suggesting the switch to **On-device (native)**.
1517

1618
The WebGPU on-device backend is not offered on phones or tablets: those models need more memory than a mobile browser engine is allowed to use, and answers take minutes on phone hardware. An iPhone or iPad with roughly 6GB of RAM or more gets the native on-device backend instead (see below). Android needs more: only a 12GB-class phone (for example a Pixel 8 Pro or a Galaxy S Ultra) qualifies, because the model runs on the CPU and its working set has to fit alongside everything else the phone is doing. An 8GB Android phone such as the Pixel 8 uses Ollama instead. Below the threshold there is no on-device choice, just a note saying so and the Ollama settings. On a desktop or in a browser, WebGPU on-device is available whenever your GPU supports WebGPU.
1719

0 commit comments

Comments
 (0)