Skip to content
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions .github/workflows/test.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
name: 'Test Pull Request'

on: pull_request

jobs:
commitlint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Install commitlint
run: npm install -D @commitlint/cli @commitlint/config-conventional
- name: Validate pull request title
if: github.event_name == 'pull_request'
run: echo "${{ github.event.pull_request.title }}" | npx commitlint --verbose
verify:
runs-on: macos-26
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: actions/setup-node@v6
with:
node-version: 22
cache: 'npm'
- run: npm ci
- run: |
npm run lint
npm run verify
16 changes: 11 additions & 5 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,13 @@
## Project Overview
Capacitor Local LLM is a Capacitor plugin that wraps on-device LLM functionality on iOS and Android.
- iOS uses on-device Foundation Models introduced in iOS 26 (a very new API — avoid assumptions about its behavior; prefer checking Apple docs)
- iOS uses Foundation Models (Apple Intelligence) for text LLM (iOS 26+) and Image Playground for image generation (iOS 18.4+). Foundation Models is a new API — avoid assumptions about its behavior; prefer checking Apple docs.
- Android uses on-device Gemini Nano via the ML Kit packages
- Web is unsupported — plugin methods should throw a "not implemented" error on Web

## Platform Requirements
- iOS: minimum **18.4**. Image generation works on iOS 18.4+. Text LLM (Foundation Models) requires iOS 26+.
- Android: minimum **API 29** (Android 10). Do not lower these — they reflect hard requirements of the underlying native APIs.

## Tech Stack
- This package is a Capacitor plugin, as well as an SPM and CocoaPods package
- Languages: TypeScript, Swift 6, Kotlin
Expand Down Expand Up @@ -45,9 +49,10 @@ This plugin follows standard Capacitor conventions:
2. `src/web.ts` — Web stub (throw `unimplemented()`)
3. iOS Swift plugin class
4. Android Kotlin plugin class
- iOS methods that require iOS 26+ must be wrapped in `#available(iOS 26.0, *)` guards and throw `LocalLLMError.unsupported` in the `else` branch. Do not assume a feature is available just because the deployment target allows the code to compile.

## Running the Example App
Note: on-device LLMs cannot run in simulators or emulators — a physical device is required.
Note: Android emulators are not supported — Gemini Nano requires a physical Android device. iOS simulators are supported as long as the host Mac supports Apple Intelligence and has it enabled.
```bash
cd example-app
npm install
Expand All @@ -60,14 +65,15 @@ ionic cap sync
- All TypeScript interfaces, types, and functions intended for public API consumption should be documented using JSDoc comments, including:
- The version the feature was introduced (`@since`)
- A one-line usage example
- Platform availability for platform-exclusive features (e.g., `@platform ios`)
- Platform availability for platform-exclusive features — note this in prose within the JSDoc description, as Capacitor's docgen does not recognize a `@platform` tag
- New functionality should be added for all platforms unless unavailable due to platform limitations. Platform-exclusive features must be noted in documentation.

## Testing
- There are no automated unit tests — verification is done manually via the example app on physical devices.
- When making changes, test on both iOS and Android physical devices before considering work complete.

## Things to Avoid
- Do not attempt to run or test using simulators or emulators — on-device LLMs require physical hardware.
- Do not attempt to run or test Android using emulators — Gemini Nano requires physical hardware. iOS simulators work with caveats (see Running the Example App above).
- Avoid adding new dependencies where possible. If a dependency is needed, flag it for review before adding.
- Do not make assumptions about Foundation Models API behavior — it is new in iOS 26 and documentation may be limited.
- Do not make assumptions about Foundation Models API behavior — it is a new API and documentation may be limited.
- Do not lower the minimum platform versions (iOS 18.4 / Android API 29) — they are set to match the minimum requirements of the underlying native APIs.
2 changes: 1 addition & 1 deletion CapacitorLocalLlm.podspec
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Pod::Spec.new do |s|
s.author = package['author']
s.source = { :git => package['repository']['url'], :tag => s.version.to_s }
s.source_files = 'ios/Sources/**/*.{swift,h,m,c,cc,mm,cpp}'
s.ios.deployment_target = '26.0'
s.ios.deployment_target = '18.4'
s.dependency 'Capacitor'
s.swift_version = '5.1'
end
136 changes: 134 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# @capacitor/local-llm

Capacitor Local LLM plugin
Run large language models entirely on-device using Apple Intelligence (Foundation Models) on iOS and Gemini Nano on Android. No network requests, no API keys, no data leaving the device.

> **Note:** On-device LLMs require physical hardware. Android emulators are not supported. iOS simulators are supported so long as the host device is capable of running Apple Intelligence and has it enabled.

## Install

Expand All @@ -9,6 +11,104 @@ npm install @capacitor/local-llm
npx cap sync
```

## Platform Requirements

| Platform | Minimum OS | Notes |
|----------|------------|-------|
| iOS | **18.4** | Image generation requires iOS 18.4+. Text LLM (Foundation Models / Apple Intelligence) requires iOS 26+. |
| Android | **10 (API 29)** | Gemini Nano via ML Kit requires a device that supports on-device AI (e.g. Pixel 6+). |

Comment thread
theproducer marked this conversation as resolved.
## iOS Setup

No additional configuration is required. Foundation Models and Image Playground are system frameworks available automatically on supported devices with Apple Intelligence enabled.

Call [`systemAvailability()`](#systemavailability) at runtime to check whether the model is ready before sending prompts.

On iOS 18, `systemAvailability()` returns `'unavailable'` for the text LLM. If `prompt()` or `warmup()` are called anyway, the promise will reject with an error. Image generation via `generateImage()` is fully functional on iOS 18.4+.

## Android Setup

Gemini Nano is distributed via Google Play Services and must be downloaded to the device before use. The model is not bundled with your app.

### Check availability and download

Call [`systemAvailability()`](#systemavailability) to inspect the current state. If the status is `downloadable`, trigger the download with [`download()`](#download) and poll `systemAvailability()` until the status becomes `available`.
Comment thread
theproducer marked this conversation as resolved.

```typescript
import { LocalLLM } from '@capacitor/local-llm';

const { status } = await LocalLLM.systemAvailability();

if (status === 'downloadable') {
await LocalLLM.download();
// Poll systemAvailability() until status === 'available'
}
```

## Usage

### Basic prompt

```typescript
import { LocalLLM } from '@capacitor/local-llm';

const { text } = await LocalLLM.prompt({
prompt: 'Summarize the theory of relativity in one paragraph.',
});

console.log(text);
```

### Multi-turn conversation

Use a `sessionId` to maintain context across multiple prompts.

```typescript
import { LocalLLM } from '@capacitor/local-llm';

const sessionId = 'my-chat-session';

await LocalLLM.prompt({
sessionId,
instructions: 'You are a helpful assistant.',
prompt: 'What is the capital of France?',
});

const { text } = await LocalLLM.prompt({
sessionId,
prompt: 'What is the population of that city?',
});

// Clean up when done
await LocalLLM.endSession({ sessionId });
```

### Reduce first-response latency with warmup

```typescript
import { LocalLLM } from '@capacitor/local-llm';

// Pre-initialize the model before the user starts typing
await LocalLLM.warmup({
sessionId: 'my-session',
promptPrefix: 'You are a customer support agent for Acme Corp.',
});
```

### Image generation (iOS only)

```typescript
import { LocalLLM } from '@capacitor/local-llm';

const { pngBase64Images } = await LocalLLM.generateImage({
prompt: 'A serene mountain lake at sunrise, photorealistic',
count: 2,
});

// Use directly in an <img> tag
const src = `data:image/png;base64,${pngBase64Images[0]}`;
```

## API

<docgen-index>
Expand All @@ -18,6 +118,7 @@ npx cap sync
* [`prompt(...)`](#prompt)
* [`endSession(...)`](#endsession)
* [`generateImage(...)`](#generateimage)
* [`warmup(...)`](#warmup)
* [Interfaces](#interfaces)
* [Type Aliases](#type-aliases)

Expand Down Expand Up @@ -127,6 +228,27 @@ as base64-encoded PNG strings in an array.
--------------------


### warmup(...)

```typescript
warmup(options: WarmupOptions) => Promise<void>
```

Warms up the on-device LLM for faster initial responses.

Use this method to pre-initialize the LLM with a prompt prefix, reducing latency
for the first actual prompt. This is useful when you know in advance the type of
prompts you'll be sending.

| Param | Type | Description |
| ------------- | ------------------------------------------------------- | ------------------------------------------------ |
| **`options`** | <code><a href="#warmupoptions">WarmupOptions</a></code> | - The warmup options including the prompt prefix |

**Since:** 1.0.0

--------------------


### Interfaces


Expand Down Expand Up @@ -167,7 +289,7 @@ Configuration options for LLM inference behavior.
| Prop | Type | Description | Since |
| ------------------------- | ------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----- |
| **`temperature`** | <code>number</code> | Controls randomness in the model's output. Higher values (e.g., 0.8) make output more random, while lower values (e.g., 0.2) make it more focused and deterministic. | 1.0.0 |
| **`maximumOutputTokens`** | <code>number</code> | The maximum number of tokens to generate in the response. * | 1.0.0 |
| **`maximumOutputTokens`** | <code>number</code> | The maximum number of tokens to generate in the response. On Android, this must be between 1 and 256. | 1.0.0 |


#### EndSessionOptions
Expand Down Expand Up @@ -199,6 +321,16 @@ Options for generating an image from a text prompt.
| **`count`** | <code>number</code> | The number of image variations to generate. Defaults to 1 if not specified. | <code>1</code> | 1.0.0 |


#### WarmupOptions

Options for warming up the on-device LLM.

| Prop | Type | Description | Since |
| ------------------ | ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----- |
| **`sessionId`** | <code>string</code> | The session identifier for the warmup. This identifier will be associated with the warmed-up session, allowing you to use the same session for subsequent prompts. | 1.0.0 |
| **`promptPrefix`** | <code>string</code> | The prompt prefix to use for warming up the LLM. This text will be used to pre-initialize the model, reducing latency for subsequent prompts with similar prefixes. | 1.0.0 |


### Type Aliases


Expand Down
2 changes: 1 addition & 1 deletion android/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ android {
namespace "io.ionic.localllm.plugin"
compileSdk project.hasProperty('compileSdkVersion') ? rootProject.ext.compileSdkVersion : 35
defaultConfig {
minSdkVersion 26
minSdkVersion 29
targetSdkVersion project.hasProperty('targetSdkVersion') ? rootProject.ext.targetSdkVersion : 35
versionCode 1
versionName "1.0"
Expand Down
1 change: 1 addition & 0 deletions commitlint.config.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
export default { extends: ['@commitlint/config-conventional'] };
2 changes: 1 addition & 1 deletion ios/Sources/LocalLLMPlugin/LocalLLM.swift
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ public class LocalLLM {

func warmup(sessionId: String, promptPrefix: String?) throws {
if #available(iOS 26.0, *) {
let session = getOrCreateSession(sessionId: sessionId)
let session = getOrCreateSession(sessionId: sessionId, instructions: promptPrefix)

session.prewarm(promptPrefix: .init(promptPrefix))
} else {
Expand Down
Loading
Loading