Benchmark runners for evaluating LLM inference engines on edge devices, from host-native CLIs to native iOS and Android apps. The CLIs and the Android app share a common Rust core (the iOS app is native Swift). Remote runs can submit results to the pipette management server; local runs stay local.
The clients are the benchmark runners you build and run.
Command-line benchmark runners that run on the host machine. They share one
operating workflow: init → register → install runtime → run/sync, plus an
optional planner claim loop (pipette worker) that pulls jobs from the
management server.
| CLI | Inference backends | Build |
|---|---|---|
pipette |
llama.cpp, MLX, OpenVINO, vLLM, SGLang |
cargo build --release -p pipette-cli |
pipette-plan |
orchestrates pipette across remote devices |
cargo build --release -p pipette-plan |
Mobile clients run benchmarks on-device with engines compiled into the app. The Android app shares the Rust workspace; the iOS app is native Swift. Linked engines are platform-specific. See docs/README.md. Each app has its own build pipeline:
| App | Location | Build instructions |
|---|---|---|
| iOS | ios/Pipette/ |
docs/pipette-ios/build.md |
| Android | android/Pipette/ |
docs/pipette-android/build.md |
Not a client itself. crates/ holds the shared Rust workspace for host CLIs,
runtime libraries, artifact stores, plan types, management client, execution
helpers, and the Android native core.
cargo build --release -p pipette-cli # produces target/release/pipette
export PATH="$PWD/target/release:$PATH" # or copy it somewhere on PATHThen follow the usage guide (init → register → install a runtime → run → sync). Mobile app builds are linked in the table above.
Full index and documentation convention: docs/README.md. Architecture lives in docs/architecture.md; benchmark measurement rules in docs/methodology/README.md.
This is an Apache-2.0 licensed open source project. See CONTRIBUTING.md, CODE_OF_CONDUCT.md, and SECURITY.md.
Copyright 2026 Liquid AI, Inc.
Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
The full license text is in LICENSE, with attribution in NOTICE and THIRD_PARTY_NOTICES.md.