Skip to content

Commit 4e2dfe7

Browse files
authored
fix(litert): correct header/binary ABI skew + real model-load smoke (#8)
* fix(litert): source headers from the lib's commit; smoke now loads+compiles a model The packaged LiteRT archive paired v2.1.5 release headers with a prebuilt libLiteRt from a newer main commit. Between those, upstream added a leading LiteRtEnvironment parameter to LiteRtCreateModelFromFile/FromBuffer, so the shipped headers mis-declared the ABI by one register: model load read the filename from &model (empty -> "Could not open ''", status 500) and the buffer pointer from the size value (-> segfault in strncmp). The binary was fine; the headers were from the wrong commit. - stage.sh: source litert/c/*.h per-mode so headers match the staged lib — prebuilt mode pulls them from the repo tree at $PREBUILT_SHA; build-from-source keeps the v${VER} SDK (matches its v${VER} source clone). - test/smoke.cpp: env-only check couldn't catch this. Now load a .tflite from file AND buffer, then CPU-compile it — the real consumption path. - test/CMakeLists.txt: fetch tiny add.bin model and pass it to the smoke. - .gitignore: ignore stage.sh header-source scratch. Verified: re-staged the macOS-arm64 prebuilt leg with the corrected script and the smoke passes (load file+buffer + compile). anira's InferenceLiteRt suite (GuitarLSTM vs the TFLite reference WAV) passes end-to-end against it. * fix(litert): build from-source legs at PREBUILT_SHA too (coherent ABI across all legs) CI exposed a deeper incoherence: prebuilt legs ship libLiteRt from the main $PREBUILT_SHA (env-leading 3-arg LiteRtCreateModelFrom*), but the from-source legs (macOS x86_64 shared + all static, incl. iOS) built from the v2.1.5 tag — the OLD 2-arg API. So the package shipped two incompatible model-load ABIs depending on platform/kind, and the new load+compile smoke failed to compile on the v2.1.5-sourced legs. A single consumer (anira's LiteRtProcessor) cannot link both. Align the whole matrix on $PREBUILT_SHA: - stage.sh + ios.sh: shallow-fetch the source at $PREBUILT_SHA (fetch-by-sha) instead of cloning the v${VER} tag; headers always from that same commit. - ios.sh: add PREBUILT_SHA (kept in sync with stage.sh). - VERSION stays as the asset-name label only; PREBUILT_SHA pins the actual code. Verified fetch-by-sha works on GitHub and the tree carries the 3-arg signature.
1 parent fd6c3d8 commit 4e2dfe7

6 files changed

Lines changed: 119 additions & 31 deletions

File tree

.gitignore

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,3 +10,9 @@ emsdk-cache-*/
1010

1111
# onnxruntime build
1212
engines/onnxruntime/onnxruntime-src/
13+
14+
# litert stage.sh header-source scratch (fetched per-mode: SDK zip for build, repo tarball for prebuilt)
15+
engines/litert/litert_cc_sdk/
16+
engines/litert/litert_cc_sdk.zip
17+
engines/litert/litert-src.tar.gz
18+
engines/litert/LiteRT-*/

engines/litert/README.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -67,8 +67,8 @@ macOS, GNU `ar` on Linux, `lib.exe`/`llvm-lib` on Windows. Per-leg specifics:
6767

6868
| File | Purpose |
6969
| --------------------- | -------------------------------------------------------------------- |
70-
| `VERSION` | Pinned LiteRT **release** tag (not `main`) |
71-
| `stage.sh` | Prebuilt repackage or Bazel build of `libLiteRt`; shared + static |
70+
| `VERSION` | LiteRT version label for asset names; actual code pinned by `PREBUILT_SHA` in `stage.sh` |
71+
| `stage.sh` | Prebuilt repackage or Bazel build of `libLiteRt` (both @ `PREBUILT_SHA`); shared + static |
7272
| `ios.sh` | Build device + simulator **static** libs from source, merge each, → static xcframework |
7373
| `test/CMakeLists.txt` | CMake smoke (link `libLiteRt`; run via the smoke action/ctest) |
74-
| `test/smoke.cpp` | Link + load: `LiteRtCreateEnvironment` / `…Destroy…` |
74+
| `test/smoke.cpp` | Link + env + load a `.tflite` (file & buffer) + CPU compile |

engines/litert/ios.sh

Lines changed: 13 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
# static from source so iOS matches the rest of the matrix (static is the preferred iOS linkage —
66
# no embedded framework to sign, dead-code-stripped into the app/appex). Same transitive-archive
77
# -merge recipe as the desktop/Android static legs in stage.sh, run once per Apple slice.
8-
# Headers from litert_cc_sdk.zip. Produces dist/<archive>.zip.
8+
# Headers + source both from the pinned PREBUILT_SHA (see stage.sh). Produces dist/<archive>.zip.
99
#
1010
# NOTE: the exact Bazel iOS flags below are best-effort and may need CI iteration on a macOS
1111
# runner (Apple platform transition + static-xcframework platform metadata are the fiddly bits).
@@ -15,12 +15,16 @@ set -euo pipefail
1515
ARCHIVE="${1:?archive name}"
1616
HERE="$(cd "$(dirname "$0")" && pwd)"
1717
VER="$(tr -d '[:space:]' < "$HERE/VERSION")"
18+
# Pinned LiteRT main commit — MUST match stage.sh's PREBUILT_SHA so the iOS static slice ships the
19+
# same model-load ABI (env-leading LiteRtCreateModelFrom*) as every other leg. Keep in sync.
20+
PREBUILT_SHA="89c838788bba9c2ec6bbefd52971daf39d8e2856"
1821

19-
# ---- Headers: SDK litert/c/*.h + synthesized CPU-only build_config.h (same set as other legs) --
20-
sdk="$HERE/litert_cc_sdk"
22+
# ---- Headers: litert/c/*.h from $PREBUILT_SHA + synthesized CPU-only build_config.h (same ref as
23+
# the from-source build below, so headers and lib never skew — see stage.sh for the why) ---------
24+
sdk="$HERE/LiteRT-${PREBUILT_SHA}"
2125
if [ ! -d "$sdk/litert/c" ]; then
22-
curl -fsSL "https://github.com/google-ai-edge/LiteRT/releases/download/v${VER}/litert_cc_sdk.zip" -o "$HERE/litert_cc_sdk.zip"
23-
( cd "$HERE" && cmake -E tar xf litert_cc_sdk.zip )
26+
curl -fsSL "https://github.com/google-ai-edge/LiteRT/archive/${PREBUILT_SHA}.tar.gz" -o "$HERE/litert-src.tar.gz"
27+
( cd "$HERE" && cmake -E tar xzf litert-src.tar.gz )
2428
fi
2529
hdr="$HERE/ios_include"; rm -rf "$hdr"; mkdir -p "$hdr/litert/build_common"
2630
( cd "$sdk" && find litert/c -name '*.h' | while IFS= read -r h; do mkdir -p "$hdr/$(dirname "$h")"; cp "$h" "$hdr/$h"; done )
@@ -41,7 +45,10 @@ EOF
4145
# ---- Source + host CC toolchain (configure.py), same setup as stage.sh -------------------------
4246
SRC="$HERE/litert-src"
4347
if [ ! -d "$SRC/.git" ]; then
44-
git clone --depth 1 --branch "v${VER}" https://github.com/google-ai-edge/LiteRT "$SRC"
48+
git init -q "$SRC"
49+
git -C "$SRC" remote add origin https://github.com/google-ai-edge/LiteRT
50+
git -C "$SRC" fetch -q --depth 1 origin "$PREBUILT_SHA"
51+
git -C "$SRC" checkout -q --detach FETCH_HEAD
4552
fi
4653
# configure.py generates the host CC toolchain (else "@@local_config_cc//:toolchain ... cpu").
4754
export PYTHON_BIN_PATH="$(python3 -c 'import sys; print(sys.executable)')"

engines/litert/stage.sh

Lines changed: 22 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,13 @@
11
#!/usr/bin/env bash
22
# Stage LiteRT's NATIVE C API (libLiteRt — LiteRt* symbols) into <staging> as
33
# include/litert/c + lib/libLiteRt.{so,dylib,dll}. Distinct from the `tflite` engine (legacy
4-
# TfLite* C API). Headers (both modes) come from the litert_cc_sdk.zip release + a synthesized
5-
# CPU-only build_config.h. Two lib modes:
4+
# TfLite* C API). Headers AND lib both come from the same pinned commit ($PREBUILT_SHA) + a
5+
# synthesized CPU-only build_config.h, so the C ABI never skews across legs. Two lib modes:
66
# source=prebuilt — fetch the official prebuilt libLiteRt from google-ai-edge/LiteRT's
77
# litert/prebuilt/<platform>/ (Git-LFS, via the media endpoint), pinned to a
88
# main commit (upstream ships these mobile/desktop prebuilts UNVERSIONED).
9-
# source=build — build from source via Bazel, CPU-only. Used where no prebuilt exists:
9+
# source=build — build from source via Bazel at that SAME commit, CPU-only. Used where no
10+
# prebuilt exists:
1011
# macOS x86_64 (shared), and ALL static legs (upstream ships no static lib —
1112
# we build the C API impl and merge its transitive .a closure into libLiteRt.a).
1213
#
@@ -21,12 +22,17 @@ VER="$(tr -d '[:space:]' < "$HERE/VERSION")"
2122
# upstream). Bump deliberately and re-verify the LiteRt* symbols after.
2223
PREBUILT_SHA="89c838788bba9c2ec6bbefd52971daf39d8e2856"
2324

24-
# ---- Headers (both modes): SDK litert/c/*.h + synthesized CPU-only build_config.h --------------
25+
# ---- Headers: from the pinned LiteRT commit ($PREBUILT_SHA) — the SAME ref BOTH modes use for the
26+
# lib (prebuilt binary AND from-source Bazel build, below), so headers and binary never skew. A
27+
# leading LiteRtEnvironment param was added to LiteRtCreateModelFrom{File,Buffer} after the v2.1.5
28+
# tag; sourcing headers (or the lib) from any other ref mis-shifts the args by a register — model
29+
# load then reads the path from &model (empty -> "Could not open ''", status 500) and the buffer
30+
# ptr from the size (-> segfault). Upstream ships prebuilts only off main, so main is the ref.
2531
mkdir -p "$ST/include/litert/build_common" "$ST/lib"
26-
sdk="$HERE/litert_cc_sdk"
32+
sdk="$HERE/LiteRT-${PREBUILT_SHA}"
2733
if [ ! -d "$sdk/litert/c" ]; then
28-
curl -fsSL "https://github.com/google-ai-edge/LiteRT/releases/download/v${VER}/litert_cc_sdk.zip" -o "$HERE/litert_cc_sdk.zip"
29-
( cd "$HERE" && cmake -E tar xf litert_cc_sdk.zip ) # -> $HERE/litert_cc_sdk/
34+
curl -fsSL "https://github.com/google-ai-edge/LiteRT/archive/${PREBUILT_SHA}.tar.gz" -o "$HERE/litert-src.tar.gz"
35+
( cd "$HERE" && cmake -E tar xzf litert-src.tar.gz ) # -> $HERE/LiteRT-${PREBUILT_SHA}/
3036
fi
3137
( cd "$sdk" && find litert/c -name '*.h' | while IFS= read -r h; do
3238
mkdir -p "$ST/include/$(dirname "$h")"; cp "$h" "$ST/include/$h"; done )
@@ -70,10 +76,17 @@ if [ "$SOURCE" = "prebuilt" ]; then
7076
exit 0
7177
fi
7278

73-
# ---- source=build: Bazel (CPU-only) — for platforms with no prebuilt (macOS x86_64) ------------
79+
# ---- source=build: Bazel (CPU-only) — for platforms with no prebuilt (macOS x86_64) + all static.
80+
# Build from the SAME commit as the prebuilts ($PREBUILT_SHA), NOT the v${VER} tag: upstream ships
81+
# prebuilts only off main, and the model-load ABI (env-leading LiteRtCreateModelFrom*) must be
82+
# identical on every leg or a single consumer can't link both. Shallow-fetch the exact SHA
83+
# (GitHub allows fetch-by-sha via allowAnySHA1InWant).
7484
SRC="$HERE/litert-src"
7585
if [ ! -d "$SRC/.git" ]; then
76-
git clone --depth 1 --branch "v${VER}" https://github.com/google-ai-edge/LiteRT "$SRC"
86+
git init -q "$SRC"
87+
git -C "$SRC" remote add origin https://github.com/google-ai-edge/LiteRT
88+
git -C "$SRC" fetch -q --depth 1 origin "$PREBUILT_SHA"
89+
git -C "$SRC" checkout -q --detach FETCH_HEAD
7790
fi
7891

7992
# ---- source=build, kind=static, Android: build inside LiteRT's ml-build container --------------

engines/litert/test/CMakeLists.txt

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -40,5 +40,18 @@ endif()
4040
# Shared lib loads from the staging lib dir at runtime (rpath on unix; Windows copies the DLL).
4141
set_target_properties(smoke PROPERTIES BUILD_WITH_INSTALL_RPATH TRUE INSTALL_RPATH "${LITERT_LIBDIR}")
4242

43+
# A tiny .tflite model for the load+compile smoke (LiteRT loads the .tflite flatbuffer like
44+
# TFLite). Reuse TFLite's pinned add.bin test model — the LiteRt archive ships no model.
45+
set(MODEL "${CMAKE_BINARY_DIR}/add.bin")
46+
if(NOT EXISTS "${MODEL}")
47+
file(DOWNLOAD
48+
"https://raw.githubusercontent.com/tensorflow/tensorflow/v2.17.0/tensorflow/lite/testdata/add.bin"
49+
"${MODEL}" STATUS _dl)
50+
list(GET _dl 0 _code)
51+
if(NOT _code EQUAL 0)
52+
message(FATAL_ERROR "add.bin download failed: ${_dl}")
53+
endif()
54+
endif()
55+
4356
enable_testing()
44-
add_test(NAME smoke COMMAND smoke)
57+
add_test(NAME smoke COMMAND smoke "${MODEL}")

engines/litert/test/smoke.cpp

Lines changed: 61 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,22 +1,71 @@
1-
// LiteRT native C API smoke: link libLiteRt and exercise the C API entry point —
2-
// create + destroy a LiteRtEnvironment (no model needed). The link proves the packaged
3-
// lib is symbol-complete (LiteRt* symbols resolve); the run proves it loads + initializes.
4-
// (A full forward pass — LiteRtCreateCompiledModel + LiteRtRunCompiledModel on a .tflite —
5-
// comes once the per-platform Bazel build is green.)
1+
// LiteRT native C API smoke: link libLiteRt and exercise the real consumption path —
2+
// create an environment, LOAD a .tflite model (both from file and from a memory buffer),
3+
// and COMPILE it for the CPU accelerator. The link proves the packaged lib is symbol-complete;
4+
// the run proves the packaged headers match the binary's ABI and that model loading works.
65
//
7-
// Usage: smoke (exit 0 = pass)
6+
// This guards against a header/binary version skew: LiteRtCreateModelFrom{File,Buffer} gained a
7+
// leading LiteRtEnvironment parameter upstream, so headers from a different commit than the lib
8+
// silently mis-shift the arguments — model load then opens '' (status 500) or segfaults. Env
9+
// create/destroy alone does NOT catch that; loading + compiling a real model does.
10+
//
11+
// (Numerical correctness of the forward pass is validated end-to-end in anira's test suite.)
12+
//
13+
// Usage: smoke <path/to/model.tflite> (exit 0 = pass)
814
#include <cstdio>
15+
#include <fstream>
16+
#include <vector>
17+
918
#include "litert/c/litert_common.h"
1019
#include "litert/c/litert_environment.h"
20+
#include "litert/c/litert_model.h"
21+
#include "litert/c/litert_options.h"
22+
#include "litert/c/litert_compiled_model.h"
23+
24+
static int fail(const char* what, LiteRtStatus s) {
25+
std::printf("FAIL: %s (status=%d)\n", what, static_cast<int>(s));
26+
return 1;
27+
}
28+
29+
int main(int argc, char** argv) {
30+
if (argc < 2) { std::printf("usage: smoke <model.tflite>\n"); return 1; }
31+
const char* path = argv[1];
1132

12-
int main() {
1333
LiteRtEnvironment env = nullptr;
1434
LiteRtStatus s = LiteRtCreateEnvironment(/*num_options=*/0, /*options=*/nullptr, &env);
15-
if (s != kLiteRtStatusOk || env == nullptr) {
16-
std::printf("FAIL: LiteRtCreateEnvironment status=%d\n", static_cast<int>(s));
17-
return 1;
18-
}
35+
if (s != kLiteRtStatusOk || env == nullptr) return fail("LiteRtCreateEnvironment", s);
36+
37+
// 1) Load from file.
38+
LiteRtModel model_file = nullptr;
39+
s = LiteRtCreateModelFromFile(env, path, &model_file);
40+
if (s != kLiteRtStatusOk || model_file == nullptr) return fail("LiteRtCreateModelFromFile", s);
41+
42+
// 2) Load the same bytes from a memory buffer (must outlive the model).
43+
std::ifstream f(path, std::ios::binary | std::ios::ate);
44+
if (!f) { std::printf("FAIL: cannot read %s\n", path); return 1; }
45+
const std::streamsize n = f.tellg();
46+
f.seekg(0);
47+
std::vector<char> buf(static_cast<size_t>(n));
48+
f.read(buf.data(), n);
49+
LiteRtModel model_buf = nullptr;
50+
s = LiteRtCreateModelFromBuffer(env, buf.data(), static_cast<size_t>(n), &model_buf);
51+
if (s != kLiteRtStatusOk || model_buf == nullptr) return fail("LiteRtCreateModelFromBuffer", s);
52+
53+
// 3) Compile (CPU) — exercises the rest of the model-consumption ABI.
54+
LiteRtOptions opts = nullptr;
55+
s = LiteRtCreateOptions(&opts);
56+
if (s != kLiteRtStatusOk) return fail("LiteRtCreateOptions", s);
57+
s = LiteRtSetOptionsHardwareAccelerators(opts, kLiteRtHwAcceleratorCpu);
58+
if (s != kLiteRtStatusOk) return fail("LiteRtSetOptionsHardwareAccelerators", s);
59+
LiteRtCompiledModel compiled = nullptr;
60+
s = LiteRtCreateCompiledModel(env, model_file, opts, &compiled);
61+
if (s != kLiteRtStatusOk || compiled == nullptr) return fail("LiteRtCreateCompiledModel", s);
62+
63+
LiteRtDestroyCompiledModel(compiled);
64+
LiteRtDestroyOptions(opts);
65+
LiteRtDestroyModel(model_buf);
66+
LiteRtDestroyModel(model_file);
1967
LiteRtDestroyEnvironment(env);
20-
std::printf("PASS: LiteRt environment create/destroy OK\n");
68+
69+
std::printf("PASS: LiteRt env + model load (file+buffer) + CPU compile OK\n");
2170
return 0;
2271
}

0 commit comments

Comments
 (0)