perf(android): replace flutter_gemma with llamadart — 150 MB → 42 MB APK (#3)

* perf(android): reduce APK size with ABI splitting, R8, and lib exclusions

- Split release APK per ABI (arm64 + x64 only), dropping universal fat build
- Enable R8 minification and resource shrinking for release builds
- Exclude Vulkan debug validation layer from release packaging
- Exclude unused MediaPipe image generation libs from release packaging

* refactor(models): add HNSW vector index and app-owned RetrievalResult

Add @HnswIndex(dimensions: 384) to BiomarkerResult.embedding so
ObjectBox can run native nearest-neighbor search — eliminating the
separate koshika_vectors.db SQLite store that flutter_gemma required.

Introduce RetrievalResult as an app-owned type so the RAG pipeline
(ChatContextBuilder, CitationExtractor) no longer imports from
flutter_gemma. Add LlmModelConfig and LlmModelRegistry with 4 curated
public GGUF models (SmolLM2 360M, Qwen3 0.6B default, Llama 3.2 1B,
Gemma 3 1B) and a custom-URL escape hatch.

Regenerate ObjectBox bindings to pick up the HNSW annotation.

* feat(services): add model downloader with progress and resume support

Add ModelDownloader to handle GGUF file downloads from arbitrary URLs
(no HuggingFace token needed — all curated models are public).

Supports progress callbacks for UI, cancellation via a flag checked
between chunks, and download resume via HTTP Range headers using a
.part temp file. Static helpers expose the models directory path so
services and the splash migration can share one download location.

* refactor(ai): replace flutter_gemma services with llamadart

Swap the entire inference layer from flutter_gemma (MediaPipe/LiteRT,
~133 MB native libs) to llamadart (llama.cpp via Dart Native Assets,
~5.3 MB compact CPU build).

LlmService replaces GemmaService: model-agnostic, takes an
LlmModelConfig, uses ChatML prompt format compatible with Qwen3,
SmolLM2, and Llama-3. No HuggingFace token needed — all GGUF models
are public.

LlmEmbeddingService replaces EmbeddingService: bge-small-en-v1.5 at
384 dimensions (down from 768). Embeddings are now async (Future) not
sync, so all callers in VectorStoreService are updated with await.

VectorStoreService is fully rewritten to use ObjectBox HNSW queries
instead of a separate SQLite vector DB. No koshika_vectors.db.

* refactor(app): wire llamadart services into shell, add lite/full entry points

Introduce kAiEnabled global flag (set by entry point) to gate all AI
features: Chat tab visibility, AI Models section in Settings, and
vector indexing after PDF import.

Add main_full.dart (aiEnabled: true) and main_lite.dart (aiEnabled:
false) as separate Gradle flavor entry points. SplashScreen initialises
LlmService and LlmEmbeddingService only when kAiEnabled, and runs a
one-time migration to re-embed any stale 768-dim vectors from the old
EmbeddingGemma model.

Settings AI section is fully redesigned: users can pick from 4 curated
GGUF models or paste a custom URL, with inline download/load controls
and progress. No HuggingFace token prompt — all models are public.

* feat(android): add lite/full build flavors and llamadart backend config

Add two Gradle product flavors sharing the appType dimension:
- full: includes all llama.cpp native libs (~5.3 MB, arm64)
- lite: strips libllama.so, libggml*.so, libmtmd.so, libllamadart.so

Uses androidComponents.onVariants for per-variant jniLibs exclusion —
flavor-level packaging.jniLibs.excludes applies globally in Gradle and
would strip AI libs from both variants.

Configure llamadart's Native Assets hook in pubspec.yaml to use
cpu_profile: compact (1 baseline CPU variant instead of 7) and
backends: [cpu] (no Vulkan), reducing native lib overhead from 99 MB
to 5.3 MB. Final APK sizes: full arm64 = 41.8 MB, lite = 35.7 MB
(down from 150 MB on the pre-Phase-2 branch).

CI builds both flavors and uploads all four APKs (arm64/x64 × lite/full)
to the GitHub release.

* fix(ai): harden model switching, downloads, and migration

- cancel and await in-flight model downloads before switching configs\n- ignore stale download callbacks so old transfers cannot mutate the new model state\n- dispose LlamaEngine instances on unload and error paths in both chat and embedding services\n- reject non-success HTTP responses before caching model files and normalize forced-cancel errors\n- share GGUF URL validation and filename derivation between settings and model config creation\n- move embedding migration into a reusable post-load flow used by splash and settings\n- keep flutter analyze and both lite/full release APK builds passing

Author: priyavratuniyal

.github/workflows/release.yml

@@ -34,8 +34,10 @@ jobs:
       - name: Generate ObjectBox bindings
         run: dart run build_runner build --delete-conflicting-outputs
 
-      - name: Build release APK
-        run: flutter build apk --release --split-per-abi --target-platform android-arm64,android-x64
+      - name: Build release APKs
+        run: |
+          flutter build apk --flavor lite -t lib/main_lite.dart --release --split-per-abi --target-platform android-arm64,android-x64
+          flutter build apk --flavor full -t lib/main_full.dart --release --split-per-abi --target-platform android-arm64,android-x64
 
       - name: Rename APKs with version
         run: |

android/app/build.gradle.kts

@@ -20,10 +20,7 @@ android {
     }
 
     defaultConfig {
-        // TODO: Specify your own unique Application ID (https://developer.android.com/studio/build/application-id.html).
         applicationId = "dev.koshika.koshika"
-        // You can update the following values to match your application needs.
-        // For more information, see: https://flutter.dev/to/review-gradle-config.
         minSdk = flutter.minSdkVersion
         targetSdk = flutter.targetSdkVersion
         versionCode = flutter.versionCode
@@ -32,8 +29,6 @@ android {
 
     buildTypes {
         release {
-            // TODO: Add your own signing config for the release build.
-            // Signing with the debug keys for now, so `flutter run --release` works.
             signingConfig = signingConfigs.getByName("debug")
             isMinifyEnabled = true
             isShrinkResources = true
@@ -48,6 +43,32 @@ android {
             excludes += "**/libimagegenerator_gpu.so"
         }
     }
+
+    flavorDimensions += "appType"
+    productFlavors {
+        create("lite") {
+            dimension = "appType"
+            applicationIdSuffix = ".lite"
+            versionNameSuffix = "-lite"
+        }
+        create("full") {
+            dimension = "appType"
+        }
+    }
+}
+
+// Strip llama.cpp native libs from the lite flavor APK.
+// This runs after packaging to remove AI inference libraries,
+// keeping the lite APK small (~15MB smaller).
+androidComponents {
+    onVariants(selector().withFlavor("appType" to "lite")) { variant ->
+        variant.packaging.jniLibs.excludes.addAll(listOf(
+            "**/libllama.so",
+            "**/libggml*.so",
+            "**/libmtmd.so",
+            "**/libllamadart.so",
+        ))
+    }
 }
 
 flutter {

android/app/proguard-rules.pro

@@ -1,7 +1,3 @@
-# MediaPipe optional proto classes (not needed at runtime)
--dontwarn com.google.mediapipe.proto.**
--dontwarn com.google.mediapipe.framework.GraphProfiler
-
 # ML Kit optional language model classes (only Latin script is used)
 -dontwarn com.google.mlkit.vision.text.chinese.**
 -dontwarn com.google.mlkit.vision.text.devanagari.**

lib/constants/ai_prompts.dart

@@ -3,12 +3,11 @@
 /// Keeping prompts here (rather than inlined in service files) makes them
 /// easy to review, iterate, and A/B test without touching service logic.
 abstract final class AiPrompts {
-  /// System prompt for the Gemma chat model.
+  /// System prompt injected before every user query.
   ///
-  /// Injected as a [Message.systemInfo] turn before every user query.
   /// Lab data context is added separately to the user turn so that
   /// small (1B-param) models cannot ignore it.
-  static const String gemmaSystemPrompt = '''
+  static const String systemPrompt = '''
 You are Koshika AI, a helpful on-device health assistant built into the Koshika app. You help users understand their lab report results.
 
 CRITICAL RULES:

lib/main.dart

@@ -1,7 +1,6 @@
 import 'dart:ui';
 
 import 'package:flutter/material.dart';
-import 'package:flutter_gemma/flutter_gemma.dart';
 
 import 'screens/chat_screen.dart';
 import 'screens/dashboard_screen.dart';
@@ -11,25 +10,67 @@ import 'screens/settings_screen.dart';
 import 'screens/splash_screen.dart';
 import 'services/objectbox_store.dart';
 import 'services/biomarker_dictionary.dart';
-import 'services/embedding_service.dart';
-import 'services/gemma_service.dart';
+import 'services/llm_embedding_service.dart';
+import 'services/llm_service.dart';
 import 'services/vector_store_service.dart';
 import 'theme/app_colors.dart';
 import 'theme/koshika_design_system.dart';
 
 /// Global references — initialized in SplashScreen before navigation.
 late ObjectBoxStore objectbox;
 late BiomarkerDictionary biomarkerDictionary;
-late GemmaService gemmaService;
-late EmbeddingService embeddingService;
+late LlmService llmService;
+late LlmEmbeddingService embeddingService;
 late VectorStoreService vectorStoreService;
+Future<void>? _embeddingMigrationTask;
 
-Future<void> main() async {
+/// Whether AI features are enabled. Set by the entry point
+/// (main_full.dart vs main_lite.dart).
+bool kAiEnabled = true;
+
+/// Default entry point — full flavor with AI enabled.
+/// For flavor-specific builds, use main_full.dart or main_lite.dart.
+Future<void> main() async => appMain(aiEnabled: true);
+
+/// Shared app bootstrap — called from entry-point files.
+Future<void> appMain({required bool aiEnabled}) async {
   WidgetsFlutterBinding.ensureInitialized();
-  await FlutterGemma.initialize(maxDownloadRetries: 5);
+  kAiEnabled = aiEnabled;
   runApp(const KoshikaApp());
 }
 
+Future<void> migrateEmbeddingsIfNeeded() {
+  if (!kAiEnabled || !embeddingService.isLoaded) {
+    return Future.value();
+  }
+
+  return _embeddingMigrationTask ??= _runEmbeddingMigration().whenComplete(() {
+    _embeddingMigrationTask = null;
+  });
+}
+
+Future<void> _runEmbeddingMigration() async {
+  try {
+    final allResults = objectbox.biomarkerResultBox.getAll();
+    if (allResults.isEmpty) return;
+
+    final needsMigration = allResults.any(
+      (result) =>
+          result.embedding == null ||
+          result.embedding!.isEmpty ||
+          result.embedding!.length != 384,
+    );
+    if (!needsMigration) return;
+
+    debugPrint(
+      'KoshikaApp: migrating ${allResults.length} embeddings to 384-dim',
+    );
+    await vectorStoreService.rebuildIndex(allResults);
+  } catch (e) {
+    debugPrint('Embedding migration failed (non-fatal): $e');
+  }
+}
+
 class KoshikaApp extends StatelessWidget {
   const KoshikaApp({super.key});
 
@@ -123,19 +164,19 @@ class HomeScreen extends StatefulWidget {
 class _HomeScreenState extends State<HomeScreen> {
   int _currentIndex = 0;
 
+  /// Screens available depends on whether AI is enabled.
+  List<Widget> get _screens => kAiEnabled
+      ? const [
+          DashboardScreen(),
+          ReportsScreen(),
+          ChatScreen(),
+          SettingsScreen(),
+        ]
+      : const [DashboardScreen(), ReportsScreen(), SettingsScreen()];
+
   Widget _buildCurrentScreen() {
-    switch (_currentIndex) {
-      case 0:
-        return const DashboardScreen();
-      case 1:
-        return const ReportsScreen();
-      case 2:
-        return const ChatScreen();
-      case 3:
-        return const SettingsScreen();
-      default:
-        return const DashboardScreen();
-    }
+    if (_currentIndex < _screens.length) return _screens[_currentIndex];
+    return const DashboardScreen();
   }
 
   @override
@@ -175,21 +216,23 @@ class _HomeScreenState extends State<HomeScreen> {
                       label: 'Reports',
                       onTap: () => setState(() => _currentIndex = 1),
                     ),
+                    if (kAiEnabled)
+                      _NavItem(
+                        index: 2,
+                        currentIndex: _currentIndex,
+                        icon: Icons.chat_outlined,
+                        activeIcon: Icons.chat,
+                        label: 'Chat',
+                        onTap: () => setState(() => _currentIndex = 2),
+                      ),
                     _NavItem(
-                      index: 2,
-                      currentIndex: _currentIndex,
-                      icon: Icons.chat_outlined,
-                      activeIcon: Icons.chat,
-                      label: 'Chat',
-                      onTap: () => setState(() => _currentIndex = 2),
-                    ),
-                    _NavItem(
-                      index: 3,
+                      index: kAiEnabled ? 3 : 2,
                       currentIndex: _currentIndex,
                       icon: Icons.settings_outlined,
                       activeIcon: Icons.settings,
                       label: 'Settings',
-                      onTap: () => setState(() => _currentIndex = 3),
+                      onTap: () =>
+                          setState(() => _currentIndex = kAiEnabled ? 3 : 2),
                     ),
                   ],
                 ),

lib/main_full.dart

@@ -0,0 +1,4 @@
+import 'main.dart';
+
+/// Entry point for the full flavor — includes all AI features.
+Future<void> main() async => appMain(aiEnabled: true);

lib/main_lite.dart

@@ -0,0 +1,4 @@
+import 'main.dart';
+
+/// Entry point for the lite flavor — no AI, no model downloads.
+Future<void> main() async => appMain(aiEnabled: false);

lib/models/biomarker_result.dart

@@ -61,8 +61,9 @@ class BiomarkerResult {
   @Index()
   DateTime testDate;
 
-  /// Embedding vector for RAG (stored as float list, used with HNSW later)
+  /// Embedding vector for RAG (384-dim from bge-small-en-v1.5).
   @Property(type: PropertyType.floatVector)
+  @HnswIndex(dimensions: 384)
   List<double>? embedding;
 
   BiomarkerResult({