Updated Bert Qa Android to use Task Library and Interpreter

lite/examples/bert_qa/android/.gitignore

@@ -0,0 +1,3 @@
+/app/src/main/assets/
+/lib_interpreter/src/main/assets/
+/lib_task_api/src/main/assets/

lite/examples/bert_qa/android/EXPLORE_THE_CODE.md

@@ -0,0 +1,169 @@
+# TensorFlow Lite BERT QA Android example
+
+This document walks through the code of a simple Android mobile application that
+demonstrates
+[BERT Question and Answer](https://www.tensorflow.org/lite/examples/bert_qa/overview).
+
+## Explore the code
+
+The app is written entirely in Java and uses the TensorFlow Lite
+[Java library](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/java)
+for performing BERT Question and Answer.
+
+We're now going to walk through the most important parts of the sample code.
+
+### Get the question and the context of the question
+
+This mobile application gets the question and the context of the question using the functions defined in the
+file
+[`QaActivity.java`](https://github.com/tensorflow/examples/blob/master/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ui/QaActivity.java).
+
+
+### Answerer
+
+This BERT QA Android reference app demonstrates two implementation
+solutions,
+[`lib_task_api`](https://github.com/SunitRoy2703/examples/tree/bertQa-android-task-lib/lite/examples/bert_qa/android/lib_task_api)
+that leverages the out-of-box API from the
+[TensorFlow Lite Task Library](https://www.tensorflow.org/lite/inference_with_metadata/task_library/bert_question_answerer),
+and
+[`lib_interpreter`](https://github.com/SunitRoy2703/examples/tree/bertQa-android-task-lib/lite/examples/bert_qa/android/lib_interpreter)
+that creates the custom inference pipleline using the
+[TensorFlow Lite Interpreter Java API](https://www.tensorflow.org/lite/guide/inference#load_and_run_a_model_in_java).
+
+Both solutions implement the file `QaClient.java` (see
+[the one in lib_task_api](https://github.com/SunitRoy2703/examples/blob/bertQa-android-task-lib/lite/examples/bert_qa/android/lib_task_api/src/main/java/org/tensorflow/lite/examples/bertqa/ml/QaClient.java)
+and
+[the one in lib_interpreter](https://github.com/SunitRoy2703/examples/blob/bertQa-android-task-lib/lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/ml/QaClient.java)
+that contains most of the complex logic for processing the text input and
+running inference.
+
+#### Using the TensorFlow Lite Task Library
+
+Inference can be done using just a few lines of code with the
+[`BertQuestionAnswerer`](https://www.tensorflow.org/lite/inference_with_metadata/task_library/bert_question_answerer)
+in the TensorFlow Lite Task Library.
+
+##### Load model and create BertQuestionAnswerer
+
+`BertQuestionAnswerer` expects a model populated with the
+[model metadata](https://www.tensorflow.org/lite/convert/metadata) and the label
+file. See the
+[model compatibility requirements](https://www.tensorflow.org/lite/inference_with_metadata/task_library/bert_question_answerer#model_compatibility_requirements)
+for more details.
+
+
+```java
+/**
+ * Load TFLite model and create BertQuestionAnswerer instance.
+ */
+ public void loadModel() {
+     try {
+         answerer = BertQuestionAnswerer.createFromFile(context, MODEL_PATH);
+     } catch (IOException e) {
+         Log.e(TAG, e.getMessage());
+     }
+ }
+```
+
+`BertQuestionAnswerer` currently does not support configuring delegates and
+multithread, but those are on our roadmap. Please stay tuned!
+
+##### Run inference
+
+The following code runs inference using `BertQuestionAnswerer` and predicts the possible answers
+
+```java
+ /**
+  * Run inference and predict the possible answers.
+  */
+      List<QaAnswer> apiResult = answerer.answer(contextOfTheQuestion, questionToAsk);
+
+```
+
+The output of `BertQuestionAnswerer` is a list of [`QaAnswer`](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/java/src/java/org/tensorflow/lite/task/text/qa/QaAnswer.java) instance, where
+each `QaAnswer` element is a single head classification result. All the
+demo models are single head models, therefore, `results` only contains one
+`QaAnswer` object.
+
+To match the implementation of
+[`lib_interpreter`](https://github.com/SunitRoy2703/examples/tree/bertQa-android-task-lib/lite/examples/bert_qa/android/lib_interpreter),
+`results` is converted into List<[`Answer`](https://github.com/SunitRoy2703/examples/blob/bertQa-android-task-lib/lite/examples/bert_qa/android/lib_task_api/src/main/java/org/tensorflow/lite/examples/bertqa/ml/Answer.java)>.
+
+#### Using the TensorFlow Lite Interpreter
+
+##### Load model and create interpreter
+
+To perform inference, we need to load a model file and instantiate an
+`Interpreter`. This happens in the `loadModel` method of the `QaClient` class. Information about number of threads is used to configure the `Interpreter` via the
+`Interpreter.Options` instance passed into its constructor.
+
+```java
+Interpreter.Options opt = new Interpreter.Options();
+      opt.setNumThreads(NUM_LITE_THREADS);
+      tflite = new Interpreter(buffer, opt);
+...
+```
+
+##### Pre-process query & content
+
+Next in the `predict` method of the `QaClient` class, we take the input of query & content,
+convert it to a `Feature` format for efficient processing and pre-process
+it. The steps are shown in the public 'FeatureConverter.convert()' method:
+
+```java
+
+public Feature convert(String query, String context) {
+    List<String> queryTokens = tokenizer.tokenize(query);
+    if (queryTokens.size() > maxQueryLen) {
+      queryTokens = queryTokens.subList(0, maxQueryLen);
+    }
+
+    List<String> origTokens = Arrays.asList(context.trim().split("\\s+"));
+    List<Integer> tokenToOrigIndex = new ArrayList<>();
+    List<String> allDocTokens = new ArrayList<>();
+    for (int i = 0; i < origTokens.size(); i++) {
+      String token = origTokens.get(i);
+      List<String> subTokens = tokenizer.tokenize(token);
+      for (String subToken : subTokens) {
+        tokenToOrigIndex.add(i);
+        allDocTokens.add(subToken);
+      }
+    }
+
+```
+
+##### Run inference
+
+Inference is performed using the following in `QaClient` class:
+
+```java
+tflite.runForMultipleInputsOutputs(inputs, output);
+```
+
+### Display results
+
+The QaClient is invoked and inference results are displayed by the
+`presentAnswer()` function in
+[`QaActivity.java`](https://github.com/SunitRoy2703/examples/blob/bertQa-android-task-lib/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/QaActivity.java).
+
+```java
+private void presentAnswer(Answer answer) {
+        // Highlight answer.
+        Spannable spanText = new SpannableString(content);
+        int offset = content.indexOf(answer.text, 0);
+        if (offset >= 0) {
+            spanText.setSpan(
+                    new BackgroundColorSpan(getColor(R.color.tfe_qa_color_highlight)),
+                    offset,
+                    offset + answer.text.length(),
+                    Spannable.SPAN_EXCLUSIVE_EXCLUSIVE);
+        }
+        contentTextView.setText(spanText);
+
+        // Use TTS to speak out the answer.
+        if (textToSpeech != null) {
+            textToSpeech.speak(answer.text, TextToSpeech.QUEUE_FLUSH, null, answer.text);
+        }
+    }
+```

lite/examples/bert_qa/android/README.md

@@ -1,13 +1,29 @@
 # TensorFlow Lite BERT QA Android Example Application
 
+
+<img src="https://user-images.githubusercontent.com/67560900/122643946-37d0d380-d130-11eb-8e7c-f467b90cb0dd.mp4" width="400px" alt="Video">
+
 ## Overview
 
-This is an end-to-end example of BERT Question & Answer application built with
-TensorFlow 2.0, and tested on SQuAD dataset. The demo app provides 48 passages
-from the dataset for users to choose from, and gives 5 most possible answers
-corresponding to the input passage and query.
+This is an end-to-end example of [BERT] Question & Answer application built with
+TensorFlow 2.0, and tested on [SQuAD] dataset version 1.1. The demo app provides
+48 passages from the dataset for users to choose from, and gives 5 most possible
+answers corresponding to the input passage and query.
+
+These instructions walk you through running the demo on an Android device. For an explanation of the source, see
+[TensorFlow Lite BERT QA Android example](EXPLORE_THE_CODE.md).
+
+### Model used
+
+[BERT], or Bidirectional Encoder Representations from Transformers, is a method
+of pre-training language representations which obtains state-of-the-art results
+on a wide array of Natural Language Processing tasks.
+
+This app uses [MobileBERT], a compressed version of [BERT] that runs 4x faster and
+has 4x smaller model size.
+
+For more information, refer to the [BERT github page][BERT].
 
-These instructions walk you through running the demo on an Android device.
 
 ## Build the demo using Android Studio
 
@@ -19,10 +35,10 @@ These instructions walk you through running the demo on an Android device.
 
 *   Android Studio 3.2 or later.
     - Gradle 4.6 or higher.
-    - SDK Build Tools 28.0.3 or higher.
+    - SDK Build Tools 29.0.2 or higher.
 
 *   You need an Android device or Android emulator and Android development
-    environment with minimum API 15.
+    environment with minimum API 21.
 
 ### Building
 
@@ -41,14 +57,46 @@ These instructions walk you through running the demo on an Android device.
 ### Running
 
 *   You need to have an Android device plugged in with developer options enabled
-    at this point. See [here](https://developer.android.com/studio/run/device)
+    at this point. See [here](https://developer.android.com/studio/run/device "Download Link")
     for more details on setting up developer devices.
 
 *   If you already have Android emulator installed in Android Studio, select a
     virtual device with minimum API 15.
 
 *   Click `Run` to run the demo app on your Android device.
 
+#### Switch between inference solutions (Task library vs TFLite Interpreter)
+
+This BERT QA Android reference app demonstrates two implementation
+solutions:
+
+(1)
+[`lib_task_api`](https://github.com/SunitRoy2703/examples/tree/bertQa-android-task-lib/lite/examples/bert_qa/android/lib_task_api)
+that leverages the out-of-box API from the
+[TensorFlow Lite Task Library](https://www.tensorflow.org/lite/inference_with_metadata/task_library/bert_question_answerer);
+
+(2)
+[`lib_interpreter`](https://github.com/SunitRoy2703/examples/tree/bertQa-android-task-lib/lite/examples/bert_qa/android/lib_interpreter)
+that creates the custom inference pipleline using the
+[TensorFlow Lite Interpreter Java API](https://www.tensorflow.org/lite/guide/inference#load_and_run_a_model_in_java).
+
+The [`build.gradle`](app/build.gradle) inside `app` folder shows how to change
+`flavorDimensions "tfliteInference"` to switch between the two solutions.
+
+Inside **Android Studio**, you can change the build variant to whichever one you
+want to build and run—just go to `Build > Select Build Variant` and select one
+from the drop-down menu. See
+[configure product flavors in Android Studio](https://developer.android.com/studio/build/build-variants#product-flavors)
+for more details.
+
+For gradle CLI, running `./gradlew build` can create APKs for both solutions
+under `app/build/outputs/apk`.
+
+*Note: If you simply want the out-of-box API to run the app, we recommend
+`lib_task_api`for inference. If you want to customize your own models and
+control the detail of inputs and outputs, it might be easier to adapt your model
+inputs and outputs by using `lib_interpreter`.*
+
 ## Build the demo using gradle (command line)
 
 ### Building and Installing
@@ -66,3 +114,13 @@ cd lite/examples/bert_qa/android   # Folder for Android app.
 ```
 adb install app/build/outputs/apk/debug/app-debug.apk
 ```
+
+## Assets folder
+
+_Do not delete the assets folder content_. If you explicitly deleted the files,
+choose `Build -> Rebuild` to re-download the deleted model files into the assets
+folder.
+
+[BERT]: https://github.com/google-research/bert "Bert"
+[SQuAD]: https://rajpurkar.github.io/SQuAD-explorer/ "SQuAD"
+[MobileBERT]:https://tfhub.dev/tensorflow/tfjs-model/mobilebert/1 "MobileBERT"

lite/examples/bert_qa/android/app/build.gradle

@@ -2,10 +2,10 @@ apply plugin: 'com.android.application'
 
 android {
     compileSdkVersion 29
-    buildToolsVersion "29.0.0"
+    buildToolsVersion "29.0.2"
     defaultConfig {
         applicationId "org.tensorflow.lite.examples.bertapp"
-        minSdkVersion 26
+        minSdkVersion 21
         targetSdkVersion 29
         versionCode 1
         versionName "1.0"
@@ -34,32 +34,33 @@ android {
     lintOptions {
       abortOnError false
     }
-}
 
-// App assets folder:
-project.ext.ASSET_DIR = projectDir.toString() + '/src/main/assets/'
+    flavorDimensions "tfliteInference"
+    productFlavors {
+        // The TFLite inference is built using the TFLite Java interpreter.
+        interpreter {
+            dimension "tfliteInference"
+        }
+        // Default: The TFLite inference is built using the TFLite Task library (high-level API).
+        taskApi {
+            getIsDefault().set(true)
+            dimension "tfliteInference"
+        }
+    }
+}
 
-// Download TF Lite model.
-apply from: 'download.gradle'
 
 dependencies {
-    implementation fileTree(dir: 'libs', include: ['*.jar'])
+    interpreterImplementation project(":lib_interpreter")
+    taskApiImplementation project(":lib_task_api")
 
     implementation 'androidx.appcompat:appcompat:1.1.0'
     implementation 'androidx.constraintlayout:constraintlayout:1.1.3'
     implementation 'androidx.coordinatorlayout:coordinatorlayout:1.1.0'
     implementation 'androidx.recyclerview:recyclerview:1.1.0'
     implementation 'com.google.android.material:material:1.0.0'
-
     implementation 'com.google.code.gson:gson:2.8.5'
-    implementation 'com.google.guava:guava:28.1-android'
-    implementation ('org.tensorflow:tensorflow-lite:2.4.0')
-    implementation 'org.tensorflow:tensorflow-lite-metadata:0.1.0'
 
-    testImplementation 'junit:junit:4.12'
-    testImplementation 'androidx.test:core:1.2.0'
-    testImplementation 'com.google.truth:truth:1.0'
-    testImplementation 'org.robolectric:robolectric:4.3.1'
     androidTestImplementation 'androidx.test:runner:1.2.0'
     androidTestImplementation 'androidx.test.espresso:espresso-core:3.2.0'
 }

lite/examples/bert_qa/android/app/src/main/AndroidManifest.xml

@@ -28,16 +28,16 @@
       tools:ignore="GoogleAppIndexingWarning">
 
     <activity
-        android:name=".ui.QaActivity"
+        android:name=".QaActivity"
         android:exported="false"
-        android:parentActivityName=".ui.DatasetListActivity">
+        android:parentActivityName=".DatasetListActivity">
       <meta-data
           android:name="android.support.PARENT_ACTIVITY"
-          android:value=".ui.DatasetListActivity" />
+          android:value=".DatasetListActivity" />
     </activity>
 
     <activity
-        android:name=".ui.DatasetListActivity"
+        android:name=".DatasetListActivity"
         android:exported="true"
         android:theme="@style/AppTheme.QA.NoTitleActivity" >
       <intent-filter>