From fe4c932d6a5b53c99843fae7c13aca806d851a90 Mon Sep 17 00:00:00 2001 From: Sunit Roy <2703iamsry@gmail.com> Date: Sun, 20 Jun 2021 02:49:07 +0530 Subject: [PATCH 1/7] Updated Bert Qa Android to use Task Library --- lite/examples/bert_qa/android/.gitignore | 1 + lite/examples/bert_qa/android/README.md | 30 +- .../examples/bert_qa/android/app/build.gradle | 12 +- .../lite/examples/bertqa/UnitTest.java | 44 +++ .../bertqa/data/LoadDatasetClient.java | 93 ++++++ .../lite/examples/bertqa/ml/Feature.java | 41 --- .../examples/bertqa/ml/FeatureConverter.java | 105 ------ .../examples/bertqa/ml/LoadDatasetClient.java | 105 ------ .../lite/examples/bertqa/ml/QaAnswer.java | 48 --- .../lite/examples/bertqa/ml/QaClient.java | 314 ------------------ .../lite/examples/bertqa/tflite/QaClient.java | 49 +++ .../bertqa/tokenization/BasicTokenizer.java | 104 ------ .../bertqa/tokenization/CharChecker.java | 58 ---- .../bertqa/tokenization/FullTokenizer.java | 52 --- .../tokenization/WordpieceTokenizer.java | 94 ------ .../bertqa/ui/DatasetListActivity.java | 2 +- .../lite/examples/bertqa/ui/QaActivity.java | 7 +- 17 files changed, 220 insertions(+), 939 deletions(-) create mode 100644 lite/examples/bert_qa/android/.gitignore create mode 100644 lite/examples/bert_qa/android/app/src/androidTest/java/org/tensorflow/lite/examples/bertqa/UnitTest.java create mode 100644 lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/data/LoadDatasetClient.java delete mode 100644 lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ml/Feature.java delete mode 100644 lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ml/FeatureConverter.java delete mode 100644 lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ml/LoadDatasetClient.java delete mode 100644 lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ml/QaAnswer.java delete mode 100644 lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ml/QaClient.java create mode 100644 lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/tflite/QaClient.java delete mode 100644 lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/tokenization/BasicTokenizer.java delete mode 100644 lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/tokenization/CharChecker.java delete mode 100644 lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/tokenization/FullTokenizer.java delete mode 100644 lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/tokenization/WordpieceTokenizer.java diff --git a/lite/examples/bert_qa/android/.gitignore b/lite/examples/bert_qa/android/.gitignore new file mode 100644 index 00000000000..bbacc76e892 --- /dev/null +++ b/lite/examples/bert_qa/android/.gitignore @@ -0,0 +1 @@ +/app/src/main/assets/ \ No newline at end of file diff --git a/lite/examples/bert_qa/android/README.md b/lite/examples/bert_qa/android/README.md index 271c7024ff5..eae315db913 100644 --- a/lite/examples/bert_qa/android/README.md +++ b/lite/examples/bert_qa/android/README.md @@ -1,14 +1,29 @@ # TensorFlow Lite BERT QA Android Example Application + +Video + ## Overview -This is an end-to-end example of BERT Question & Answer application built with -TensorFlow 2.0, and tested on SQuAD dataset. The demo app provides 48 passages -from the dataset for users to choose from, and gives 5 most possible answers -corresponding to the input passage and query. +This is an end-to-end example of [BERT] Question & Answer application built with +TensorFlow 2.0, and tested on [SQuAD] dataset version 1.1. The demo app provides +48 passages from the dataset for users to choose from, and gives 5 most possible +answers corresponding to the input passage and query. These instructions walk you through running the demo on an Android device. +### Model used + +[BERT], or Bidirectional Encoder Representations from Transformers, is a method +of pre-training language representations which obtains state-of-the-art results +on a wide array of Natural Language Processing tasks. + +This app uses [MobileBERT](https://tfhub.dev/tensorflow/tfjs-model/mobilebert/1 "MobileBERT"), a compressed version of [BERT] that runs 4x faster and +has 4x smaller model size. + +For more information, refer to the [BERT github page][BERT]. + + ## Build the demo using Android Studio ### Prerequisites @@ -19,7 +34,7 @@ These instructions walk you through running the demo on an Android device. * Android Studio 3.2 or later. - Gradle 4.6 or higher. - - SDK Build Tools 28.0.3 or higher. + - SDK Build Tools 29.0.2 or higher. * You need an Android device or Android emulator and Android development environment with minimum API 15. @@ -41,7 +56,7 @@ These instructions walk you through running the demo on an Android device. ### Running * You need to have an Android device plugged in with developer options enabled - at this point. See [here](https://developer.android.com/studio/run/device) + at this point. See [here](https://developer.android.com/studio/run/device "Download Link") for more details on setting up developer devices. * If you already have Android emulator installed in Android Studio, select a @@ -66,3 +81,6 @@ cd lite/examples/bert_qa/android # Folder for Android app. ``` adb install app/build/outputs/apk/debug/app-debug.apk ``` + +[BERT]: https://github.com/google-research/bert "Bert" +[SQuAD]: https://rajpurkar.github.io/SQuAD-explorer/ "SQuAD" \ No newline at end of file diff --git a/lite/examples/bert_qa/android/app/build.gradle b/lite/examples/bert_qa/android/app/build.gradle index 81dda5a0d1e..dce99defa82 100644 --- a/lite/examples/bert_qa/android/app/build.gradle +++ b/lite/examples/bert_qa/android/app/build.gradle @@ -2,7 +2,7 @@ apply plugin: 'com.android.application' android { compileSdkVersion 29 - buildToolsVersion "29.0.0" + buildToolsVersion "29.0.2" defaultConfig { applicationId "org.tensorflow.lite.examples.bertapp" minSdkVersion 26 @@ -52,14 +52,10 @@ dependencies { implementation 'com.google.android.material:material:1.0.0' implementation 'com.google.code.gson:gson:2.8.5' - implementation 'com.google.guava:guava:28.1-android' - implementation ('org.tensorflow:tensorflow-lite:2.4.0') - implementation 'org.tensorflow:tensorflow-lite-metadata:0.1.0' - testImplementation 'junit:junit:4.12' - testImplementation 'androidx.test:core:1.2.0' - testImplementation 'com.google.truth:truth:1.0' - testImplementation 'org.robolectric:robolectric:4.3.1' + //Task Text Library dependency + implementation 'org.tensorflow:tensorflow-lite-task-text:0.2.0' + androidTestImplementation 'androidx.test:runner:1.2.0' androidTestImplementation 'androidx.test.espresso:espresso-core:3.2.0' } diff --git a/lite/examples/bert_qa/android/app/src/androidTest/java/org/tensorflow/lite/examples/bertqa/UnitTest.java b/lite/examples/bert_qa/android/app/src/androidTest/java/org/tensorflow/lite/examples/bertqa/UnitTest.java new file mode 100644 index 00000000000..51b4a009236 --- /dev/null +++ b/lite/examples/bert_qa/android/app/src/androidTest/java/org/tensorflow/lite/examples/bertqa/UnitTest.java @@ -0,0 +1,44 @@ +package org.tensorflow.lite.examples.bertqa; + +import android.content.Context; + +import androidx.test.platform.app.InstrumentationRegistry; +import androidx.test.runner.AndroidJUnit4; + +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.tensorflow.lite.examples.bertqa.tflite.QaClient; +import org.tensorflow.lite.task.text.qa.QaAnswer; + +import static org.junit.Assert.*; + + +/** Tests of {@link QaClient} */ +@RunWith(AndroidJUnit4.class) +public final class UnitTest { + private QaClient client; + private String contextOfTheQuestion = "Nikola Tesla (Serbian Cyrillic: Никола Тесла; 10 July 1856 – 7 January 1943) was a Serbian American inventor, electrical engineer, mechanical engineer, physicist, and futurist best known for his contributions to the design of the modern alternating current (AC) electricity supply system."; + + @Before + public void setUp() { + // Context of the app under test. + Context appContext = InstrumentationRegistry.getInstrumentation().getTargetContext(); + + client = new QaClient(appContext); + client.loadModel(); + } + + @Test + public void loadModelTest() { + assertNotNull(client.answerer); + } + + @Test + public void predictTest() { + + QaAnswer answer = client.predict("In what year was Nikola Tesla born? ", contextOfTheQuestion).get(0); + + assertEquals("1856", answer.text); + } +} \ No newline at end of file diff --git a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/data/LoadDatasetClient.java b/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/data/LoadDatasetClient.java new file mode 100644 index 00000000000..0cd668286d9 --- /dev/null +++ b/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/data/LoadDatasetClient.java @@ -0,0 +1,93 @@ +/* Copyright 2019 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +package org.tensorflow.lite.examples.bertqa.data; + +import android.content.Context; +import android.util.Log; + +import com.google.gson.Gson; +import com.google.gson.stream.JsonReader; + +import java.io.BufferedReader; +import java.io.IOException; +import java.io.InputStream; +import java.io.InputStreamReader; +import java.util.HashMap; +import java.util.List; +import java.util.Map; + +/** + * Interface to load squad dataset. Provide passages for users to choose from & provide questions + * for autoCompleteTextView. + */ +public class LoadDatasetClient { + private static final String TAG = "BertAppDemo"; + private static final String JSON_DIR = "qa.json"; + private static final String DIC_DIR = "vocab.txt"; + private final Context context; + + private String[] contents; + private String[] titles; + private String[][] questions; + + public LoadDatasetClient(Context context) { + this.context = context; + loadJson(); + } + + private void loadJson() { + try { + InputStream is = context.getAssets().open(JSON_DIR); + JsonReader reader = new JsonReader(new InputStreamReader(is)); + HashMap>> map = new Gson().fromJson(reader, HashMap.class); + List> jsonTitles = map.get("titles"); + List> jsonContents = map.get("contents"); + List> jsonQuestions = map.get("questions"); + + titles = listToArray(jsonTitles); + contents = listToArray(jsonContents); + + questions = new String[jsonQuestions.size()][]; + int index = 0; + for (List item : jsonQuestions) { + questions[index++] = item.toArray(new String[item.size()]); + } + } catch (IOException ex) { + Log.e(TAG, ex.toString()); + } + } + + private static String[] listToArray(List> list) { + String[] answer = new String[list.size()]; + int index = 0; + for (List item : list) { + answer[index++] = item.get(0); + } + return answer; + } + + public String[] getTitles() { + return titles; + } + + public String getContent(int index) { + return contents[index]; + } + + public String[] getQuestions(int index) { + return questions[index]; + } + +} diff --git a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ml/Feature.java b/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ml/Feature.java deleted file mode 100644 index 2232eb5d3b1..00000000000 --- a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ml/Feature.java +++ /dev/null @@ -1,41 +0,0 @@ -/* Copyright 2019 The TensorFlow Authors. All Rights Reserved. - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -==============================================================================*/ -package org.tensorflow.lite.examples.bertqa.ml; - -import com.google.common.primitives.Ints; -import java.util.List; -import java.util.Map; - -/** Feature to be fed into the Bert model. */ -public final class Feature { - public final int[] inputIds; - public final int[] inputMask; - public final int[] segmentIds; - public final List origTokens; - public final Map tokenToOrigMap; - - public Feature( - List inputIds, - List inputMask, - List segmentIds, - List origTokens, - Map tokenToOrigMap) { - this.inputIds = Ints.toArray(inputIds); - this.inputMask = Ints.toArray(inputMask); - this.segmentIds = Ints.toArray(segmentIds); - this.origTokens = origTokens; - this.tokenToOrigMap = tokenToOrigMap; - } -} diff --git a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ml/FeatureConverter.java b/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ml/FeatureConverter.java deleted file mode 100644 index 32fe4240358..00000000000 --- a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ml/FeatureConverter.java +++ /dev/null @@ -1,105 +0,0 @@ -/* Copyright 2019 The TensorFlow Authors. All Rights Reserved. - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -==============================================================================*/ -package org.tensorflow.lite.examples.bertqa.ml; - -import java.util.ArrayList; -import java.util.Arrays; -import java.util.Collections; -import java.util.HashMap; -import java.util.List; -import java.util.Map; -import org.tensorflow.lite.examples.bertqa.tokenization.FullTokenizer; - -/** Convert String to features that can be fed into BERT model. */ -public final class FeatureConverter { - private final FullTokenizer tokenizer; - private final int maxQueryLen; - private final int maxSeqLen; - - public FeatureConverter( - Map inputDic, boolean doLowerCase, int maxQueryLen, int maxSeqLen) { - this.tokenizer = new FullTokenizer(inputDic, doLowerCase); - this.maxQueryLen = maxQueryLen; - this.maxSeqLen = maxSeqLen; - } - - public Feature convert(String query, String context) { - List queryTokens = tokenizer.tokenize(query); - if (queryTokens.size() > maxQueryLen) { - queryTokens = queryTokens.subList(0, maxQueryLen); - } - - List origTokens = Arrays.asList(context.trim().split("\\s+")); - List tokenToOrigIndex = new ArrayList<>(); - List allDocTokens = new ArrayList<>(); - for (int i = 0; i < origTokens.size(); i++) { - String token = origTokens.get(i); - List subTokens = tokenizer.tokenize(token); - for (String subToken : subTokens) { - tokenToOrigIndex.add(i); - allDocTokens.add(subToken); - } - } - - // -3 accounts for [CLS], [SEP] and [SEP]. - int maxContextLen = maxSeqLen - queryTokens.size() - 3; - if (allDocTokens.size() > maxContextLen) { - allDocTokens = allDocTokens.subList(0, maxContextLen); - } - - List tokens = new ArrayList<>(); - List segmentIds = new ArrayList<>(); - - // Map token index to original index (in feature.origTokens). - Map tokenToOrigMap = new HashMap<>(); - - // Start of generating the features. - tokens.add("[CLS]"); - segmentIds.add(0); - - // For query input. - for (String queryToken : queryTokens) { - tokens.add(queryToken); - segmentIds.add(0); - } - - // For Separation. - tokens.add("[SEP]"); - segmentIds.add(0); - - // For Text Input. - for (int i = 0; i < allDocTokens.size(); i++) { - String docToken = allDocTokens.get(i); - tokens.add(docToken); - segmentIds.add(1); - tokenToOrigMap.put(tokens.size(), tokenToOrigIndex.get(i)); - } - - // For ending mark. - tokens.add("[SEP]"); - segmentIds.add(1); - - List inputIds = tokenizer.convertTokensToIds(tokens); - List inputMask = new ArrayList<>(Collections.nCopies(inputIds.size(), 1)); - - while (inputIds.size() < maxSeqLen) { - inputIds.add(0); - inputMask.add(0); - segmentIds.add(0); - } - - return new Feature(inputIds, inputMask, segmentIds, origTokens, tokenToOrigMap); - } -} diff --git a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ml/LoadDatasetClient.java b/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ml/LoadDatasetClient.java deleted file mode 100644 index 585052c8966..00000000000 --- a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ml/LoadDatasetClient.java +++ /dev/null @@ -1,105 +0,0 @@ -/* Copyright 2019 The TensorFlow Authors. All Rights Reserved. - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -==============================================================================*/ -package org.tensorflow.lite.examples.bertqa.ml; - -import android.content.Context; -import android.util.Log; -import com.google.gson.Gson; -import com.google.gson.stream.JsonReader; -import java.io.BufferedReader; -import java.io.IOException; -import java.io.InputStream; -import java.io.InputStreamReader; -import java.util.HashMap; -import java.util.List; -import java.util.Map; - -/** - * Interface to load squad dataset. Provide passages for users to choose from & provide questions - * for autoCompleteTextView. - */ -public class LoadDatasetClient { - private static final String TAG = "BertAppDemo"; - private static final String JSON_DIR = "qa.json"; - private static final String DIC_DIR = "vocab.txt"; - private final Context context; - - private String[] contents; - private String[] titles; - private String[][] questions; - - public LoadDatasetClient(Context context) { - this.context = context; - loadJson(); - } - - private void loadJson() { - try { - InputStream is = context.getAssets().open(JSON_DIR); - JsonReader reader = new JsonReader(new InputStreamReader(is)); - HashMap>> map = new Gson().fromJson(reader, HashMap.class); - List> jsonTitles = map.get("titles"); - List> jsonContents = map.get("contents"); - List> jsonQuestions = map.get("questions"); - - titles = listToArray(jsonTitles); - contents = listToArray(jsonContents); - - questions = new String[jsonQuestions.size()][]; - int index = 0; - for (List item : jsonQuestions) { - questions[index++] = item.toArray(new String[item.size()]); - } - } catch (IOException ex) { - Log.e(TAG, ex.toString()); - } - } - - private static String[] listToArray(List> list) { - String[] answer = new String[list.size()]; - int index = 0; - for (List item : list) { - answer[index++] = item.get(0); - } - return answer; - } - - public String[] getTitles() { - return titles; - } - - public String getContent(int index) { - return contents[index]; - } - - public String[] getQuestions(int index) { - return questions[index]; - } - - public Map loadDictionary() { - Map dic = new HashMap<>(); - try (InputStream ins = context.getAssets().open(DIC_DIR); - BufferedReader reader = new BufferedReader(new InputStreamReader(ins))) { - int index = 0; - while (reader.ready()) { - String key = reader.readLine(); - dic.put(key, index++); - } - } catch (IOException ex) { - Log.e(TAG, ex.getMessage()); - } - return dic; - } -} diff --git a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ml/QaAnswer.java b/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ml/QaAnswer.java deleted file mode 100644 index 077527e0fde..00000000000 --- a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ml/QaAnswer.java +++ /dev/null @@ -1,48 +0,0 @@ -/* Copyright 2019 The TensorFlow Authors. All Rights Reserved. - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -==============================================================================*/ -package org.tensorflow.lite.examples.bertqa.ml; - -/** QA Answer class. */ -public class QaAnswer { - public Pos pos; - public String text; - - public QaAnswer(String text, Pos pos) { - this.text = text; - this.pos = pos; - } - - public QaAnswer(String text, int start, int end, float logit) { - this(text, new Pos(start, end, logit)); - } - - /** Position and related information from the model. */ - public static class Pos implements Comparable { - public int start; - public int end; - public float logit; - - public Pos(int start, int end, float logit) { - this.start = start; - this.end = end; - this.logit = logit; - } - - @Override - public int compareTo(Pos other) { - return Float.compare(other.logit, this.logit); - } - } -} diff --git a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ml/QaClient.java b/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ml/QaClient.java deleted file mode 100644 index a0d951446fe..00000000000 --- a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ml/QaClient.java +++ /dev/null @@ -1,314 +0,0 @@ -/* Copyright 2019 The TensorFlow Authors. All Rights Reserved. - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -==============================================================================*/ -package org.tensorflow.lite.examples.bertqa.ml; - -import static com.google.common.base.Verify.verify; - -import android.content.Context; -import android.content.res.AssetFileDescriptor; -import android.content.res.AssetManager; -import android.util.Log; -import androidx.annotation.WorkerThread; -import com.google.common.base.Joiner; -import java.io.BufferedReader; -import java.io.FileInputStream; -import java.io.IOException; -import java.io.InputStream; -import java.io.InputStreamReader; -import java.nio.ByteBuffer; -import java.nio.MappedByteBuffer; -import java.nio.channels.FileChannel; -import java.util.ArrayList; -import java.util.Collections; -import java.util.HashMap; -import java.util.List; -import java.util.Map; -import org.tensorflow.lite.Interpreter; -import org.tensorflow.lite.support.metadata.MetadataExtractor; -import org.tensorflow.lite.support.metadata.schema.TensorMetadata; - -/** Interface to load TfLite model and provide predictions. */ -public class QaClient implements AutoCloseable { - private static final String TAG = "BertDemo"; - private static final String MODEL_PATH = "model.tflite"; - private static final String DIC_PATH = "vocab.txt"; - - private static final int MAX_ANS_LEN = 32; - private static final int MAX_QUERY_LEN = 64; - private static final int MAX_SEQ_LEN = 384; - private static final boolean DO_LOWER_CASE = true; - private static final int PREDICT_ANS_NUM = 5; - private static final int NUM_LITE_THREADS = 4; - - private static final String IDS_TENSOR_NAME = "ids"; - private static final String MASK_TENSOR_NAME = "mask"; - private static final String SEGMENT_IDS_TENSOR_NAME = "segment_ids"; - private static final String END_LOGITS_TENSOR_NAME = "end_logits"; - private static final String START_LOGITS_TENSOR_NAME = "start_logits"; - - // Need to shift 1 for outputs ([CLS]). - private static final int OUTPUT_OFFSET = 1; - - private final Context context; - private final Map dic = new HashMap<>(); - private final FeatureConverter featureConverter; - private Interpreter tflite; - private MetadataExtractor metadataExtractor = null; - - private static final Joiner SPACE_JOINER = Joiner.on(" "); - - public QaClient(Context context) { - this.context = context; - this.featureConverter = new FeatureConverter(dic, DO_LOWER_CASE, MAX_QUERY_LEN, MAX_SEQ_LEN); - } - - @WorkerThread - public synchronized void loadModel() { - try { - ByteBuffer buffer = loadModelFile(this.context.getAssets()); - Interpreter.Options opt = new Interpreter.Options(); - opt.setNumThreads(NUM_LITE_THREADS); - metadataExtractor = new MetadataExtractor(buffer); - loadDictionary(); - tflite = new Interpreter(buffer, opt); - Log.v(TAG, "TFLite model loaded."); - } catch (IOException ex) { - Log.e(TAG, ex.getMessage()); - } - } - - @WorkerThread - public synchronized void loadDictionary() { - try { - verify(metadataExtractor != null, "metadataExtractor can't be null."); - loadDictionaryFile(metadataExtractor.getAssociatedFile(DIC_PATH)); - Log.v(TAG, "Dictionary loaded."); - } catch (IOException ex) { - Log.e(TAG, ex.getMessage()); - } - } - - @WorkerThread - public synchronized void unload() { - close(); - } - - @Override - public void close() { - if (tflite != null) { - tflite.close(); - tflite = null; - } - dic.clear(); - } - - /** Load tflite model from assets. */ - public MappedByteBuffer loadModelFile(AssetManager assetManager) throws IOException { - try (AssetFileDescriptor fileDescriptor = assetManager.openFd(MODEL_PATH); - FileInputStream inputStream = new FileInputStream(fileDescriptor.getFileDescriptor())) { - FileChannel fileChannel = inputStream.getChannel(); - long startOffset = fileDescriptor.getStartOffset(); - long declaredLength = fileDescriptor.getDeclaredLength(); - return fileChannel.map(FileChannel.MapMode.READ_ONLY, startOffset, declaredLength); - } - } - - /** Load dictionary from assets. */ - public void loadDictionaryFile(InputStream inputStream) throws IOException { - try (BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream))) { - int index = 0; - while (reader.ready()) { - String key = reader.readLine(); - dic.put(key, index++); - } - } - } - - /** - * Input: Original content and query for the QA task. Later converted to Feature by - * FeatureConverter. Output: A String[] array of answers and a float[] array of corresponding - * logits. - */ - @WorkerThread - public synchronized List predict(String query, String content) { - Log.v(TAG, "TFLite model: " + MODEL_PATH + " running..."); - Log.v(TAG, "Convert Feature..."); - Feature feature = featureConverter.convert(query, content); - - Log.v(TAG, "Set inputs..."); - int[][] inputIds = new int[1][MAX_SEQ_LEN]; - int[][] inputMask = new int[1][MAX_SEQ_LEN]; - int[][] segmentIds = new int[1][MAX_SEQ_LEN]; - float[][] startLogits = new float[1][MAX_SEQ_LEN]; - float[][] endLogits = new float[1][MAX_SEQ_LEN]; - - for (int j = 0; j < MAX_SEQ_LEN; j++) { - inputIds[0][j] = feature.inputIds[j]; - inputMask[0][j] = feature.inputMask[j]; - segmentIds[0][j] = feature.segmentIds[j]; - } - - Object[] inputs = new Object[3]; - boolean useInputMetadata = false; - if (metadataExtractor != null && metadataExtractor.getInputTensorCount() == 3) { - // If metadata exists and the size of input tensors in metadata is 3, use metadata to treat - // the tensor order. Since the order of input tensors can be different for different models, - // set the inputs according to input tensor names. - useInputMetadata = true; - for (int i = 0; i < 3; i++) { - TensorMetadata inputMetadata = metadataExtractor.getInputTensorMetadata(i); - switch (inputMetadata.name()) { - case IDS_TENSOR_NAME: - inputs[i] = inputIds; - break; - case MASK_TENSOR_NAME: - inputs[i] = inputMask; - break; - case SEGMENT_IDS_TENSOR_NAME: - inputs[i] = segmentIds; - break; - default: - Log.e(TAG, "Input name in metadata doesn't match the default input tensor names."); - useInputMetadata = false; - } - } - } - if (!useInputMetadata) { - // If metadata doesn't exists or doesn't contain the info, fail back to a hard-coded order. - Log.v(TAG, "Use hard-coded order of input tensors."); - inputs[0] = inputIds; - inputs[1] = inputMask; - inputs[2] = segmentIds; - } - - Map output = new HashMap<>(); - // Hard-coded idx for output, maybe changed according to metadata below. - int endLogitsIdx = 0; - int startLogitsIdx = 1; - boolean useOutputMetadata = false; - if (metadataExtractor != null && metadataExtractor.getOutputTensorCount() == 2) { - // If metadata exists and the size of output tensors in metadata is 2, use metadata to treat - // the tensor order. Since the order of output tensors can be different for different models, - // set the indexs of the outputs according to output tensor names. - useOutputMetadata = true; - for (int i = 0; i < 2; i++) { - TensorMetadata outputMetadata = metadataExtractor.getOutputTensorMetadata(i); - switch (outputMetadata.name()) { - case END_LOGITS_TENSOR_NAME: - endLogitsIdx = i; - break; - case START_LOGITS_TENSOR_NAME: - startLogitsIdx = i; - break; - default: - Log.e(TAG, "Output name in metadata doesn't match the default output tensor names."); - useOutputMetadata = false; - } - } - } - if (!useOutputMetadata) { - Log.v(TAG, "Use hard-coded order of output tensors."); - endLogitsIdx = 0; - startLogitsIdx = 1; - } - output.put(endLogitsIdx, endLogits); - output.put(startLogitsIdx, startLogits); - - Log.v(TAG, "Run inference..."); - tflite.runForMultipleInputsOutputs(inputs, output); - - Log.v(TAG, "Convert answers..."); - List answers = getBestAnswers(startLogits[0], endLogits[0], feature); - Log.v(TAG, "Finish."); - return answers; - } - - /** Find the Best N answers & logits from the logits array and input feature. */ - private synchronized List getBestAnswers( - float[] startLogits, float[] endLogits, Feature feature) { - // Model uses the closed interval [start, end] for indices. - int[] startIndexes = getBestIndex(startLogits); - int[] endIndexes = getBestIndex(endLogits); - - List origResults = new ArrayList<>(); - for (int start : startIndexes) { - for (int end : endIndexes) { - if (!feature.tokenToOrigMap.containsKey(start + OUTPUT_OFFSET)) { - continue; - } - if (!feature.tokenToOrigMap.containsKey(end + OUTPUT_OFFSET)) { - continue; - } - if (end < start) { - continue; - } - int length = end - start + 1; - if (length > MAX_ANS_LEN) { - continue; - } - origResults.add(new QaAnswer.Pos(start, end, startLogits[start] + endLogits[end])); - } - } - - Collections.sort(origResults); - - List answers = new ArrayList<>(); - for (int i = 0; i < origResults.size(); i++) { - if (i >= PREDICT_ANS_NUM) { - break; - } - - String convertedText; - if (origResults.get(i).start > 0) { - convertedText = convertBack(feature, origResults.get(i).start, origResults.get(i).end); - } else { - convertedText = ""; - } - QaAnswer ans = new QaAnswer(convertedText, origResults.get(i)); - answers.add(ans); - } - return answers; - } - - /** Get the n-best logits from a list of all the logits. */ - @WorkerThread - private synchronized int[] getBestIndex(float[] logits) { - List tmpList = new ArrayList<>(); - for (int i = 0; i < MAX_SEQ_LEN; i++) { - tmpList.add(new QaAnswer.Pos(i, i, logits[i])); - } - Collections.sort(tmpList); - - int[] indexes = new int[PREDICT_ANS_NUM]; - for (int i = 0; i < PREDICT_ANS_NUM; i++) { - indexes[i] = tmpList.get(i).start; - } - - return indexes; - } - - /** Convert the answer back to original text form. */ - @WorkerThread - private static String convertBack(Feature feature, int start, int end) { - // Shifted index is: index of logits + offset. - int shiftedStart = start + OUTPUT_OFFSET; - int shiftedEnd = end + OUTPUT_OFFSET; - int startIndex = feature.tokenToOrigMap.get(shiftedStart); - int endIndex = feature.tokenToOrigMap.get(shiftedEnd); - // end + 1 for the closed interval. - String ans = SPACE_JOINER.join(feature.origTokens.subList(startIndex, endIndex + 1)); - return ans; - } -} diff --git a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/tflite/QaClient.java b/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/tflite/QaClient.java new file mode 100644 index 00000000000..d9795cd2cf7 --- /dev/null +++ b/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/tflite/QaClient.java @@ -0,0 +1,49 @@ +package org.tensorflow.lite.examples.bertqa.tflite; + +import android.content.Context; +import android.util.Log; + +import org.tensorflow.lite.task.text.qa.BertQuestionAnswerer; +import org.tensorflow.lite.task.text.qa.QaAnswer; + + +import java.io.IOException; +import java.util.List; + +/** Load TfLite model and provide predictions with task api. */ +public class QaClient { + + private static final String TAG = "TaskApi"; + private static final String MODEL_PATH = "model.tflite"; + + private final Context context; + + public BertQuestionAnswerer answerer; + + public QaClient(Context context) { + this.context = context; + } + + /** Load TF Lite model. */ + public void loadModel() { + try { + answerer = BertQuestionAnswerer.createFromFile(context, MODEL_PATH); + } catch (IOException e) { + Log.e(TAG, e.getMessage()); + } + } + + /** Free up resources as the client is no longer needed. */ + public void unload() { + answerer.close(); + answerer = null; + } + + + /** Run inference and predict the possible answers. */ + public List predict(String questionToAsk, String contextOfTheQuestion) { + + List apiResult = answerer.answer(contextOfTheQuestion, questionToAsk); + return apiResult; + } +} diff --git a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/tokenization/BasicTokenizer.java b/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/tokenization/BasicTokenizer.java deleted file mode 100644 index 991f0604fab..00000000000 --- a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/tokenization/BasicTokenizer.java +++ /dev/null @@ -1,104 +0,0 @@ -/* Copyright 2019 The TensorFlow Authors. All Rights Reserved. - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -==============================================================================*/ -package org.tensorflow.lite.examples.bertqa.tokenization; - -import com.google.common.base.Ascii; -import com.google.common.collect.Iterables; -import java.util.ArrayList; -import java.util.Arrays; -import java.util.List; - -/** Basic tokenization (punctuation splitting, lower casing, etc.) */ -public final class BasicTokenizer { - private final boolean doLowerCase; - - public BasicTokenizer(boolean doLowerCase) { - this.doLowerCase = doLowerCase; - } - - public List tokenize(String text) { - String cleanedText = cleanText(text); - - List origTokens = whitespaceTokenize(cleanedText); - - StringBuilder stringBuilder = new StringBuilder(); - for (String token : origTokens) { - if (doLowerCase) { - token = Ascii.toLowerCase(token); - } - List list = runSplitOnPunc(token); - for (String subToken : list) { - stringBuilder.append(subToken).append(" "); - } - } - return whitespaceTokenize(stringBuilder.toString()); - } - - /* Performs invalid character removal and whitespace cleanup on text. */ - static String cleanText(String text) { - if (text == null) { - throw new NullPointerException("The input String is null."); - } - - StringBuilder stringBuilder = new StringBuilder(""); - for (int index = 0; index < text.length(); index++) { - char ch = text.charAt(index); - - // Skip the characters that cannot be used. - if (CharChecker.isInvalid(ch) || CharChecker.isControl(ch)) { - continue; - } - if (CharChecker.isWhitespace(ch)) { - stringBuilder.append(" "); - } else { - stringBuilder.append(ch); - } - } - return stringBuilder.toString(); - } - - /* Runs basic whitespace cleaning and splitting on a piece of text. */ - static List whitespaceTokenize(String text) { - if (text == null) { - throw new NullPointerException("The input String is null."); - } - return Arrays.asList(text.split(" ")); - } - - /* Splits punctuation on a piece of text. */ - static List runSplitOnPunc(String text) { - if (text == null) { - throw new NullPointerException("The input String is null."); - } - - List tokens = new ArrayList<>(); - boolean startNewWord = true; - for (int i = 0; i < text.length(); i++) { - char ch = text.charAt(i); - if (CharChecker.isPunctuation(ch)) { - tokens.add(String.valueOf(ch)); - startNewWord = true; - } else { - if (startNewWord) { - tokens.add(""); - startNewWord = false; - } - tokens.set(tokens.size() - 1, Iterables.getLast(tokens) + ch); - } - } - - return tokens; - } -} diff --git a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/tokenization/CharChecker.java b/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/tokenization/CharChecker.java deleted file mode 100644 index dc08dbe5e70..00000000000 --- a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/tokenization/CharChecker.java +++ /dev/null @@ -1,58 +0,0 @@ -/* Copyright 2019 The TensorFlow Authors. All Rights Reserved. - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -==============================================================================*/ -package org.tensorflow.lite.examples.bertqa.tokenization; - -/** To check whether a char is whitespace/control/punctuation. */ -final class CharChecker { - - /** To judge whether it's an empty or unknown character. */ - public static boolean isInvalid(char ch) { - return (ch == 0 || ch == 0xfffd); - } - - /** To judge whether it's a control character(exclude whitespace). */ - public static boolean isControl(char ch) { - if (Character.isWhitespace(ch)) { - return false; - } - int type = Character.getType(ch); - return (type == Character.CONTROL || type == Character.FORMAT); - } - - /** To judge whether it can be regarded as a whitespace. */ - public static boolean isWhitespace(char ch) { - if (Character.isWhitespace(ch)) { - return true; - } - int type = Character.getType(ch); - return (type == Character.SPACE_SEPARATOR - || type == Character.LINE_SEPARATOR - || type == Character.PARAGRAPH_SEPARATOR); - } - - /** To judge whether it's a punctuation. */ - public static boolean isPunctuation(char ch) { - int type = Character.getType(ch); - return (type == Character.CONNECTOR_PUNCTUATION - || type == Character.DASH_PUNCTUATION - || type == Character.START_PUNCTUATION - || type == Character.END_PUNCTUATION - || type == Character.INITIAL_QUOTE_PUNCTUATION - || type == Character.FINAL_QUOTE_PUNCTUATION - || type == Character.OTHER_PUNCTUATION); - } - - private CharChecker() {} -} diff --git a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/tokenization/FullTokenizer.java b/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/tokenization/FullTokenizer.java deleted file mode 100644 index 01c78d9ef6a..00000000000 --- a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/tokenization/FullTokenizer.java +++ /dev/null @@ -1,52 +0,0 @@ -/* Copyright 2019 The TensorFlow Authors. All Rights Reserved. - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -==============================================================================*/ -package org.tensorflow.lite.examples.bertqa.tokenization; - -import java.util.ArrayList; -import java.util.List; -import java.util.Map; - -/** - * A java realization of Bert tokenization. Original python code: - * https://github.com/google-research/bert/blob/master/tokenization.py runs full tokenization to - * tokenize a String into split subtokens or ids. - */ -public final class FullTokenizer { - private final BasicTokenizer basicTokenizer; - private final WordpieceTokenizer wordpieceTokenizer; - private final Map dic; - - public FullTokenizer(Map inputDic, boolean doLowerCase) { - dic = inputDic; - basicTokenizer = new BasicTokenizer(doLowerCase); - wordpieceTokenizer = new WordpieceTokenizer(inputDic); - } - - public List tokenize(String text) { - List splitTokens = new ArrayList<>(); - for (String token : basicTokenizer.tokenize(text)) { - splitTokens.addAll(wordpieceTokenizer.tokenize(token)); - } - return splitTokens; - } - - public List convertTokensToIds(List tokens) { - List outputIds = new ArrayList<>(); - for (String token : tokens) { - outputIds.add(dic.get(token)); - } - return outputIds; - } -} diff --git a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/tokenization/WordpieceTokenizer.java b/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/tokenization/WordpieceTokenizer.java deleted file mode 100644 index 446f80bcf31..00000000000 --- a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/tokenization/WordpieceTokenizer.java +++ /dev/null @@ -1,94 +0,0 @@ -/* Copyright 2019 The TensorFlow Authors. All Rights Reserved. - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -==============================================================================*/ -package org.tensorflow.lite.examples.bertqa.tokenization; - -import java.util.ArrayList; -import java.util.List; -import java.util.Map; - -/** Word piece tokenization to split a piece of text into its word pieces. */ -public final class WordpieceTokenizer { - private final Map dic; - - private static final String UNKNOWN_TOKEN = "[UNK]"; // For unknown words. - private static final int MAX_INPUTCHARS_PER_WORD = 200; - - public WordpieceTokenizer(Map vocab) { - dic = vocab; - } - - /** - * Tokenizes a piece of text into its word pieces. This uses a greedy longest-match-first - * algorithm to perform tokenization using the given vocabulary. For example: input = "unaffable", - * output = ["un", "##aff", "##able"]. - * - * @param text: A single token or whitespace separated tokens. This should have already been - * passed through `BasicTokenizer. - * @return A list of wordpiece tokens. - */ - public List tokenize(String text) { - if (text == null) { - throw new NullPointerException("The input String is null."); - } - - List outputTokens = new ArrayList<>(); - for (String token : BasicTokenizer.whitespaceTokenize(text)) { - - if (token.length() > MAX_INPUTCHARS_PER_WORD) { - outputTokens.add(UNKNOWN_TOKEN); - continue; - } - - boolean isBad = false; // Mark if a word cannot be tokenized into known subwords. - int start = 0; - List subTokens = new ArrayList<>(); - - while (start < token.length()) { - String curSubStr = ""; - - int end = token.length(); // Longer substring matches first. - while (start < end) { - String subStr = - (start == 0) ? token.substring(start, end) : "##" + token.substring(start, end); - if (dic.containsKey(subStr)) { - curSubStr = subStr; - break; - } - end--; - } - - // The word doesn't contain any known subwords. - if ("".equals(curSubStr)) { - isBad = true; - break; - } - - // curSubStr is the longeset subword that can be found. - subTokens.add(curSubStr); - - // Proceed to tokenize the resident string. - start = end; - } - - if (isBad) { - outputTokens.add(UNKNOWN_TOKEN); - } else { - outputTokens.addAll(subTokens); - } - } - - return outputTokens; - } -} diff --git a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ui/DatasetListActivity.java b/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ui/DatasetListActivity.java index 80441f87f02..7382b343950 100644 --- a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ui/DatasetListActivity.java +++ b/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ui/DatasetListActivity.java @@ -19,7 +19,7 @@ import android.widget.ArrayAdapter; import android.widget.ListView; import org.tensorflow.lite.examples.bertqa.R; -import org.tensorflow.lite.examples.bertqa.ml.LoadDatasetClient; +import org.tensorflow.lite.examples.bertqa.data.LoadDatasetClient; /** * An activity representing a list of Datasets. This activity has different presentations for diff --git a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ui/QaActivity.java b/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ui/QaActivity.java index 09f251b7c30..b44e09f45d3 100644 --- a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ui/QaActivity.java +++ b/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ui/QaActivity.java @@ -39,10 +39,11 @@ import com.google.android.material.textfield.TextInputEditText; import java.util.List; import java.util.Locale; + +import org.tensorflow.lite.examples.bertqa.tflite.QaClient; import org.tensorflow.lite.examples.bertqa.R; -import org.tensorflow.lite.examples.bertqa.ml.LoadDatasetClient; -import org.tensorflow.lite.examples.bertqa.ml.QaAnswer; -import org.tensorflow.lite.examples.bertqa.ml.QaClient; +import org.tensorflow.lite.examples.bertqa.data.LoadDatasetClient; +import org.tensorflow.lite.task.text.qa.QaAnswer; /** Activity for doing Q&A on a specific dataset */ public class QaActivity extends AppCompatActivity { From 6be2884673962db3c5b6a2d0e789932e8502901d Mon Sep 17 00:00:00 2001 From: Sunit Roy <2703iamsry@gmail.com> Date: Fri, 25 Jun 2021 01:16:59 +0530 Subject: [PATCH 2/7] renaming folders & readme update --- lite/examples/bert_qa/android/README.md | 13 +- .../lite/examples/bertqa/UnitTest.java | 2 +- .../{data => ml}/LoadDatasetClient.java | 3 +- .../bertqa/{tflite => ml}/QaClient.java | 18 +- .../bertqa/ui/DatasetListActivity.java | 43 +- .../lite/examples/bertqa/ui/QaActivity.java | 409 +++++++++--------- .../examples/bertqa/ui/QuestionAdapter.java | 83 ++-- 7 files changed, 303 insertions(+), 268 deletions(-) rename lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/{data => ml}/LoadDatasetClient.java (96%) rename lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/{tflite => ml}/QaClient.java (77%) diff --git a/lite/examples/bert_qa/android/README.md b/lite/examples/bert_qa/android/README.md index eae315db913..8c58e784012 100644 --- a/lite/examples/bert_qa/android/README.md +++ b/lite/examples/bert_qa/android/README.md @@ -18,14 +18,14 @@ These instructions walk you through running the demo on an Android device. of pre-training language representations which obtains state-of-the-art results on a wide array of Natural Language Processing tasks. -This app uses [MobileBERT](https://tfhub.dev/tensorflow/tfjs-model/mobilebert/1 "MobileBERT"), a compressed version of [BERT] that runs 4x faster and +This app uses [MobileBERT], a compressed version of [BERT] that runs 4x faster and has 4x smaller model size. For more information, refer to the [BERT github page][BERT]. ## Build the demo using Android Studio - + ### Prerequisites * If you don't have already, install @@ -82,5 +82,12 @@ cd lite/examples/bert_qa/android # Folder for Android app. adb install app/build/outputs/apk/debug/app-debug.apk ``` +## Assets folder + +_Do not delete the assets folder content_. If you explicitly deleted the files, +choose `Build -> Rebuild` to re-download the deleted model files into the assets +folder. + [BERT]: https://github.com/google-research/bert "Bert" -[SQuAD]: https://rajpurkar.github.io/SQuAD-explorer/ "SQuAD" \ No newline at end of file +[SQuAD]: https://rajpurkar.github.io/SQuAD-explorer/ "SQuAD" +[MobileBERT]:https://tfhub.dev/tensorflow/tfjs-model/mobilebert/1 "MobileBERT" \ No newline at end of file diff --git a/lite/examples/bert_qa/android/app/src/androidTest/java/org/tensorflow/lite/examples/bertqa/UnitTest.java b/lite/examples/bert_qa/android/app/src/androidTest/java/org/tensorflow/lite/examples/bertqa/UnitTest.java index 51b4a009236..65bd56b10e2 100644 --- a/lite/examples/bert_qa/android/app/src/androidTest/java/org/tensorflow/lite/examples/bertqa/UnitTest.java +++ b/lite/examples/bert_qa/android/app/src/androidTest/java/org/tensorflow/lite/examples/bertqa/UnitTest.java @@ -8,7 +8,7 @@ import org.junit.Before; import org.junit.Test; import org.junit.runner.RunWith; -import org.tensorflow.lite.examples.bertqa.tflite.QaClient; +import org.tensorflow.lite.examples.bertqa.ml.QaClient; import org.tensorflow.lite.task.text.qa.QaAnswer; import static org.junit.Assert.*; diff --git a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/data/LoadDatasetClient.java b/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ml/LoadDatasetClient.java similarity index 96% rename from lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/data/LoadDatasetClient.java rename to lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ml/LoadDatasetClient.java index 0cd668286d9..9e6064ad9d9 100644 --- a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/data/LoadDatasetClient.java +++ b/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ml/LoadDatasetClient.java @@ -12,7 +12,7 @@ See the License for the specific language governing permissions and limitations under the License. ==============================================================================*/ -package org.tensorflow.lite.examples.bertqa.data; +package org.tensorflow.lite.examples.bertqa.ml; import android.content.Context; import android.util.Log; @@ -35,7 +35,6 @@ public class LoadDatasetClient { private static final String TAG = "BertAppDemo"; private static final String JSON_DIR = "qa.json"; - private static final String DIC_DIR = "vocab.txt"; private final Context context; private String[] contents; diff --git a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/tflite/QaClient.java b/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ml/QaClient.java similarity index 77% rename from lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/tflite/QaClient.java rename to lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ml/QaClient.java index d9795cd2cf7..184f079634f 100644 --- a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/tflite/QaClient.java +++ b/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ml/QaClient.java @@ -1,4 +1,4 @@ -package org.tensorflow.lite.examples.bertqa.tflite; +package org.tensorflow.lite.examples.bertqa.ml; import android.content.Context; import android.util.Log; @@ -10,7 +10,9 @@ import java.io.IOException; import java.util.List; -/** Load TfLite model and provide predictions with task api. */ +/** + * Load TfLite model and provide predictions with task api. + */ public class QaClient { private static final String TAG = "TaskApi"; @@ -24,7 +26,9 @@ public QaClient(Context context) { this.context = context; } - /** Load TF Lite model. */ + /** + * Load TF Lite model. + */ public void loadModel() { try { answerer = BertQuestionAnswerer.createFromFile(context, MODEL_PATH); @@ -33,14 +37,18 @@ public void loadModel() { } } - /** Free up resources as the client is no longer needed. */ + /** + * Free up resources as the client is no longer needed. + */ public void unload() { answerer.close(); answerer = null; } - /** Run inference and predict the possible answers. */ + /** + * Run inference and predict the possible answers. + */ public List predict(String questionToAsk, String contextOfTheQuestion) { List apiResult = answerer.answer(contextOfTheQuestion, questionToAsk); diff --git a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ui/DatasetListActivity.java b/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ui/DatasetListActivity.java index 7382b343950..eefe2537b5e 100644 --- a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ui/DatasetListActivity.java +++ b/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ui/DatasetListActivity.java @@ -15,11 +15,14 @@ package org.tensorflow.lite.examples.bertqa.ui; import android.os.Bundle; + import androidx.appcompat.app.AppCompatActivity; + import android.widget.ArrayAdapter; import android.widget.ListView; + import org.tensorflow.lite.examples.bertqa.R; -import org.tensorflow.lite.examples.bertqa.data.LoadDatasetClient; +import org.tensorflow.lite.examples.bertqa.ml.LoadDatasetClient; /** * An activity representing a list of Datasets. This activity has different presentations for @@ -29,23 +32,23 @@ */ public class DatasetListActivity extends AppCompatActivity { - @Override - protected void onCreate(Bundle savedInstanceState) { - super.onCreate(savedInstanceState); - setContentView(R.layout.tfe_qa_activity_dataset_list); - - ListView listView = findViewById(R.id.dataset_list); - assert listView != null; - - LoadDatasetClient datasetClient = new LoadDatasetClient(this); - ArrayAdapter datasetAdapter = - new ArrayAdapter<>( - this, android.R.layout.simple_selectable_list_item, datasetClient.getTitles()); - listView.setAdapter(datasetAdapter); - - listView.setOnItemClickListener( - (parent, view, position, id) -> { - startActivity(QaActivity.newInstance(this, position)); - }); - } + @Override + protected void onCreate(Bundle savedInstanceState) { + super.onCreate(savedInstanceState); + setContentView(R.layout.tfe_qa_activity_dataset_list); + + ListView listView = findViewById(R.id.dataset_list); + assert listView != null; + + LoadDatasetClient datasetClient = new LoadDatasetClient(this); + ArrayAdapter datasetAdapter = + new ArrayAdapter<>( + this, android.R.layout.simple_selectable_list_item, datasetClient.getTitles()); + listView.setAdapter(datasetAdapter); + + listView.setOnItemClickListener( + (parent, view, position, id) -> { + startActivity(QaActivity.newInstance(this, position)); + }); + } } diff --git a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ui/QaActivity.java b/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ui/QaActivity.java index b44e09f45d3..64e8372f5d0 100644 --- a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ui/QaActivity.java +++ b/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ui/QaActivity.java @@ -20,9 +20,11 @@ import android.os.Handler; import android.os.HandlerThread; import android.speech.tts.TextToSpeech; + import androidx.appcompat.app.AppCompatActivity; import androidx.recyclerview.widget.LinearLayoutManager; import androidx.recyclerview.widget.RecyclerView; + import android.text.Editable; import android.text.Spannable; import android.text.SpannableString; @@ -35,228 +37,235 @@ import android.view.inputmethod.InputMethodManager; import android.widget.ImageButton; import android.widget.TextView; + import com.google.android.material.snackbar.Snackbar; import com.google.android.material.textfield.TextInputEditText; + import java.util.List; import java.util.Locale; -import org.tensorflow.lite.examples.bertqa.tflite.QaClient; +import org.tensorflow.lite.examples.bertqa.ml.QaClient; import org.tensorflow.lite.examples.bertqa.R; -import org.tensorflow.lite.examples.bertqa.data.LoadDatasetClient; +import org.tensorflow.lite.examples.bertqa.ml.LoadDatasetClient; import org.tensorflow.lite.task.text.qa.QaAnswer; -/** Activity for doing Q&A on a specific dataset */ +/** + * Activity for doing Q&A on a specific dataset + */ public class QaActivity extends AppCompatActivity { - private static final String DATASET_POSITION_KEY = "DATASET_POSITION"; - private static final String TAG = "QaActivity"; - private static final boolean DISPLAY_RUNNING_TIME = false; - - private TextInputEditText questionEditText; - private TextView contentTextView; - private TextToSpeech textToSpeech; - - private boolean questionAnswered = false; - private String content; - private Handler handler; - private QaClient qaClient; - - public static Intent newInstance(Context context, int datasetPosition) { - Intent intent = new Intent(context, QaActivity.class); - intent.putExtra(DATASET_POSITION_KEY, datasetPosition); - return intent; - } - - @Override - protected void onCreate(Bundle savedInstanceState) { - Log.v(TAG, "onCreate"); - super.onCreate(savedInstanceState); - setContentView(R.layout.tfe_qa_activity_qa); - - // Get content of the selected dataset. - int datasetPosition = getIntent().getIntExtra(DATASET_POSITION_KEY, -1); - LoadDatasetClient datasetClient = new LoadDatasetClient(this); - - // Show the dataset title. - TextView titleText = findViewById(R.id.title_text); - titleText.setText(datasetClient.getTitles()[datasetPosition]); - - // Show the text content of the selected dataset. - content = datasetClient.getContent(datasetPosition); - contentTextView = findViewById(R.id.content_text); - contentTextView.setText(content); - contentTextView.setMovementMethod(new ScrollingMovementMethod()); - - // Setup question suggestion list. - RecyclerView questionSuggestionsView = findViewById(R.id.suggestion_list); - QuestionAdapter adapter = - new QuestionAdapter(this, datasetClient.getQuestions(datasetPosition)); - adapter.setOnQuestionSelectListener(question -> answerQuestion(question)); - questionSuggestionsView.setAdapter(adapter); - LinearLayoutManager layoutManager = - new LinearLayoutManager(this, LinearLayoutManager.HORIZONTAL, false); - questionSuggestionsView.setLayoutManager(layoutManager); - - // Setup ask button. - ImageButton askButton = findViewById(R.id.ask_button); - askButton.setOnClickListener( - view -> answerQuestion(questionEditText.getText().toString())); - - // Setup text edit where users can input their question. - questionEditText = findViewById(R.id.question_edit_text); - questionEditText.setOnFocusChangeListener( - (view, hasFocus) -> { - // If we already answer current question, clear the question so that user can input a new - // one. - if (hasFocus && questionAnswered) { - questionEditText.setText(null); - } - }); - questionEditText.addTextChangedListener( - new TextWatcher() { - @Override - public void beforeTextChanged(CharSequence charSequence, int i, int i1, int i2) {} - - @Override - public void onTextChanged(CharSequence charSequence, int i, int i1, int i2) { - // Only allow clicking Ask button if there is a question. - boolean shouldAskButtonActive = !charSequence.toString().isEmpty(); - askButton.setClickable(shouldAskButtonActive); - askButton.setImageResource( - shouldAskButtonActive ? R.drawable.ic_ask_active : R.drawable.ic_ask_inactive); - } - - @Override - public void afterTextChanged(Editable editable) {} - }); - questionEditText.setOnKeyListener( - (v, keyCode, event) -> { - if (event.getAction() == KeyEvent.ACTION_UP && keyCode == KeyEvent.KEYCODE_ENTER) { - answerQuestion(questionEditText.getText().toString()); - } - return false; - }); - - // Setup QA client to and background thread to run inference. - HandlerThread handlerThread = new HandlerThread("QAClient"); - handlerThread.start(); - handler = new Handler(handlerThread.getLooper()); - qaClient = new QaClient(this); - } - - @Override - protected void onStart() { - Log.v(TAG, "onStart"); - super.onStart(); - handler.post( - () -> { - qaClient.loadModel(); - }); - - textToSpeech = - new TextToSpeech( - this, - status -> { - if (status == TextToSpeech.SUCCESS) { - textToSpeech.setLanguage(Locale.US); - } else { - textToSpeech = null; - } - }); - } - - @Override - protected void onStop() { - Log.v(TAG, "onStop"); - super.onStop(); - handler.post(() -> qaClient.unload()); - - if (textToSpeech != null) { - textToSpeech.stop(); - textToSpeech.shutdown(); - } - } + private static final String DATASET_POSITION_KEY = "DATASET_POSITION"; + private static final String TAG = "QaActivity"; + private static final boolean DISPLAY_RUNNING_TIME = false; - private void answerQuestion(String question) { - question = question.trim(); - if (question.isEmpty()) { - questionEditText.setText(question); - return; - } + private TextInputEditText questionEditText; + private TextView contentTextView; + private TextToSpeech textToSpeech; - // Append question mark '?' if not ended with '?'. - // This aligns with question format that trains the model. - if (!question.endsWith("?")) { - question += '?'; + private boolean questionAnswered = false; + private String content; + private Handler handler; + private QaClient qaClient; + + public static Intent newInstance(Context context, int datasetPosition) { + Intent intent = new Intent(context, QaActivity.class); + intent.putExtra(DATASET_POSITION_KEY, datasetPosition); + return intent; } - final String questionToAsk = question; - questionEditText.setText(questionToAsk); - - // Delete all pending tasks. - handler.removeCallbacksAndMessages(null); - - // Hide keyboard and dismiss focus on text edit. - InputMethodManager imm = - (InputMethodManager) getSystemService(AppCompatActivity.INPUT_METHOD_SERVICE); - imm.hideSoftInputFromWindow(getWindow().getDecorView().getWindowToken(), 0); - View focusView = getCurrentFocus(); - if (focusView != null) { - focusView.clearFocus(); + + @Override + protected void onCreate(Bundle savedInstanceState) { + Log.v(TAG, "onCreate"); + super.onCreate(savedInstanceState); + setContentView(R.layout.tfe_qa_activity_qa); + + // Get content of the selected dataset. + int datasetPosition = getIntent().getIntExtra(DATASET_POSITION_KEY, -1); + LoadDatasetClient datasetClient = new LoadDatasetClient(this); + + // Show the dataset title. + TextView titleText = findViewById(R.id.title_text); + titleText.setText(datasetClient.getTitles()[datasetPosition]); + + // Show the text content of the selected dataset. + content = datasetClient.getContent(datasetPosition); + contentTextView = findViewById(R.id.content_text); + contentTextView.setText(content); + contentTextView.setMovementMethod(new ScrollingMovementMethod()); + + // Setup question suggestion list. + RecyclerView questionSuggestionsView = findViewById(R.id.suggestion_list); + QuestionAdapter adapter = + new QuestionAdapter(this, datasetClient.getQuestions(datasetPosition)); + adapter.setOnQuestionSelectListener(question -> answerQuestion(question)); + questionSuggestionsView.setAdapter(adapter); + LinearLayoutManager layoutManager = + new LinearLayoutManager(this, LinearLayoutManager.HORIZONTAL, false); + questionSuggestionsView.setLayoutManager(layoutManager); + + // Setup ask button. + ImageButton askButton = findViewById(R.id.ask_button); + askButton.setOnClickListener( + view -> answerQuestion(questionEditText.getText().toString())); + + // Setup text edit where users can input their question. + questionEditText = findViewById(R.id.question_edit_text); + questionEditText.setOnFocusChangeListener( + (view, hasFocus) -> { + // If we already answer current question, clear the question so that user can input a new + // one. + if (hasFocus && questionAnswered) { + questionEditText.setText(null); + } + }); + questionEditText.addTextChangedListener( + new TextWatcher() { + @Override + public void beforeTextChanged(CharSequence charSequence, int i, int i1, int i2) { + } + + @Override + public void onTextChanged(CharSequence charSequence, int i, int i1, int i2) { + // Only allow clicking Ask button if there is a question. + boolean shouldAskButtonActive = !charSequence.toString().isEmpty(); + askButton.setClickable(shouldAskButtonActive); + askButton.setImageResource( + shouldAskButtonActive ? R.drawable.ic_ask_active : R.drawable.ic_ask_inactive); + } + + @Override + public void afterTextChanged(Editable editable) { + } + }); + questionEditText.setOnKeyListener( + (v, keyCode, event) -> { + if (event.getAction() == KeyEvent.ACTION_UP && keyCode == KeyEvent.KEYCODE_ENTER) { + answerQuestion(questionEditText.getText().toString()); + } + return false; + }); + + // Setup QA client to and background thread to run inference. + HandlerThread handlerThread = new HandlerThread("QAClient"); + handlerThread.start(); + handler = new Handler(handlerThread.getLooper()); + qaClient = new QaClient(this); } - // Reset content text view - contentTextView.setText(content); + @Override + protected void onStart() { + Log.v(TAG, "onStart"); + super.onStart(); + handler.post( + () -> { + qaClient.loadModel(); + }); - questionAnswered = false; + textToSpeech = + new TextToSpeech( + this, + status -> { + if (status == TextToSpeech.SUCCESS) { + textToSpeech.setLanguage(Locale.US); + } else { + textToSpeech = null; + } + }); + } - Snackbar runningSnackbar = - Snackbar.make(contentTextView, "Looking up answer...", Integer.MAX_VALUE); - runningSnackbar.show(); + @Override + protected void onStop() { + Log.v(TAG, "onStop"); + super.onStop(); + handler.post(() -> qaClient.unload()); - // Run TF Lite model to get the answer. - handler.post( - () -> { - long beforeTime = System.currentTimeMillis(); - final List answers = qaClient.predict(questionToAsk, content); - long afterTime = System.currentTimeMillis(); - double totalSeconds = (afterTime - beforeTime) / 1000.0; + if (textToSpeech != null) { + textToSpeech.stop(); + textToSpeech.shutdown(); + } + } - if (!answers.isEmpty()) { - // Get the top answer - QaAnswer topAnswer = answers.get(0); - // Show the answer. - runOnUiThread( + private void answerQuestion(String question) { + question = question.trim(); + if (question.isEmpty()) { + questionEditText.setText(question); + return; + } + + // Append question mark '?' if not ended with '?'. + // This aligns with question format that trains the model. + if (!question.endsWith("?")) { + question += '?'; + } + final String questionToAsk = question; + questionEditText.setText(questionToAsk); + + // Delete all pending tasks. + handler.removeCallbacksAndMessages(null); + + // Hide keyboard and dismiss focus on text edit. + InputMethodManager imm = + (InputMethodManager) getSystemService(AppCompatActivity.INPUT_METHOD_SERVICE); + imm.hideSoftInputFromWindow(getWindow().getDecorView().getWindowToken(), 0); + View focusView = getCurrentFocus(); + if (focusView != null) { + focusView.clearFocus(); + } + + // Reset content text view + contentTextView.setText(content); + + questionAnswered = false; + + // Start showing Looking up snackbar + Snackbar runningSnackbar = + Snackbar.make(contentTextView, "Looking up answer...", Snackbar.LENGTH_INDEFINITE); + runningSnackbar.show(); + + // Run TF Lite model to get the answer. + handler.post( () -> { - runningSnackbar.dismiss(); - presentAnswer(topAnswer); - - String displayMessage = "Top answer was successfully highlighted."; - if (DISPLAY_RUNNING_TIME) { - displayMessage = String.format("%s %.3fs.", displayMessage, totalSeconds); - } - Snackbar.make(contentTextView, displayMessage, Snackbar.LENGTH_LONG).show(); - questionAnswered = true; + long beforeTime = System.currentTimeMillis(); + final List answers = qaClient.predict(questionToAsk, content); + long afterTime = System.currentTimeMillis(); + double totalSeconds = (afterTime - beforeTime) / 1000.0; + + if (!answers.isEmpty()) { + // Get the top answer + QaAnswer topAnswer = answers.get(0); + // Dismiss the snackbar and show the answer. + runOnUiThread( + () -> { + runningSnackbar.dismiss(); + presentAnswer(topAnswer); + + String displayMessage = "Top answer was successfully highlighted."; + if (DISPLAY_RUNNING_TIME) { + displayMessage = String.format("%s %.3fs.", displayMessage, totalSeconds); + } + Snackbar.make(contentTextView, displayMessage, Snackbar.LENGTH_LONG).show(); + questionAnswered = true; + }); + } }); - } - }); - } - - private void presentAnswer(QaAnswer answer) { - // Highlight answer. - Spannable spanText = new SpannableString(content); - int offset = content.indexOf(answer.text, 0); - if (offset >= 0) { - spanText.setSpan( - new BackgroundColorSpan(getColor(R.color.tfe_qa_color_highlight)), - offset, - offset + answer.text.length(), - Spannable.SPAN_EXCLUSIVE_EXCLUSIVE); } - contentTextView.setText(spanText); - // Use TTS to speak out the answer. - if (textToSpeech != null) { - textToSpeech.speak(answer.text, TextToSpeech.QUEUE_FLUSH, null, answer.text); + private void presentAnswer(QaAnswer answer) { + // Highlight answer. + Spannable spanText = new SpannableString(content); + int offset = content.indexOf(answer.text, 0); + if (offset >= 0) { + spanText.setSpan( + new BackgroundColorSpan(getColor(R.color.tfe_qa_color_highlight)), + offset, + offset + answer.text.length(), + Spannable.SPAN_EXCLUSIVE_EXCLUSIVE); + } + contentTextView.setText(spanText); + + // Use TTS to speak out the answer. + if (textToSpeech != null) { + textToSpeech.speak(answer.text, TextToSpeech.QUEUE_FLUSH, null, answer.text); + } } - } } diff --git a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ui/QuestionAdapter.java b/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ui/QuestionAdapter.java index c4f69ac22d3..30366136a12 100644 --- a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ui/QuestionAdapter.java +++ b/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ui/QuestionAdapter.java @@ -15,63 +15,72 @@ package org.tensorflow.lite.examples.bertqa.ui; import android.content.Context; + import androidx.recyclerview.widget.RecyclerView; + import android.view.LayoutInflater; import android.view.View; import android.view.ViewGroup; + import androidx.annotation.NonNull; + import com.google.android.material.chip.Chip; + import org.tensorflow.lite.examples.bertqa.R; -/** Adapter class to show question suggestion chips. */ +/** + * Adapter class to show question suggestion chips. + */ public class QuestionAdapter extends RecyclerView.Adapter { - private LayoutInflater inflater; - private String[] questions; - private OnQuestionSelectListener onQuestionSelectListener; + private LayoutInflater inflater; + private String[] questions; + private OnQuestionSelectListener onQuestionSelectListener; - public QuestionAdapter(Context context, String[] questions) { - inflater = LayoutInflater.from(context); - this.questions = questions; - } + public QuestionAdapter(Context context, String[] questions) { + inflater = LayoutInflater.from(context); + this.questions = questions; + } - @Override - public QuestionAdapter.MyViewHolder onCreateViewHolder(@NonNull ViewGroup parent, int viewType) { + @Override + public QuestionAdapter.MyViewHolder onCreateViewHolder(@NonNull ViewGroup parent, int viewType) { - View view = inflater.inflate(R.layout.tfe_qa_question_chip, parent, false); - MyViewHolder holder = new MyViewHolder(view); + View view = inflater.inflate(R.layout.tfe_qa_question_chip, parent, false); + MyViewHolder holder = new MyViewHolder(view); - return holder; - } + return holder; + } - @Override - public void onBindViewHolder(QuestionAdapter.MyViewHolder holder, int position) { - holder.chip.setText(questions[position]); - holder.chip.setOnClickListener( - view -> onQuestionSelectListener.onQuestionSelect(questions[position])); - } + @Override + public void onBindViewHolder(QuestionAdapter.MyViewHolder holder, int position) { + holder.chip.setText(questions[position]); + holder.chip.setOnClickListener( + view -> onQuestionSelectListener.onQuestionSelect(questions[position])); + } - @Override - public int getItemCount() { - return questions.length; - } + @Override + public int getItemCount() { + return questions.length; + } - public void setOnQuestionSelectListener(OnQuestionSelectListener onQuestionSelectListener) { - this.onQuestionSelectListener = onQuestionSelectListener; - } + public void setOnQuestionSelectListener(OnQuestionSelectListener onQuestionSelectListener) { + this.onQuestionSelectListener = onQuestionSelectListener; + } - class MyViewHolder extends RecyclerView.ViewHolder { + class MyViewHolder extends RecyclerView.ViewHolder { - Chip chip; + Chip chip; - public MyViewHolder(View itemView) { - super(itemView); - chip = itemView.findViewById(R.id.chip); + public MyViewHolder(View itemView) { + super(itemView); + chip = itemView.findViewById(R.id.chip); + } } - } - /** Interface for callback when a question is selected. */ - public interface OnQuestionSelectListener { - void onQuestionSelect(String question); - } + /** + * Interface for callback when a question is selected. + */ + public interface OnQuestionSelectListener { + void onQuestionSelect(String question); + } } From 1273c41c66ff38cec3023f7383f74db34f8b8eaa Mon Sep 17 00:00:00 2001 From: Sunit Roy <2703iamsry@gmail.com> Date: Tue, 29 Jun 2021 18:59:36 +0530 Subject: [PATCH 3/7] Explore_the_CODE.md added --- lite/examples/bert_qa/android/EXPLORE_THE_CODE.md | 0 lite/examples/bert_qa/android/README.md | 3 ++- 2 files changed, 2 insertions(+), 1 deletion(-) create mode 100644 lite/examples/bert_qa/android/EXPLORE_THE_CODE.md diff --git a/lite/examples/bert_qa/android/EXPLORE_THE_CODE.md b/lite/examples/bert_qa/android/EXPLORE_THE_CODE.md new file mode 100644 index 00000000000..e69de29bb2d diff --git a/lite/examples/bert_qa/android/README.md b/lite/examples/bert_qa/android/README.md index 8c58e784012..947a24947b7 100644 --- a/lite/examples/bert_qa/android/README.md +++ b/lite/examples/bert_qa/android/README.md @@ -10,7 +10,8 @@ TensorFlow 2.0, and tested on [SQuAD] dataset version 1.1. The demo app provides 48 passages from the dataset for users to choose from, and gives 5 most possible answers corresponding to the input passage and query. -These instructions walk you through running the demo on an Android device. +These instructions walk you through running the demo on an Android device. For an explanation of the source, see +[TensorFlow Lite BERT QA Android example](EXPLORE_THE_CODE.md). ### Model used From 02bf4aae12143e44a449ad82be27639b3da603c3 Mon Sep 17 00:00:00 2001 From: Sunit Roy <2703iamsry@gmail.com> Date: Sat, 24 Jul 2021 02:18:29 +0530 Subject: [PATCH 4/7] Updated project to use separated modules for Task Lib & Interpreter --- .../examples/bert_qa/android/app/build.gradle | 25 +- .../lite/examples/bertqa/UnitTest.java | 44 --- .../android/app/src/main/AndroidManifest.xml | 8 +- .../bertqa/{ui => }/DatasetListActivity.java | 8 +- .../bertqa/{ml => }/LoadDatasetClient.java | 4 +- .../examples/bertqa/{ui => }/QaActivity.java | 25 +- .../bertqa/{ui => }/QuestionAdapter.java | 4 +- .../layout/tfe_qa_activity_dataset_list.xml | 2 +- .../main/res/layout/tfe_qa_activity_qa.xml | 2 +- .../android/lib_interpreter/.gitignore | 1 + .../android/lib_interpreter/build.gradle | 49 +++ .../lib_interpreter/consumer-rules.pro | 0 .../{app => lib_interpreter}/download.gradle | 0 .../lib_interpreter/proguard-rules.pro | 21 ++ .../lite/examples/bertqa/ml/QaClientTest.java | 66 +++++ .../tokenization/BasicTokenizerTest.java | 95 ++++++ .../tokenization/FullTokenizerTest.java | 70 +++++ .../tokenization/WordpieceTokenizerTest.java | 56 ++++ .../src/main/AndroidManifest.xml | 5 + .../lib_interpreter/src/main/assets/qa.json | 1 + .../lite/examples/bertqa/ml/Answer.java | 39 +++ .../lite/examples/bertqa/ml/Feature.java | 42 +++ .../examples/bertqa/ml/FeatureConverter.java | 106 +++++++ .../examples/bertqa/ml/LoadDatasetClient.java | 89 ++++++ .../lite/examples/bertqa/ml/ModelHelper.java | 86 ++++++ .../lite/examples/bertqa/ml/QaClient.java | 280 ++++++++++++++++++ .../bertqa/tokenization/BasicTokenizer.java | 105 +++++++ .../bertqa/tokenization/CharChecker.java | 58 ++++ .../bertqa/tokenization/FullTokenizer.java | 52 ++++ .../tokenization/WordpieceTokenizer.java | 94 ++++++ .../bert_qa/android/lib_task_api/.gitignore | 1 + .../bert_qa/android/lib_task_api/build.gradle | 52 ++++ .../android/lib_task_api/consumer-rules.pro | 0 .../android/lib_task_api/download.gradle | 15 + .../android/lib_task_api/proguard-rules.pro | 21 ++ .../lite/examples/bertqa/QaClientTest.java | 68 +++++ .../lib_task_api/src/main/AndroidManifest.xml | 5 + .../lib_task_api/src/main/assets/qa.json | 1 + .../lite/examples/bertqa/ml/Answer.java | 39 +++ .../lite/examples/bertqa/ml/QaClient.java | 13 +- lite/examples/bert_qa/android/settings.gradle | 2 + 41 files changed, 1565 insertions(+), 89 deletions(-) delete mode 100644 lite/examples/bert_qa/android/app/src/androidTest/java/org/tensorflow/lite/examples/bertqa/UnitTest.java rename lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/{ui => }/DatasetListActivity.java (92%) rename lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/{ml => }/LoadDatasetClient.java (96%) rename lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/{ui => }/QaActivity.java (96%) rename lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/{ui => }/QuestionAdapter.java (96%) create mode 100644 lite/examples/bert_qa/android/lib_interpreter/.gitignore create mode 100644 lite/examples/bert_qa/android/lib_interpreter/build.gradle create mode 100644 lite/examples/bert_qa/android/lib_interpreter/consumer-rules.pro rename lite/examples/bert_qa/android/{app => lib_interpreter}/download.gradle (100%) create mode 100644 lite/examples/bert_qa/android/lib_interpreter/proguard-rules.pro create mode 100644 lite/examples/bert_qa/android/lib_interpreter/src/androidTest/java/org/tensorflow/lite/examples/bertqa/ml/QaClientTest.java create mode 100644 lite/examples/bert_qa/android/lib_interpreter/src/androidTest/java/org/tensorflow/lite/examples/bertqa/tokenization/BasicTokenizerTest.java create mode 100644 lite/examples/bert_qa/android/lib_interpreter/src/androidTest/java/org/tensorflow/lite/examples/bertqa/tokenization/FullTokenizerTest.java create mode 100644 lite/examples/bert_qa/android/lib_interpreter/src/androidTest/java/org/tensorflow/lite/examples/bertqa/tokenization/WordpieceTokenizerTest.java create mode 100644 lite/examples/bert_qa/android/lib_interpreter/src/main/AndroidManifest.xml create mode 100644 lite/examples/bert_qa/android/lib_interpreter/src/main/assets/qa.json create mode 100644 lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/ml/Answer.java create mode 100644 lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/ml/Feature.java create mode 100644 lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/ml/FeatureConverter.java create mode 100644 lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/ml/LoadDatasetClient.java create mode 100644 lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/ml/ModelHelper.java create mode 100644 lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/ml/QaClient.java create mode 100644 lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/tokenization/BasicTokenizer.java create mode 100644 lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/tokenization/CharChecker.java create mode 100644 lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/tokenization/FullTokenizer.java create mode 100644 lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/tokenization/WordpieceTokenizer.java create mode 100644 lite/examples/bert_qa/android/lib_task_api/.gitignore create mode 100644 lite/examples/bert_qa/android/lib_task_api/build.gradle create mode 100644 lite/examples/bert_qa/android/lib_task_api/consumer-rules.pro create mode 100644 lite/examples/bert_qa/android/lib_task_api/download.gradle create mode 100644 lite/examples/bert_qa/android/lib_task_api/proguard-rules.pro create mode 100644 lite/examples/bert_qa/android/lib_task_api/src/androidTest/java/org/tensorflow/lite/examples/bertqa/QaClientTest.java create mode 100644 lite/examples/bert_qa/android/lib_task_api/src/main/AndroidManifest.xml create mode 100644 lite/examples/bert_qa/android/lib_task_api/src/main/assets/qa.json create mode 100644 lite/examples/bert_qa/android/lib_task_api/src/main/java/org/tensorflow/lite/examples/bertqa/ml/Answer.java rename lite/examples/bert_qa/android/{app => lib_task_api}/src/main/java/org/tensorflow/lite/examples/bertqa/ml/QaClient.java (71%) diff --git a/lite/examples/bert_qa/android/app/build.gradle b/lite/examples/bert_qa/android/app/build.gradle index dce99defa82..ae563af28a0 100644 --- a/lite/examples/bert_qa/android/app/build.gradle +++ b/lite/examples/bert_qa/android/app/build.gradle @@ -34,28 +34,33 @@ android { lintOptions { abortOnError false } -} -// App assets folder: -project.ext.ASSET_DIR = projectDir.toString() + '/src/main/assets/' + flavorDimensions "tfliteInference" + productFlavors { + // The TFLite inference is built using the TFLite Java interpreter. + interpreter { + dimension "tfliteInference" + } + // Default: The TFLite inference is built using the TFLite Task library (high-level API). + taskApi { + getIsDefault().set(true) + dimension "tfliteInference" + } + } +} -// Download TF Lite model. -apply from: 'download.gradle' dependencies { - implementation fileTree(dir: 'libs', include: ['*.jar']) + interpreterImplementation project(":lib_interpreter") + taskApiImplementation project(":lib_task_api") implementation 'androidx.appcompat:appcompat:1.1.0' implementation 'androidx.constraintlayout:constraintlayout:1.1.3' implementation 'androidx.coordinatorlayout:coordinatorlayout:1.1.0' implementation 'androidx.recyclerview:recyclerview:1.1.0' implementation 'com.google.android.material:material:1.0.0' - implementation 'com.google.code.gson:gson:2.8.5' - //Task Text Library dependency - implementation 'org.tensorflow:tensorflow-lite-task-text:0.2.0' - androidTestImplementation 'androidx.test:runner:1.2.0' androidTestImplementation 'androidx.test.espresso:espresso-core:3.2.0' } diff --git a/lite/examples/bert_qa/android/app/src/androidTest/java/org/tensorflow/lite/examples/bertqa/UnitTest.java b/lite/examples/bert_qa/android/app/src/androidTest/java/org/tensorflow/lite/examples/bertqa/UnitTest.java deleted file mode 100644 index 65bd56b10e2..00000000000 --- a/lite/examples/bert_qa/android/app/src/androidTest/java/org/tensorflow/lite/examples/bertqa/UnitTest.java +++ /dev/null @@ -1,44 +0,0 @@ -package org.tensorflow.lite.examples.bertqa; - -import android.content.Context; - -import androidx.test.platform.app.InstrumentationRegistry; -import androidx.test.runner.AndroidJUnit4; - -import org.junit.Before; -import org.junit.Test; -import org.junit.runner.RunWith; -import org.tensorflow.lite.examples.bertqa.ml.QaClient; -import org.tensorflow.lite.task.text.qa.QaAnswer; - -import static org.junit.Assert.*; - - -/** Tests of {@link QaClient} */ -@RunWith(AndroidJUnit4.class) -public final class UnitTest { - private QaClient client; - private String contextOfTheQuestion = "Nikola Tesla (Serbian Cyrillic: Никола Тесла; 10 July 1856 – 7 January 1943) was a Serbian American inventor, electrical engineer, mechanical engineer, physicist, and futurist best known for his contributions to the design of the modern alternating current (AC) electricity supply system."; - - @Before - public void setUp() { - // Context of the app under test. - Context appContext = InstrumentationRegistry.getInstrumentation().getTargetContext(); - - client = new QaClient(appContext); - client.loadModel(); - } - - @Test - public void loadModelTest() { - assertNotNull(client.answerer); - } - - @Test - public void predictTest() { - - QaAnswer answer = client.predict("In what year was Nikola Tesla born? ", contextOfTheQuestion).get(0); - - assertEquals("1856", answer.text); - } -} \ No newline at end of file diff --git a/lite/examples/bert_qa/android/app/src/main/AndroidManifest.xml b/lite/examples/bert_qa/android/app/src/main/AndroidManifest.xml index 68b68d37d39..c32d769dbb2 100644 --- a/lite/examples/bert_qa/android/app/src/main/AndroidManifest.xml +++ b/lite/examples/bert_qa/android/app/src/main/AndroidManifest.xml @@ -28,16 +28,16 @@ tools:ignore="GoogleAppIndexingWarning"> + android:parentActivityName=".DatasetListActivity"> + android:value=".DatasetListActivity" /> diff --git a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ui/DatasetListActivity.java b/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/DatasetListActivity.java similarity index 92% rename from lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ui/DatasetListActivity.java rename to lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/DatasetListActivity.java index eefe2537b5e..c753e985692 100644 --- a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ui/DatasetListActivity.java +++ b/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/DatasetListActivity.java @@ -12,17 +12,13 @@ See the License for the specific language governing permissions and limitations under the License. ==============================================================================*/ -package org.tensorflow.lite.examples.bertqa.ui; +package org.tensorflow.lite.examples.bertqa; import android.os.Bundle; - -import androidx.appcompat.app.AppCompatActivity; - import android.widget.ArrayAdapter; import android.widget.ListView; -import org.tensorflow.lite.examples.bertqa.R; -import org.tensorflow.lite.examples.bertqa.ml.LoadDatasetClient; +import androidx.appcompat.app.AppCompatActivity; /** * An activity representing a list of Datasets. This activity has different presentations for diff --git a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ml/LoadDatasetClient.java b/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/LoadDatasetClient.java similarity index 96% rename from lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ml/LoadDatasetClient.java rename to lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/LoadDatasetClient.java index 9e6064ad9d9..919951bda67 100644 --- a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ml/LoadDatasetClient.java +++ b/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/LoadDatasetClient.java @@ -12,7 +12,7 @@ See the License for the specific language governing permissions and limitations under the License. ==============================================================================*/ -package org.tensorflow.lite.examples.bertqa.ml; +package org.tensorflow.lite.examples.bertqa; import android.content.Context; import android.util.Log; @@ -20,13 +20,11 @@ import com.google.gson.Gson; import com.google.gson.stream.JsonReader; -import java.io.BufferedReader; import java.io.IOException; import java.io.InputStream; import java.io.InputStreamReader; import java.util.HashMap; import java.util.List; -import java.util.Map; /** * Interface to load squad dataset. Provide passages for users to choose from & provide questions diff --git a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ui/QaActivity.java b/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/QaActivity.java similarity index 96% rename from lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ui/QaActivity.java rename to lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/QaActivity.java index 64e8372f5d0..ee394a9a5a1 100644 --- a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ui/QaActivity.java +++ b/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/QaActivity.java @@ -12,7 +12,7 @@ See the License for the specific language governing permissions and limitations under the License. ==============================================================================*/ -package org.tensorflow.lite.examples.bertqa.ui; +package org.tensorflow.lite.examples.bertqa; import android.content.Context; import android.content.Intent; @@ -20,11 +20,6 @@ import android.os.Handler; import android.os.HandlerThread; import android.speech.tts.TextToSpeech; - -import androidx.appcompat.app.AppCompatActivity; -import androidx.recyclerview.widget.LinearLayoutManager; -import androidx.recyclerview.widget.RecyclerView; - import android.text.Editable; import android.text.Spannable; import android.text.SpannableString; @@ -38,17 +33,19 @@ import android.widget.ImageButton; import android.widget.TextView; +import androidx.appcompat.app.AppCompatActivity; +import androidx.recyclerview.widget.LinearLayoutManager; +import androidx.recyclerview.widget.RecyclerView; + import com.google.android.material.snackbar.Snackbar; import com.google.android.material.textfield.TextInputEditText; +import org.tensorflow.lite.examples.bertqa.ml.Answer; +import org.tensorflow.lite.examples.bertqa.ml.QaClient; + import java.util.List; import java.util.Locale; -import org.tensorflow.lite.examples.bertqa.ml.QaClient; -import org.tensorflow.lite.examples.bertqa.R; -import org.tensorflow.lite.examples.bertqa.ml.LoadDatasetClient; -import org.tensorflow.lite.task.text.qa.QaAnswer; - /** * Activity for doing Q&A on a specific dataset */ @@ -226,13 +223,13 @@ private void answerQuestion(String question) { handler.post( () -> { long beforeTime = System.currentTimeMillis(); - final List answers = qaClient.predict(questionToAsk, content); + final List answers = qaClient.predict(questionToAsk, content); long afterTime = System.currentTimeMillis(); double totalSeconds = (afterTime - beforeTime) / 1000.0; if (!answers.isEmpty()) { // Get the top answer - QaAnswer topAnswer = answers.get(0); + Answer topAnswer = answers.get(0); // Dismiss the snackbar and show the answer. runOnUiThread( () -> { @@ -250,7 +247,7 @@ private void answerQuestion(String question) { }); } - private void presentAnswer(QaAnswer answer) { + private void presentAnswer(Answer answer) { // Highlight answer. Spannable spanText = new SpannableString(content); int offset = content.indexOf(answer.text, 0); diff --git a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ui/QuestionAdapter.java b/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/QuestionAdapter.java similarity index 96% rename from lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ui/QuestionAdapter.java rename to lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/QuestionAdapter.java index 30366136a12..e53ea9f7168 100644 --- a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ui/QuestionAdapter.java +++ b/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/QuestionAdapter.java @@ -12,7 +12,7 @@ See the License for the specific language governing permissions and limitations under the License. ==============================================================================*/ -package org.tensorflow.lite.examples.bertqa.ui; +package org.tensorflow.lite.examples.bertqa; import android.content.Context; @@ -26,8 +26,6 @@ import com.google.android.material.chip.Chip; -import org.tensorflow.lite.examples.bertqa.R; - /** * Adapter class to show question suggestion chips. */ diff --git a/lite/examples/bert_qa/android/app/src/main/res/layout/tfe_qa_activity_dataset_list.xml b/lite/examples/bert_qa/android/app/src/main/res/layout/tfe_qa_activity_dataset_list.xml index e188739e97d..e8415011186 100644 --- a/lite/examples/bert_qa/android/app/src/main/res/layout/tfe_qa_activity_dataset_list.xml +++ b/lite/examples/bert_qa/android/app/src/main/res/layout/tfe_qa_activity_dataset_list.xml @@ -56,7 +56,7 @@ app:layout_constraintEnd_toEndOf="parent" app:layout_constraintStart_toStartOf="parent" app:layout_constraintTop_toTopOf="parent" - tools:context="org.tensorflow.lite.examples.bertqa.ui.DatasetListActivity"> + tools:context="org.tensorflow.lite.examples.bertqa.DatasetListActivity"> + tools:context="org.tensorflow.lite.examples.bertqa.QaActivity"> predict0 = client.predict("What is Tesla's home country?", content); + assertThat(getTexts(predict0)).contains("Serbian"); + List predict1 = client.predict("What was Nikola Tesla's ethnicity?", content); + assertThat(getTexts(predict1)).contains("Serbian"); + List predict2 = client.predict("What does AC stand for?", content); + assertThat(getTexts(predict2)).contains("alternating current"); + List predict3 = client.predict("When was Tesla born?", content); + assertThat(getTexts(predict3)).contains("10 July 1856"); + } + + private static List getTexts(List answers) { + List texts = new ArrayList<>(); + for (Answer ans : answers) { + texts.add(ans.text); + } + return texts; + } +} diff --git a/lite/examples/bert_qa/android/lib_interpreter/src/androidTest/java/org/tensorflow/lite/examples/bertqa/tokenization/BasicTokenizerTest.java b/lite/examples/bert_qa/android/lib_interpreter/src/androidTest/java/org/tensorflow/lite/examples/bertqa/tokenization/BasicTokenizerTest.java new file mode 100644 index 00000000000..8648a919562 --- /dev/null +++ b/lite/examples/bert_qa/android/lib_interpreter/src/androidTest/java/org/tensorflow/lite/examples/bertqa/tokenization/BasicTokenizerTest.java @@ -0,0 +1,95 @@ +/* Copyright 2019 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +package org.tensorflow.lite.examples.bertqa.tokenization; + +import static com.google.common.truth.Truth.assertThat; + +import androidx.test.ext.junit.runners.AndroidJUnit4; + +import org.junit.Assert; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Tests of {@link BasicTokenizer} */ +@RunWith(AndroidJUnit4.class) +public final class BasicTokenizerTest { + @Test + public void cleanTextTest() throws Exception { + String testExample = "This is an\rexample.\n"; + char testChar = 0; + char unknownChar = 0xfffd; + assertThat(BasicTokenizer.cleanText(testExample)).isEqualTo("This is an example. "); + assertThat(BasicTokenizer.cleanText(testExample + testChar)).isEqualTo("This is an example. "); + assertThat(BasicTokenizer.cleanText(testExample + unknownChar)) + .isEqualTo("This is an example. "); + + String nullString = null; + Assert.assertThrows(NullPointerException.class, () -> BasicTokenizer.cleanText(nullString)); + } + + @Test + public void whitespaceTokenizeTest() throws Exception { + assertThat(BasicTokenizer.whitespaceTokenize("Hi , This is an example. ")) + .containsExactly("Hi", ",", "This", "is", "an", "example.") + .inOrder(); + assertThat(BasicTokenizer.whitespaceTokenize(" ")).isEmpty(); + + String nullString = null; + Assert.assertThrows( + NullPointerException.class, () -> BasicTokenizer.whitespaceTokenize(nullString)); + } + + @Test + public void runSplitOnPuncTest() throws Exception { + assertThat(BasicTokenizer.runSplitOnPunc("Hi,there.")) + .containsExactly("Hi", ",", "there", ".") + .inOrder(); + assertThat(BasicTokenizer.runSplitOnPunc("I'm \"Spider-Man\"")) // Input: I'm "Spider-Man" + .containsExactly("I", "\'", "m ", "\"", "Spider", "-", "Man", "\"") + .inOrder(); + + String nullString = null; + Assert.assertThrows( + NullPointerException.class, () -> BasicTokenizer.runSplitOnPunc(nullString)); + } + + @Test + public void tokenizeWithLowerCaseTest() throws Exception { + BasicTokenizer tokenizer = new BasicTokenizer(/*doLowerCase=*/ true); + assertThat(tokenizer.tokenize(" Hi, This\tis an example.\n")) + .containsExactly("hi", ",", "this", "is", "an", "example", ".") + .inOrder(); + assertThat(tokenizer.tokenize("Hello,How are you?")) + .containsExactly("hello", ",", "how", "are", "you", "?") + .inOrder(); + + String nullString = null; + Assert.assertThrows(NullPointerException.class, () -> tokenizer.tokenize(nullString)); + } + + @Test + public void tokenizeTest() throws Exception { + BasicTokenizer tokenizer = new BasicTokenizer(/*doLowerCase=*/ false); + assertThat(tokenizer.tokenize(" Hi, This\tis an example.\n")) + .containsExactly("Hi", ",", "This", "is", "an", "example", ".") + .inOrder(); + assertThat(tokenizer.tokenize("Hello,How are you?")) + .containsExactly("Hello", ",", "How", "are", "you", "?") + .inOrder(); + + String nullString = null; + Assert.assertThrows(NullPointerException.class, () -> tokenizer.tokenize(nullString)); + } +} diff --git a/lite/examples/bert_qa/android/lib_interpreter/src/androidTest/java/org/tensorflow/lite/examples/bertqa/tokenization/FullTokenizerTest.java b/lite/examples/bert_qa/android/lib_interpreter/src/androidTest/java/org/tensorflow/lite/examples/bertqa/tokenization/FullTokenizerTest.java new file mode 100644 index 00000000000..ff0b5bb1ca6 --- /dev/null +++ b/lite/examples/bert_qa/android/lib_interpreter/src/androidTest/java/org/tensorflow/lite/examples/bertqa/tokenization/FullTokenizerTest.java @@ -0,0 +1,70 @@ +/* Copyright 2019 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +package org.tensorflow.lite.examples.bertqa.tokenization; + +import static com.google.common.truth.Truth.assertThat; + +import androidx.test.core.app.ApplicationProvider; +import androidx.test.ext.junit.runners.AndroidJUnit4; + +import org.junit.Assert; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.tensorflow.lite.examples.bertqa.ml.ModelHelper; +import org.tensorflow.lite.support.metadata.MetadataExtractor; + +import java.io.IOException; +import java.nio.ByteBuffer; +import java.util.Arrays; +import java.util.List; +import java.util.Map; + +/** Tests of {@link FullTokenizer} */ +@RunWith(AndroidJUnit4.class) +public final class FullTokenizerTest { + private Map dic; + + @Before + public void setUp() throws IOException { + ByteBuffer buffer = ModelHelper.loadModelFile(ApplicationProvider.getApplicationContext()); + MetadataExtractor metadataExtractor = new MetadataExtractor(buffer); + dic = ModelHelper.extractDictionary(metadataExtractor); + assertThat(dic).isNotNull(); + assertThat(dic).isNotEmpty(); + } + + @Test + public void tokenizeTest() throws Exception { + FullTokenizer tokenizer = new FullTokenizer(dic, /* doLowerCase= */ true); + assertThat(tokenizer.tokenize("Good morning, I'm your teacher.\n")) + .containsExactly("good", "morning", ",", "i", "'", "m", "your", "teacher", ".") + .inOrder(); + assertThat(tokenizer.tokenize("")).isEmpty(); + + String nullString = null; + Assert.assertThrows(NullPointerException.class, () -> tokenizer.tokenize(nullString)); + } + + @Test + public void convertTokensToIdsTest() throws Exception { + FullTokenizer tokenizer = new FullTokenizer(dic, /* doLowerCase= */ true); + List testExample = + Arrays.asList("good", "morning", ",", "i", "'", "m", "your", "teacher", "."); + assertThat(tokenizer.convertTokensToIds(testExample)) + .containsExactly(2204, 2851, 1010, 1045, 1005, 1049, 2115, 3836, 1012) + .inOrder(); + } +} diff --git a/lite/examples/bert_qa/android/lib_interpreter/src/androidTest/java/org/tensorflow/lite/examples/bertqa/tokenization/WordpieceTokenizerTest.java b/lite/examples/bert_qa/android/lib_interpreter/src/androidTest/java/org/tensorflow/lite/examples/bertqa/tokenization/WordpieceTokenizerTest.java new file mode 100644 index 00000000000..c05ea116c9c --- /dev/null +++ b/lite/examples/bert_qa/android/lib_interpreter/src/androidTest/java/org/tensorflow/lite/examples/bertqa/tokenization/WordpieceTokenizerTest.java @@ -0,0 +1,56 @@ +/* Copyright 2019 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +package org.tensorflow.lite.examples.bertqa.tokenization; + +import static com.google.common.truth.Truth.assertThat; + +import androidx.test.core.app.ApplicationProvider; +import androidx.test.ext.junit.runners.AndroidJUnit4; + +import org.junit.Assert; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.tensorflow.lite.examples.bertqa.ml.ModelHelper; +import org.tensorflow.lite.support.metadata.MetadataExtractor; + +import java.io.IOException; +import java.nio.ByteBuffer; +import java.util.Map; + +/** Tests of {@link WordpieceTokenizer} */ +@RunWith(AndroidJUnit4.class) +public final class WordpieceTokenizerTest { + private Map dic; + + @Before + public void setUp() throws IOException { + ByteBuffer buffer = ModelHelper.loadModelFile(ApplicationProvider.getApplicationContext()); + MetadataExtractor metadataExtractor = new MetadataExtractor(buffer); + dic = ModelHelper.extractDictionary(metadataExtractor); + assertThat(dic).isNotNull(); + assertThat(dic).isNotEmpty(); + } + + @Test + public void tokenizeTest() throws Exception { + WordpieceTokenizer tokenizer = new WordpieceTokenizer(dic); + assertThat(tokenizer.tokenize("meaningfully")).containsExactly("meaningful", "##ly").inOrder(); + assertThat(tokenizer.tokenize("teacher")).containsExactly("teacher").inOrder(); + + String nullString = null; + Assert.assertThrows(NullPointerException.class, () -> tokenizer.tokenize(nullString)); + } +} diff --git a/lite/examples/bert_qa/android/lib_interpreter/src/main/AndroidManifest.xml b/lite/examples/bert_qa/android/lib_interpreter/src/main/AndroidManifest.xml new file mode 100644 index 00000000000..22c3a8fc635 --- /dev/null +++ b/lite/examples/bert_qa/android/lib_interpreter/src/main/AndroidManifest.xml @@ -0,0 +1,5 @@ + + + + \ No newline at end of file diff --git a/lite/examples/bert_qa/android/lib_interpreter/src/main/assets/qa.json b/lite/examples/bert_qa/android/lib_interpreter/src/main/assets/qa.json new file mode 100644 index 00000000000..94b7c9c4b50 --- /dev/null +++ b/lite/examples/bert_qa/android/lib_interpreter/src/main/assets/qa.json @@ -0,0 +1 @@ +{"titles": [["Super_Bowl_50"], ["Warsaw"], ["Normans"], ["Nikola_Tesla"], ["Computational_complexity_theory"], ["Teacher"], ["Martin_Luther"], ["Southern_California"], ["Sky_(United_Kingdom)"], ["Victoria_(Australia)"], ["Huguenot"], ["Steam_engine"], ["Oxygen"], ["1973_oil_crisis"], ["Apollo_program"], ["European_Union_law"], ["Amazon_rainforest"], ["Ctenophora"], ["Fresno,_California"], ["Packet_switching"], ["Black_Death"], ["Geology"], ["Newcastle_upon_Tyne"], ["Victoria_and_Albert_Museum"], ["American_Broadcasting_Company"], ["Genghis_Khan"], ["Pharmacy"], ["Immune_system"], ["Civil_disobedience"], ["Construction"], ["Private_school"], ["Harvard_University"], ["Jacksonville,_Florida"], ["Economic_inequality"], ["Doctor_Who"], ["University_of_Chicago"], ["Yuan_dynasty"], ["Kenya"], ["Intergovernmental_Panel_on_Climate_Change"], ["Chloroplast"], ["Prime_number"], ["Rhine"], ["Scottish_Parliament"], ["Islamism"], ["Imperialism"], ["United_Methodist_Church"], ["French_and_Indian_War"], ["Force"]], "contents": [["Super Bowl 50 was an American football game to determine the champion of the National Football League (NFL) for the 2015 season. The American Football Conference (AFC) champion Denver Broncos defeated the National Football Conference (NFC) champion Carolina Panthers 24\u201310 to earn their third Super Bowl title. The game was played on February 7, 2016, at Levi's Stadium in the San Francisco Bay Area at Santa Clara, California. As this was the 50th Super Bowl, the league emphasized the \"golden anniversary\" with various gold-themed initiatives, as well as temporarily suspending the tradition of naming each Super Bowl game with Roman numerals (under which the game would have been known as \"Super Bowl L\"), so that the logo could prominently feature the Arabic numerals 50."], ["One of the most famous people born in Warsaw was Maria Sk\u0142odowska-Curie, who achieved international recognition for her research on radioactivity and was the first female recipient of the Nobel Prize. Famous musicians include W\u0142adys\u0142aw Szpilman and Fr\u00e9d\u00e9ric Chopin. Though Chopin was born in the village of \u017belazowa Wola, about 60 km (37 mi) from Warsaw, he moved to the city with his family when he was seven months old. Casimir Pulaski, a Polish general and hero of the American Revolutionary War, was born here in 1745."], ["The Normans (Norman: Nourmands; French: Normands; Latin: Normanni) were the people who in the 10th and 11th centuries gave their name to Normandy, a region in France. They were descended from Norse (\"Norman\" comes from \"Norseman\") raiders and pirates from Denmark, Iceland and Norway who, under their leader Rollo, agreed to swear fealty to King Charles III of West Francia. Through generations of assimilation and mixing with the native Frankish and Roman-Gaulish populations, their descendants would gradually merge with the Carolingian-based cultures of West Francia. The distinct cultural and ethnic identity of the Normans emerged initially in the first half of the 10th century, and it continued to evolve over the succeeding centuries."], ["Nikola Tesla (Serbian Cyrillic: \u041d\u0438\u043a\u043e\u043b\u0430 \u0422\u0435\u0441\u043b\u0430; 10 July 1856 \u2013 7 January 1943) was a Serbian American inventor, electrical engineer, mechanical engineer, physicist, and futurist best known for his contributions to the design of the modern alternating current (AC) electricity supply system."], ["Computational complexity theory is a branch of the theory of computation in theoretical computer science that focuses on classifying computational problems according to their inherent difficulty, and relating those classes to each other. A computational problem is understood to be a task that is in principle amenable to being solved by a computer, which is equivalent to stating that the problem may be solved by mechanical application of mathematical steps, such as an algorithm."], ["The role of teacher is often formal and ongoing, carried out at a school or other place of formal education. In many countries, a person who wishes to become a teacher must first obtain specified professional qualifications or credentials from a university or college. These professional qualifications may include the study of pedagogy, the science of teaching. Teachers, like other professionals, may have to continue their education after they qualify, a process known as continuing professional development. Teachers may use a lesson plan to facilitate student learning, providing a course of study which is called the curriculum."], ["Martin Luther (/\u02c8lu\u02d0\u03b8\u0259r/ or /\u02c8lu\u02d0\u00f0\u0259r/; German: [\u02c8ma\u0250\u032fti\u02d0n \u02c8l\u028at\u0250] ( listen); 10 November 1483 \u2013 18 February 1546) was a German professor of theology, composer, priest, former monk and a seminal figure in the Protestant Reformation. Luther came to reject several teachings and practices of the Late Medieval Catholic Church. He strongly disputed the claim that freedom from God's punishment for sin could be purchased with money. He proposed an academic discussion of the power and usefulness of indulgences in his Ninety-Five Theses of 1517. His refusal to retract all of his writings at the demand of Pope Leo X in 1520 and the Holy Roman Emperor Charles V at the Diet of Worms in 1521 resulted in his excommunication by the Pope and condemnation as an outlaw by the Emperor."], ["Southern California, often abbreviated SoCal, is a geographic and cultural region that generally comprises California's southernmost 10 counties. The region is traditionally described as \"eight counties\", based on demographics and economic ties: Imperial, Los Angeles, Orange, Riverside, San Bernardino, San Diego, Santa Barbara, and Ventura. The more extensive 10-county definition, including Kern and San Luis Obispo counties, is also used based on historical political divisions. Southern California is a major economic center for the state of California and the United States."], ["Formed in November 1990 by the equal merger of Sky Television and British Satellite Broadcasting, BSkyB became the UK's largest digital subscription television company. Following BSkyB's 2014 acquisition of Sky Italia and a majority 90.04% interest in Sky Deutschland in November 2014, its holding company British Sky Broadcasting Group plc changed its name to Sky plc. The United Kingdom operations also changed the company name from British Sky Broadcasting Limited to Sky UK Limited, still trading as Sky."], ["The economy of Victoria is highly diversified: service sectors including financial and property services, health, education, wholesale, retail, hospitality and manufacturing constitute the majority of employment. Victoria's total gross state product (GSP) is ranked second in Australia, although Victoria is ranked fourth in terms of GSP per capita because of its limited mining activity. Culturally, Melbourne is home to a number of museums, art galleries and theatres and is also described as the \"sporting capital of Australia\". The Melbourne Cricket Ground is the largest stadium in Australia, and the host of the 1956 Summer Olympics and the 2006 Commonwealth Games. The ground is also considered the \"spiritual home\" of Australian cricket and Australian rules football, and hosts the grand final of the Australian Football League (AFL) each year, usually drawing crowds of over 95,000 people. Victoria includes eight public universities, with the oldest, the University of Melbourne, having been founded in 1853."], ["Huguenot numbers peaked near an estimated two million by 1562, concentrated mainly in the southern and central parts of France, about one-eighth the number of French Catholics. As Huguenots gained influence and more openly displayed their faith, Catholic hostility grew, in spite of increasingly liberal political concessions and edicts of toleration from the French crown. A series of religious conflicts followed, known as the Wars of Religion, fought intermittently from 1562 to 1598. The wars finally ended with the granting of the Edict of Nantes, which granted the Huguenots substantial religious, political and military autonomy."], ["Steam engines are external combustion engines, where the working fluid is separate from the combustion products. Non-combustion heat sources such as solar power, nuclear power or geothermal energy may be used. The ideal thermodynamic cycle used to analyze this process is called the Rankine cycle. In the cycle, water is heated and transforms into steam within a boiler operating at a high pressure. When expanded through pistons or turbines, mechanical work is done. The reduced-pressure steam is then condensed and pumped back into the boiler."], ["Oxygen is a chemical element with symbol O and atomic number 8. It is a member of the chalcogen group on the periodic table and is a highly reactive nonmetal and oxidizing agent that readily forms compounds (notably oxides) with most elements. By mass, oxygen is the third-most abundant element in the universe, after hydrogen and helium. At standard temperature and pressure, two atoms of the element bind to form dioxygen, a colorless and odorless diatomic gas with the formula O\n2. Diatomic oxygen gas constitutes 20.8% of the Earth's atmosphere. However, monitoring of atmospheric oxygen levels show a global downward trend, because of fossil-fuel burning. Oxygen is the most abundant element by mass in the Earth's crust as part of oxide compounds such as silicon dioxide, making up almost half of the crust's mass."], ["The 1973 oil crisis began in October 1973 when the members of the Organization of Arab Petroleum Exporting Countries (OAPEC, consisting of the Arab members of OPEC plus Egypt and Syria) proclaimed an oil embargo. By the end of the embargo in March 1974, the price of oil had risen from US$3 per barrel to nearly $12 globally; US prices were significantly higher. The embargo caused an oil crisis, or \"shock\", with many short- and long-term effects on global politics and the global economy. It was later called the \"first oil shock\", followed by the 1979 oil crisis, termed the \"second oil shock.\""], ["The Apollo program, also known as Project Apollo, was the third United States human spaceflight program carried out by the National Aeronautics and Space Administration (NASA), which accomplished landing the first humans on the Moon from 1969 to 1972. First conceived during Dwight D. Eisenhower's administration as a three-man spacecraft to follow the one-man Project Mercury which put the first Americans in space, Apollo was later dedicated to President John F. Kennedy's national goal of \"landing a man on the Moon and returning him safely to the Earth\" by the end of the 1960s, which he proposed in a May 25, 1961, address to Congress. Project Mercury was followed by the two-man Project Gemini (1962\u201366). The first manned flight of Apollo was in 1968."], ["European Union law is a body of treaties and legislation, such as Regulations and Directives, which have direct effect or indirect effect on the laws of European Union member states. The three sources of European Union law are primary law, secondary law and supplementary law. The main sources of primary law are the Treaties establishing the European Union. Secondary sources include regulations and directives which are based on the Treaties. The legislature of the European Union is principally composed of the European Parliament and the Council of the European Union, which under the Treaties may establish secondary law to pursue the objective set out in the Treaties."], ["The Amazon rainforest (Portuguese: Floresta Amaz\u00f4nica or Amaz\u00f4nia; Spanish: Selva Amaz\u00f3nica, Amazon\u00eda or usually Amazonia; French: For\u00eat amazonienne; Dutch: Amazoneregenwoud), also known in English as Amazonia or the Amazon Jungle, is a moist broadleaf forest that covers most of the Amazon basin of South America. This basin encompasses 7,000,000 square kilometres (2,700,000 sq mi), of which 5,500,000 square kilometres (2,100,000 sq mi) are covered by the rainforest. This region includes territory belonging to nine nations. The majority of the forest is contained within Brazil, with 60% of the rainforest, followed by Peru with 13%, Colombia with 10%, and with minor amounts in Venezuela, Ecuador, Bolivia, Guyana, Suriname and French Guiana. States or departments in four nations contain \"Amazonas\" in their names. The Amazon represents over half of the planet's remaining rainforests, and comprises the largest and most biodiverse tract of tropical rainforest in the world, with an estimated 390 billion individual trees divided into 16,000 species."], ["Ctenophora (/t\u1d7b\u02c8n\u0252f\u0259r\u0259/; singular ctenophore, /\u02c8t\u025bn\u0259f\u0254\u02d0r/ or /\u02c8ti\u02d0n\u0259f\u0254\u02d0r/; from the Greek \u03ba\u03c4\u03b5\u03af\u03c2 kteis 'comb' and \u03c6\u03ad\u03c1\u03c9 pher\u014d 'carry'; commonly known as comb jellies) is a phylum of animals that live in marine waters worldwide. Their most distinctive feature is the \u2018combs\u2019 \u2013 groups of cilia which they use for swimming \u2013 they are the largest animals that swim by means of cilia. Adults of various species range from a few millimeters to 1.5 m (4 ft 11 in) in size. Like cnidarians, their bodies consist of a mass of jelly, with one layer of cells on the outside and another lining the internal cavity. In ctenophores, these layers are two cells deep, while those in cnidarians are only one cell deep. Some authors combined ctenophores and cnidarians in one phylum, Coelenterata, as both groups rely on water flow through the body cavity for both digestion and respiration. Increasing awareness of the differences persuaded more recent authors to classify them as separate phyla."], ["Fresno (/\u02c8fr\u025bzno\u028a/ FREZ-noh), the county seat of Fresno County, is a city in the U.S. state of California. As of 2015, the city's population was 520,159, making it the fifth-largest city in California, the largest inland city in California and the 34th-largest in the nation. Fresno is in the center of the San Joaquin Valley and is the largest city in the Central Valley, which contains the San Joaquin Valley. It is approximately 220 miles (350 km) northwest of Los Angeles, 170 miles (270 km) south of the state capital, Sacramento, or 185 miles (300 km) south of San Francisco. The name Fresno means \"ash tree\" in Spanish, and an ash leaf is featured on the city's flag."], ["Starting in the late 1950s, American computer scientist Paul Baran developed the concept Distributed Adaptive Message Block Switching with the goal to provide a fault-tolerant, efficient routing method for telecommunication messages as part of a research program at the RAND Corporation, funded by the US Department of Defense. This concept contrasted and contradicted the theretofore established principles of pre-allocation of network bandwidth, largely fortified by the development of telecommunications in the Bell System. The new concept found little resonance among network implementers until the independent work of Donald Davies at the National Physical Laboratory (United Kingdom) (NPL) in the late 1960s. Davies is credited with coining the modern name packet switching and inspiring numerous packet switching networks in Europe in the decade following, including the incorporation of the concept in the early ARPANET in the United States."], ["The Black Death is thought to have originated in the arid plains of Central Asia, where it then travelled along the Silk Road, reaching Crimea by 1343. From there, it was most likely carried by Oriental rat fleas living on the black rats that were regular passengers on merchant ships. Spreading throughout the Mediterranean and Europe, the Black Death is estimated to have killed 30\u201360% of Europe's total population. In total, the plague reduced the world population from an estimated 450 million down to 350\u2013375 million in the 14th century. The world population as a whole did not recover to pre-plague levels until the 17th century. The plague recurred occasionally in Europe until the 19th century."], ["There are three major types of rock: igneous, sedimentary, and metamorphic. The rock cycle is an important concept in geology which illustrates the relationships between these three types of rock, and magma. When a rock crystallizes from melt (magma and/or lava), it is an igneous rock. This rock can be weathered and eroded, and then redeposited and lithified into a sedimentary rock, or be turned into a metamorphic rock due to heat and pressure that change the mineral content of the rock which gives it a characteristic fabric. The sedimentary rock can then be subsequently turned into a metamorphic rock due to heat and pressure and is then weathered, eroded, deposited, and lithified, ultimately becoming a sedimentary rock. Sedimentary rock may also be re-eroded and redeposited, and metamorphic rock may also undergo additional metamorphism. All three types of rocks may be re-melted; when this happens, a new magma is formed, from which an igneous rock may once again crystallize."], ["Newcastle upon Tyne (RP: i/\u02ccnju\u02d0k\u0251\u02d0s\u0259l \u0259\u02ccp\u0252n \u02c8ta\u026an/; Locally: i/nju\u02d0\u02cck\u00e6s\u0259l \u0259\u02ccp\u0259n \u02c8ta\u026an/), commonly known as Newcastle, is a city in Tyne and Wear, North East England, 103 miles (166 km) south of Edinburgh and 277 miles (446 km) north of London on the northern bank of the River Tyne, 8.5 mi (13.7 km) from the North Sea. Newcastle is the most populous city in the North East and Tyneside the eighth most populous conurbation in the United Kingdom. Newcastle is a member of the English Core Cities Group and is a member of the Eurocities network of European cities. Newcastle was part of the county of Northumberland until 1400, when it became a county itself, a status it retained until becoming part of Tyne and Wear in 1974.[not in citation given] The regional nickname and dialect for people from Newcastle and the surrounding area is Geordie."], ["The Victoria and Albert Museum (often abbreviated as the V&A), London, is the world's largest museum of decorative arts and design, housing a permanent collection of over 4.5 million objects. It was founded in 1852 and named after Queen Victoria and Prince Albert. The V&A is located in the Brompton district of the Royal Borough of Kensington and Chelsea, in an area that has become known as \"Albertopolis\" because of its association with Prince Albert, the Albert Memorial and the major cultural institutions with which he was associated. These include the Natural History Museum, the Science Museum and the Royal Albert Hall. The museum is a non-departmental public body sponsored by the Department for Culture, Media and Sport. Like other national British museums, entrance to the museum has been free since 2001."], ["The American Broadcasting Company (ABC) (stylized in its logo as abc since 1957) is an American commercial broadcast television network that is owned by the Disney\u2013ABC Television Group, a subsidiary of Disney Media Networks division of The Walt Disney Company. The network is part of the Big Three television networks. The network is headquartered on Columbus Avenue and West 66th Street in Manhattan, with additional major offices and production facilities in New York City, Los Angeles and Burbank, California."], ["He came to power by uniting many of the nomadic tribes of Northeast Asia. After founding the Mongol Empire and being proclaimed \"Genghis Khan\", he started the Mongol invasions that resulted in the conquest of most of Eurasia. These included raids or invasions of the Qara Khitai, Caucasus, Khwarezmid Empire, Western Xia and Jin dynasties. These campaigns were often accompanied by wholesale massacres of the civilian populations \u2013 especially in the Khwarezmian and Xia controlled lands. By the end of his life, the Mongol Empire occupied a substantial portion of Central Asia and China."], ["The word pharmacy is derived from its root word pharma which was a term used since the 15th\u201317th centuries. However, the original Greek roots from pharmakos imply sorcery or even poison. In addition to pharma responsibilities, the pharma offered general medical advice and a range of services that are now performed solely by other specialist practitioners, such as surgery and midwifery. The pharma (as it was referred to) often operated through a retail shop which, in addition to ingredients for medicines, sold tobacco and patent medicines. Often the place that did this was called an apothecary and several languages have this as the dominant term, though their practices are more akin to a modern pharmacy, in English the term apothecary would today be seen as outdated or only approproriate if herbal remedies were on offer to a large extent. The pharmas also used many other herbs not listed. The Greek word Pharmakeia (Greek: \u03c6\u03b1\u03c1\u03bc\u03b1\u03ba\u03b5\u03af\u03b1) derives from pharmakon (\u03c6\u03ac\u03c1\u03bc\u03b1\u03ba\u03bf\u03bd), meaning \"drug\", \"medicine\" (or \"poison\").[n 1]"], ["The immune system is a system of many biological structures and processes within an organism that protects against disease. To function properly, an immune system must detect a wide variety of agents, known as pathogens, from viruses to parasitic worms, and distinguish them from the organism's own healthy tissue. In many species, the immune system can be classified into subsystems, such as the innate immune system versus the adaptive immune system, or humoral immunity versus cell-mediated immunity. In humans, the blood\u2013brain barrier, blood\u2013cerebrospinal fluid barrier, and similar fluid\u2013brain barriers separate the peripheral immune system from the neuroimmune system which protects the brain."], ["One of its earliest massive implementations was brought about by Egyptians against the British occupation in the 1919 Revolution. Civil disobedience is one of the many ways people have rebelled against what they deem to be unfair laws. It has been used in many nonviolent resistance movements in India (Gandhi's campaigns for independence from the British Empire), in Czechoslovakia's Velvet Revolution and in East Germany to oust their communist governments, In South Africa in the fight against apartheid, in the American Civil Rights Movement, in the Singing Revolution to bring independence to the Baltic countries from the Soviet Union, recently with the 2003 Rose Revolution in Georgia and the 2004 Orange Revolution in Ukraine, among other various movements worldwide."], ["Construction is the process of constructing a building or infrastructure. Construction differs from manufacturing in that manufacturing typically involves mass production of similar items without a designated purchaser, while construction typically takes place on location for a known client. Construction as an industry comprises six to nine percent of the gross domestic product of developed countries. Construction starts with planning,[citation needed] design, and financing and continues until the project is built and ready for use."], ["Private schools, also known as independent schools, non-governmental, or nonstate schools, are not administered by local, state or national governments; thus, they retain the right to select their students and are funded in whole or in part by charging their students tuition, rather than relying on mandatory taxation through public (government) funding; at some private schools students may be able to get a scholarship, which makes the cost cheaper, depending on a talent the student may have (e.g. sport scholarship, art scholarship, academic scholarship), financial need, or tax credit scholarships that might be available."], ["Established originally by the Massachusetts legislature and soon thereafter named for John Harvard (its first benefactor), Harvard is the United States' oldest institution of higher learning, and the Harvard Corporation (formally, the President and Fellows of Harvard College) is its first chartered corporation. Although never formally affiliated with any denomination, the early College primarily trained Congregationalist and Unitarian clergy. Its curriculum and student body were gradually secularized during the 18th century, and by the 19th century Harvard had emerged as the central cultural establishment among Boston elites. Following the American Civil War, President Charles W. Eliot's long tenure (1869\u20131909) transformed the college and affiliated professional schools into a modern research university; Harvard was a founding member of the Association of American Universities in 1900. James Bryant Conant led the university through the Great Depression and World War II and began to reform the curriculum and liberalize admissions after the war. The undergraduate college became coeducational after its 1977 merger with Radcliffe College."], ["Jacksonville is the largest city by population in the U.S. state of Florida, and the largest city by area in the contiguous United States. It is the county seat of Duval County, with which the city government consolidated in 1968. Consolidation gave Jacksonville its great size and placed most of its metropolitan population within the city limits; with an estimated population of 853,382 in 2014, it is the most populous city proper in Florida and the Southeast, and the 12th most populous in the United States. Jacksonville is the principal city in the Jacksonville metropolitan area, with a population of 1,345,596 in 2010."], ["A study by the World Institute for Development Economics Research at United Nations University reports that the richest 1% of adults alone owned 40% of global assets in the year 2000. The three richest people in the world possess more financial assets than the lowest 48 nations combined. The combined wealth of the \"10 million dollar millionaires\" grew to nearly $41 trillion in 2008. A January 2014 report by Oxfam claims that the 85 wealthiest individuals in the world have a combined wealth equal to that of the bottom 50% of the world's population, or about 3.5 billion people. According to a Los Angeles Times analysis of the report, the wealthiest 1% owns 46% of the world's wealth; the 85 richest people, a small part of the wealthiest 1%, own about 0.7% of the human population's wealth, which is the same as the bottom half of the population. More recently, in January 2015, Oxfam reported that the wealthiest 1 percent will own more than half of the global wealth by 2016. An October 2014 study by Credit Suisse also claims that the top 1% now own nearly half of the world's wealth and that the accelerating disparity could trigger a recession. In October 2015, Credit Suisse published a study which shows global inequality continues to increase, and that half of the world's wealth is now in the hands of those in the top percentile, whose assets each exceed $759,900. A 2016 report by Oxfam claims that the 62 wealthiest individuals own as much wealth as the poorer half of the global population combined. Oxfam's claims have however been questioned on the basis of the methodology used: by using net wealth (adding up assets and subtracting debts), the Oxfam report, for instance, finds that there are more poor people in the United States and Western Europe than in China (due to a greater tendency to take on debts).[unreliable source?][unreliable source?] Anthony Shorrocks, the lead author of the Credit Suisse report which is one of the sources of Oxfam's data, considers the criticism about debt to be a \"silly argument\" and \"a non-issue . . . a diversion.\""], ["Doctor Who is a British science-fiction television programme produced by the BBC since 1963. The programme depicts the adventures of the Doctor, a Time Lord\u2014a space and time-travelling humanoid alien. He explores the universe in his TARDIS, a sentient time-travelling space ship. Its exterior appears as a blue British police box, which was a common sight in Britain in 1963 when the series first aired. Accompanied by companions, the Doctor combats a variety of foes, while working to save civilisations and help people in need."], ["The University of Chicago (UChicago, Chicago, or U of C) is a private research university in Chicago. The university, established in 1890, consists of The College, various graduate programs, interdisciplinary committees organized into four academic research divisions and seven professional schools. Beyond the arts and sciences, Chicago is also well known for its professional schools, which include the Pritzker School of Medicine, the University of Chicago Booth School of Business, the Law School, the School of Social Service Administration, the Harris School of Public Policy Studies, the Graham School of Continuing Liberal and Professional Studies and the Divinity School. The university currently enrolls approximately 5,000 students in the College and around 15,000 students overall."], ["The Yuan dynasty (Chinese: \u5143\u671d; pinyin: Yu\u00e1n Ch\u00e1o), officially the Great Yuan (Chinese: \u5927\u5143; pinyin: D\u00e0 Yu\u00e1n; Mongolian: Yehe Yuan Ulus[a]), was the empire or ruling dynasty of China established by Kublai Khan, leader of the Mongolian Borjigin clan. Although the Mongols had ruled territories including today's North China for decades, it was not until 1271 that Kublai Khan officially proclaimed the dynasty in the traditional Chinese style. His realm was, by this point, isolated from the other khanates and controlled most of present-day China and its surrounding areas, including modern Mongolia and Korea. It was the first foreign dynasty to rule all of China and lasted until 1368, after which its Genghisid rulers returned to their Mongolian homeland and continued to rule the Northern Yuan dynasty. Some of the Mongolian Emperors of the Yuan mastered the Chinese language, while others only used their native language (i.e. Mongolian) and the 'Phags-pa script."], ["Kenya (/\u02c8k\u025bnj\u0259/; locally [\u02c8k\u025b\u0272a] ( listen)), officially the Republic of Kenya, is a country in Africa and a founding member of the East African Community (EAC). Its capital and largest city is Nairobi. Kenya's territory lies on the equator and overlies the East African Rift covering a diverse and expansive terrain that extends roughly from Lake Victoria to Lake Turkana (formerly called Lake Rudolf) and further south-east to the Indian Ocean. It is bordered by Tanzania to the south, Uganda to the west, South Sudan to the north-west, Ethiopia to the north and Somalia to the north-east. Kenya covers 581,309 km2 (224,445 sq mi), and had a population of approximately 45 million people in July 2014."], ["The Intergovernmental Panel on Climate Change (IPCC) is a scientific intergovernmental body under the auspices of the United Nations, set up at the request of member governments. It was first established in 1988 by two United Nations organizations, the World Meteorological Organization (WMO) and the United Nations Environment Programme (UNEP), and later endorsed by the United Nations General Assembly through Resolution 43/53. Membership of the IPCC is open to all members of the WMO and UNEP. The IPCC produces reports that support the United Nations Framework Convention on Climate Change (UNFCCC), which is the main international treaty on climate change. The ultimate objective of the UNFCCC is to \"stabilize greenhouse gas concentrations in the atmosphere at a level that would prevent dangerous anthropogenic [i.e., human-induced] interference with the climate system\". IPCC reports cover \"the scientific, technical and socio-economic information relevant to understanding the scientific basis of risk of human-induced climate change, its potential impacts and options for adaptation and mitigation.\""], ["Chloroplasts' main role is to conduct photosynthesis, where the photosynthetic pigment chlorophyll captures the energy from sunlight and converts it and stores it in the energy-storage molecules ATP and NADPH while freeing oxygen from water. They then use the ATP and NADPH to make organic molecules from carbon dioxide in a process known as the Calvin cycle. Chloroplasts carry out a number of other functions, including fatty acid synthesis, much amino acid synthesis, and the immune response in plants. The number of chloroplasts per cell varies from 1 in algae up to 100 in plants like Arabidopsis and wheat."], ["A prime number (or a prime) is a natural number greater than 1 that has no positive divisors other than 1 and itself. A natural number greater than 1 that is not a prime number is called a composite number. For example, 5 is prime because 1 and 5 are its only positive integer factors, whereas 6 is composite because it has the divisors 2 and 3 in addition to 1 and 6. The fundamental theorem of arithmetic establishes the central role of primes in number theory: any integer greater than 1 can be expressed as a product of primes that is unique up to ordering. The uniqueness in this theorem requires excluding 1 as a prime because one can include arbitrarily many instances of 1 in any factorization, e.g., 3, 1 \u00b7 3, 1 \u00b7 1 \u00b7 3, etc. are all valid factorizations of 3."], ["The Rhine (Romansh: Rein, German: Rhein, French: le Rhin, Dutch: Rijn) is a European river that begins in the Swiss canton of Graub\u00fcnden in the southeastern Swiss Alps, forms part of the Swiss-Austrian, Swiss-Liechtenstein border, Swiss-German and then the Franco-German border, then flows through the Rhineland and eventually empties into the North Sea in the Netherlands. The biggest city on the river Rhine is Cologne, Germany with a population of more than 1,050,000 people. It is the second-longest river in Central and Western Europe (after the Danube), at about 1,230 km (760 mi),[note 2][note 1] with an average discharge of about 2,900 m3/s (100,000 cu ft/s)."], ["Following a referendum in 1997, in which the Scottish electorate voted for devolution, the current Parliament was convened by the Scotland Act 1998, which sets out its powers as a devolved legislature. The Act delineates the legislative competence of the Parliament \u2013 the areas in which it can make laws \u2013 by explicitly specifying powers that are \"reserved\" to the Parliament of the United Kingdom. The Scottish Parliament has the power to legislate in all areas that are not explicitly reserved to Westminster. The British Parliament retains the ability to amend the terms of reference of the Scottish Parliament, and can extend or reduce the areas in which it can make laws. The first meeting of the new Parliament took place on 12 May 1999."], ["Islamism, also known as Political Islam (Arabic: \u0625\u0633\u0644\u0627\u0645 \u0633\u064a\u0627\u0633\u064a\u200e isl\u0101m siy\u0101s\u012b), is an Islamic revival movement often characterized by moral conservatism, literalism, and the attempt \"to implement Islamic values in all spheres of life.\" Islamism favors the reordering of government and society in accordance with the Shari'a. The different Islamist movements have been described as \"oscillating between two poles\": at one end is a strategy of Islamization of society through state power seized by revolution or invasion; at the other \"reformist\" pole Islamists work to Islamize society gradually \"from the bottom up\". The movements have \"arguably altered the Middle East more than any trend since the modern states gained independence\", redefining \"politics and even borders\" according to one journalist (Robin Wright)."], ["Imperialism is a type of advocacy of empire. Its name originated from the Latin word \"imperium\", which means to rule over large territories. Imperialism is \"a policy of extending a country's power and influence through colonization, use of military force, or other means\". Imperialism has greatly shaped the contemporary world. It has also allowed for the rapid spread of technologies and ideas. The term imperialism has been applied to Western (and Japanese) political and economic dominance especially in Asia and Africa in the 19th and 20th centuries. Its precise meaning continues to be debated by scholars. Some writers, such as Edward Said, use the term more broadly to describe any system of domination and subordination organised with an imperial center and a periphery."], ["The United Methodist Church (UMC) is a mainline Protestant Methodist denomination. In the 19th century its main predecessor was a leader in Evangelicalism. Founded in 1968 by the union of the Methodist Church (USA) and the Evangelical United Brethren Church, the UMC traces its roots back to the revival movement of John and Charles Wesley in England as well as the Great Awakening in the United States. As such, the church's theological orientation is decidedly Wesleyan. It embraces both liturgical and evangelical elements."], ["The French and Indian War (1754\u20131763) was the North American theater of the worldwide Seven Years' War. The war was fought between the colonies of British America and New France, with both sides supported by military units from their parent countries of Great Britain and France, as well as Native American allies. At the start of the war, the French North American colonies had a population of roughly 60,000 European settlers, compared with 2 million in the British North American colonies. The outnumbered French particularly depended on the Indians. Long in conflict, the metropole nations declared war on each other in 1756, escalating the war from a regional affair into an intercontinental conflict."], ["Philosophers in antiquity used the concept of force in the study of stationary and moving objects and simple machines, but thinkers such as Aristotle and Archimedes retained fundamental errors in understanding force. In part this was due to an incomplete understanding of the sometimes non-obvious force of friction, and a consequently inadequate view of the nature of natural motion. A fundamental error was the belief that a force is required to maintain motion, even at a constant velocity. Most of the previous misunderstandings about motion and force were eventually corrected by Galileo Galilei and Sir Isaac Newton. With his mathematical insight, Sir Isaac Newton formulated laws of motion that were not improved-on for nearly three hundred years. By the early 20th century, Einstein developed a theory of relativity that correctly predicted the action of forces on objects with increasing momenta near the speed of light, and also provided insight into the forces produced by gravitation and inertia."]], "questions": [["Which NFL team represented the AFC at Super Bowl 50?", "Which NFL team represented the NFC at Super Bowl 50?", "Where did Super Bowl 50 take place?", "Which NFL team won Super Bowl 50?", "What color was used to emphasize the 50th anniversary of the Super Bowl?", "What was the theme of Super Bowl 50?", "What day was the game played on?", "What is the AFC short for?", "What was the theme of Super Bowl 50?", "What does AFC stand for?", "What day was the Super Bowl played on?", "Who won Super Bowl 50?", "What venue did Super Bowl 50 take place in?", "What city did Super Bowl 50 take place in?", "If Roman numerals were used, what would Super Bowl 50 have been called?", "Super Bowl 50 decided the NFL champion for what season?", "What year did the Denver Broncos secure a Super Bowl title for the third time?", "What city did Super Bowl 50 take place in?", "What stadium did Super Bowl 50 take place in?", "What was the final score of Super Bowl 50? ", "What month, day and year did Super Bowl 50 take place? ", "What year was Super Bowl 50?", "What team was the AFC champion?", "What team was the NFC champion?", "Who won Super Bowl 50?", "Super Bowl 50 determined the NFL champion for what season?", "Which team won Super Bowl 50.", "Where was Super Bowl 50 held?", "The name of the NFL championship game is?", "What 2015 NFL team one the AFC playoff?"], ["What was Maria Curie the first female recipient of?", "What year was Casimir Pulaski born in Warsaw?", "Who was one of the most famous people born in Warsaw?", "Who was Fr\u00e9d\u00e9ric Chopin?", "How old was Chopin when he moved to Warsaw with his family?"], ["In what country is Normandy located?", "When were the Normans in Normandy?", "From which countries did the Norse originate?", "Who was the Norse leader?", "What century did the Normans first gain their separate identity?"], ["In what year was Nikola Tesla born? ", "What was Nikola Tesla's ethnicity?", "In what year did Tesla die? ", "When was Nikola Tesla born?", "In what year did Tesla die?", "What is Tesla's home country?", "What does AC stand for?"], ["What branch of theoretical computer science deals with broadly classifying computational problems by difficulty and class of relationship?", "By what main attribute are computational problems classified utilizing computational complexity theory? ", "What is the term for a task that generally lends itself to being solved by a computer?"], ["What is a course of study called?", "What is another name to describe the science of teaching?", "Where do most teachers get their credentials from?", "What can a teacher use to help students learn?", "Where is a teacher most likely to be teaching at?"], ["Of what nationality was Martin Luther?", "When did Martin Luther die?", "What organization's teaching did Luther reject?", "What did the Church claim could be avoided with money?", "What did the Church do when Luther refused to retract his writings?"], ["What is Southern California often abbreviated as?", "Despite being traditionall described as \"eight counties\", how many counties does this region actually have?", "What is a major importance of Southern California in relation to California and the United States?", "What are the ties that best described what the \"eight counties\" are based on?", "The reasons for the las two counties to be added are based on what?"], ["What company was formed by the merger of Sky Television and British Satellite Broadcasting?", "Who is the UK's largest digital subscription television company?", "What year did BSkyB acquire Sky Italia?", "What is the name of the holding company for BSkyB?", "What is the name of the United Kingdom operation for BSkyB?"], ["What kind of economy does Victoria have?", "Where according to gross state product does Victoria rank in Australia?", "At what rank does GPS per capita set Victoria?", "What city in Victoria is called the sporting capital of Australia?", "What is the largest stadium in Australia?"], ["Where was France's Huguenot population largely centered?", "What was the proportion of Huguenots to Catholics at their peak?", "When were the Wars of Religion fought?", "What treaty ended the Wars of Religion?", "What did this agreement do?"], ["Along with geothermal and nuclear, what is a notable non-combustion heat source?", "What ideal thermodynamic cycle analyzes the process by which steam engines work?", "In the Rankine cycle, what does water turn into when heated?", "At what pressure is water heated in the Rankine cycle?", "What types of engines are steam engines?"], ["The atomic number of the periodic table for oxygen?", "What is the second most abundant element?", "How many atoms combine to form dioxygen?", "Roughly, how much oxygen makes up the Earth crust?", "Which gas makes up 20.8% of the Earth's atmosphere?", "How much of the earth's atmosphere is diatomic oxygen?", "What element makes up almost half of the earth's crust by mass?", "What is the atomic number for oxygen?", "Are atmospheric oxygen levels going up, down, or staying the same?", "What are the three most abundent elements of the universe by mass?", "What is the atomic number of the element oxygen?", "Of what group in the periodic table is oxygen a member?", "What type of compounds does oxygen most commonly form?", "Compared to other elements, how abundant does oxygen rank?", "Under normal conditions, what do two atoms of oxygen form?"], ["When did the 1973 oil crisis begin?", "What was the price of oil in March of 1974?", "When was the second oil crisis?", "What was another term used for the oil crisis?", "Who proclaimed the oil embargo?"], ["What project put the first Americans into space?", "What program was created to carry out these projects and missions?", "What year did the first manned Apollo flight occur?", "What President is credited with the original notion of putting Americans in space?", "How many people were on the project that followed Project Mercury?"], ["What are the three sources of European Union law?", "What is European Union Law?", "What are the main sources of primary law?", "What are the secondary sources of primary law?", "What are the two bodies that make up the European Union's legislature?", "What is European Union law?", "What effect does European Union law have on laws of member states?", "What are the three sources of European Union law?", "What are the main legislative bodies of the European Union?", "What are the three main sources of European Union law?", "What are the main sources of primary law?", "What is the legislature of the European Union comprised of?", "How many sources of European Union law are there?"], ["Which name is also used to describe the Amazon rainforest in English?", "How many square kilometers of rainforest is covered in the basin?", "How many nations control this region in total?", "How many nations contain \"Amazonas\" in their names?", "What percentage does the Amazon represents in rainforests on the planet?", "What is the Dutch word for the Amazon rainforest?", "What rainforest covers the majority of the Amazon basin in South America?", "In what country can most of the Amazon rainforest be found?", "The Amazon rainforest makes up what amount of Earth's rainforests?", "How many species of trees can be found in the Amazon rainforest?", "What kind of forest is the Amazon rainforest?", "How many square kilometers is the Amazon Basin?", "How many nations are within the Amazon Basin?", "Which nation contains the majority of the amazon forest?", "What is the estimate for the amount of tree species in the amazon tropical rain forest?"], ["What are Ctenophora commonly known as?", "Where do Ctenophora live?", "What size are adult Ctenophora?", "What is a ctenophora?", "What does the ctenophora use to swim?", "What does ctenophora use for digestion and respiration?", "How large can ctenophora grow?", "What is the most distinctive feature of ctenophora?", "What are ctenophora commonly known as?", "How big can ctenophora grow?", "What does ctenophora rely on for digestion and respiration?", "What does ctenophora mean in Greek?", "Where do ctenophora live?"], ["Which city is the fifth-largest city in California?", "How far is Fresno from Los Angeles?", "What does the name Fresno mean in Spanish?", "What is featured on the city of Fresno's city flag?", "How do you pronounce Fresno?"], ["What did Paul Baran develop ", "What did Distributed Adaptive Message Block Switching do", "What did this concept contradict ", "What is Donald Davies credited with ", "What did Paul Baran develop in the late 1950's", "What was the goal of the system ", "Who is credited with the modern name for this system "], ["Where did the black death originate?", "How did the black death make it to the Mediterranean and Europe?", "How much of the European population did the black death kill?", "When did the world's population finally recover from the black death?", "For how long did the plague stick around?"], ["An igneous rock is a rock that crystallizes from what?", "Sedimentary rock can be turned into which of the three types of rock?", "When the three types of rock are re-melted what is formed? ", "What are the three major types of rock? ", "What changes the mineral content of a rock?"], ["How many miles south of Edinburgh is Newcastle?", "How many miles from the north Sea is Newcastle?", "What network is Newcastle a member of?", "What county was Newcastle a part of until 1400?", "What's the regional nickname for Newcastle and its surrounding area?"], ["where is the Victoria and Albert Museum located?", "how many permanent objects are located there?", "when was the Victoria and Albert museum founded?", "Who is the museum named for?", "In which London borough is the Victoria and Albert Museum located?", "In what year was the Victoria and Albert Museum founded?", "Which monarchs was the Victoria and Albert Museum named after?", "Which department sponsors the Victoria and Albert Museum?", "In which year did the museum started charging free admission fees?"], ["What company owns the American Broadcasting Company?", "In what year did ABC stylize it's logo as abc?", "In what borough of New York City is ABC headquartered?", "On what streets is the ABC headquarters located", "Disney-ABC Television Group is a subsidiary of what division of the Walt Disney Company?"], ["What do we call the empire that Genghis Khan founded?", "Who did Genghis Khan unite before he began conquering the rest of Eurasia?", "In which regions in particular did Genghis Khan's armies massacre civilians?", "What areas did Genghis Khan control at the end of his life?", "Which other empires or dynasties did Genghis Khan conquer?"], ["What word is the word pharmacy taken from?", "What goods were sold in a pharma?", "What did the Greek root pharmakos imply?", "How would the word apothecary be viewed by contemporary English speakers?", "What else was used by pharmas?"], ["What is the immune system?", "What does the immune system protect against?", "What are two of its subsystems?", "What is the subsystem that protects the human brain?", "What is the immune system?", "What does the immune system protect against?", "What are two of its subsystems?", "The immune system protects organisms against what?", "What are the agents the immune system detects known as?", "Which part of the immune system protects the brain?", "What separates the neuroimmune system and peripheral immune system in humans?", "What are the agents detected by the immune system called?", "What are the two major subsystems of the immune system?", "What are the two different types of immunity?", "What is the immune system of the brained known as?"], ["What is it called when people in society rebel against laws they think are unfair?", "What is an example of major civil disobedience in South Africa?", "What was the the movement called that brought Baltic countries independence from the Soviet Union?", "In 2004 the Orange revolution occurred in what country?", "Where was the location of the 2003 Rose revolution?", "Which people brought forward one of the earliest examples of Civil Disobedience?", "Who was one of the earliest examples of Civil Disobedience against?", "Why do people chose civil disobedience to protest?", "What does civil disobedience protest against?", "What civil rights movement in the US was known for it's disobedience?"], ["What is the process of constructing a building or infrastructure?", "What typically involves mass production of similar items without a designated purchaser?", "What percentile of gross domestic product is construction comprised of?", "What three things are needed for construction to take place?", "Construction takes place on location for who?"], ["Along with non-governmental and nonstate schools, what is another name for private schools?", "Along with sport and art, what is a type of talent scholarship?", "Rather than taxation, what are private schools largely funded by?", "What right do private schools have that public schools don't?"], ["What individual is the school named after?", "When did the undergraduate program become coeducational?", "What was the name of the leader through the Great Depression and World War II?", "What organization did Harvard found in 1900?", "What president of the university transformed it into a modern research university?"], ["Which Florida city has the biggest population?", "What was the population Jacksonville city as of 2010?", "Based on population alone, what is Jacksonville's ranking in the United States?", "In which county does Jacksonville reside?", "What year did consolidation cause Jacksonville to become part of Duval County?"], ["What percentage of global assets does the richest 1% of people have?", "According to Oxfam, the 85 richest people have wealth equal to how many average people?", "In order to be considered in the top percentile, a person would need to amass how much money each year?", "What has caused Oxfam's findings to be questioned?", "Why does Oxfam and Credit Suisse believe their findings are being doubted?", "What percent of the global assets in 2000 were owned by just 1% of adults?", "What do the three richest people in the world posses more of than the lowest 48 nations together?", "How much was the combined wealth of the \"10 Million dollar millionaires\" in 2008?", "How much of the global wealth will the wealthiest 1 percent own by 2016?", "Why are there more poor people in the United States and Europe than China?"], ["Who is the producer of Doctor Who?", "What year did Doctor Who first show on TV?", "What is Doctor Who's space ship called?", "What does the outside of the Tardis resemble?", "What type/genre of TV show is Doctor Who?"], ["What kind of university is the University of Chicago?", "When was the University of Chicago established?", "How many professional schools does the University of Chicago have?", "How many academic research divisions does the University of Chicago have?", "How many students does the University of Chicago have enlisted?"], ["What is the Chinese name for the Yuan dynasty?", "What is the Yuan dynasty's official name?", "Who started the Yuan dynasty?", "Who led the Mongolian Borjigin clan?", "When did Khan formally declare the Yuan dynasty?"], ["Where is Kenya located?", "What is Kenya a founding member of?", "What is the capitol of Kenya?", "What country boarders the south of Kenya?", "What was the population of Kenya in 2014?"], ["What organization is the IPCC a part of?", "What UN organizations established the IPCC?", "What does the UN want to stabilize?", "What is the UN's climate change treaty?", "What UN resolution endorsed the IPCC?"], ["What is the primary purpose of chloroplasts?", "What does ATP store?", "What does NADPH store?", "What is the process of turning CO2 into organic molecules called?", "How many chloroplasts per cell does algae have?"], ["What is the only divisor besides 1 that a prime number can have?", "What are numbers greater than 1 that can be divided by 3 or more numbers called?", "What theorem defines the main role of primes in number theory?", "Any number larger than 1 can be represented as a product of what?", "Why must one be excluded in order to preserve the uniqueness of the fundamental theorem?"], ["Where does the Rhine begin? ", "Where does the Rhine empty?", "What is the largest city the Rhine runs through? ", "What river is larger than the Rhine?", "How long is the Rhine?", "Where is the Rhine? ", "What country does the Rhine empty?", "How long is the Rhine? "], ["When was the current parliament of Scotland convened?", "What act set out the Parliament's powers as a devolved legislature?", "The legislative competence of the Parliament species what areas?", "To what body are certain powers explicitly specified as being reserved for?", "The Scottish Parliament may legislate as it pleases as long as the powers aren't already reserved by where?"], ["What is an Islamic revival movement?", "What aspects of life does Islamism seek to integrate itself into?", "What goal does Islamism have when it comes to society and government?", "What have the two different Islamist movements been described as oscillating between?", "One strategy of Islamization is to seize power by what methods?"], ["The word imperialism has it's origins in which ancient language? ", "By what means is imperialism usually administered?", "The term imperialism has been applied to western countries, and which eastern county?", "Imperialism is responsible for the rapid spread of what?"], ["What does UMC stand for?", "What is the United Methodist Church?", "When was the UMC founded?", "Who founded the UMC?", "What is the church's theological orientation?"], ["When was the French and Indian War?", "Who fought in the French and Indian war?", "How many people were in French North American Colonies?", "How many people were in British North American Colonies?"], ["What concept did philosophers in antiquity use to study simple machines?", "What was the belief that maintaining motion required force?", "Who had mathmatical insite?", "How long did it take to improve on Sir Isaac Newton's laws of motion?", "Who develped the theory of relativity?"]]} \ No newline at end of file diff --git a/lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/ml/Answer.java b/lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/ml/Answer.java new file mode 100644 index 00000000000..2d5f072276e --- /dev/null +++ b/lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/ml/Answer.java @@ -0,0 +1,39 @@ +package org.tensorflow.lite.examples.bertqa.ml; + +/** QA Answer class. */ +public class Answer implements Comparable{ + public Pos pos; + public String text; + + public Answer(String text, Pos pos) { + this.text = text; + this.pos = pos; + } + + public Answer(String text, int start, int end, float logit) { + this(text, new Pos(start, end, logit)); + } + + @Override + public int compareTo(Answer other) { + return Float.compare(other.pos.logit, this.pos.logit); + } + + /** Position and related information from the model. */ + public static class Pos implements Comparable { + public int start; + public int end; + public float logit; + + public Pos(int start, int end, float logit) { + this.start = start; + this.end = end; + this.logit = logit; + } + + @Override + public int compareTo(Pos other) { + return Float.compare(other.logit, this.logit); + } + } +} diff --git a/lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/ml/Feature.java b/lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/ml/Feature.java new file mode 100644 index 00000000000..c26880edc09 --- /dev/null +++ b/lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/ml/Feature.java @@ -0,0 +1,42 @@ +/* Copyright 2019 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +package org.tensorflow.lite.examples.bertqa.ml; + +import com.google.common.primitives.Ints; + +import java.util.List; +import java.util.Map; + +/** Feature to be fed into the Bert model. */ +public final class Feature { + public final int[] inputIds; + public final int[] inputMask; + public final int[] segmentIds; + public final List origTokens; + public final Map tokenToOrigMap; + + public Feature( + List inputIds, + List inputMask, + List segmentIds, + List origTokens, + Map tokenToOrigMap) { + this.inputIds = Ints.toArray(inputIds); + this.inputMask = Ints.toArray(inputMask); + this.segmentIds = Ints.toArray(segmentIds); + this.origTokens = origTokens; + this.tokenToOrigMap = tokenToOrigMap; + } +} diff --git a/lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/ml/FeatureConverter.java b/lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/ml/FeatureConverter.java new file mode 100644 index 00000000000..f337b016315 --- /dev/null +++ b/lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/ml/FeatureConverter.java @@ -0,0 +1,106 @@ +/* Copyright 2019 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +package org.tensorflow.lite.examples.bertqa.ml; + +import org.tensorflow.lite.examples.bertqa.tokenization.FullTokenizer; + +import java.util.ArrayList; +import java.util.Arrays; +import java.util.Collections; +import java.util.HashMap; +import java.util.List; +import java.util.Map; + +/** Convert String to features that can be fed into BERT model. */ +public final class FeatureConverter { + private final FullTokenizer tokenizer; + private final int maxQueryLen; + private final int maxSeqLen; + + public FeatureConverter( + Map inputDic, boolean doLowerCase, int maxQueryLen, int maxSeqLen) { + this.tokenizer = new FullTokenizer(inputDic, doLowerCase); + this.maxQueryLen = maxQueryLen; + this.maxSeqLen = maxSeqLen; + } + + public Feature convert(String query, String context) { + List queryTokens = tokenizer.tokenize(query); + if (queryTokens.size() > maxQueryLen) { + queryTokens = queryTokens.subList(0, maxQueryLen); + } + + List origTokens = Arrays.asList(context.trim().split("\\s+")); + List tokenToOrigIndex = new ArrayList<>(); + List allDocTokens = new ArrayList<>(); + for (int i = 0; i < origTokens.size(); i++) { + String token = origTokens.get(i); + List subTokens = tokenizer.tokenize(token); + for (String subToken : subTokens) { + tokenToOrigIndex.add(i); + allDocTokens.add(subToken); + } + } + + // -3 accounts for [CLS], [SEP] and [SEP]. + int maxContextLen = maxSeqLen - queryTokens.size() - 3; + if (allDocTokens.size() > maxContextLen) { + allDocTokens = allDocTokens.subList(0, maxContextLen); + } + + List tokens = new ArrayList<>(); + List segmentIds = new ArrayList<>(); + + // Map token index to original index (in feature.origTokens). + Map tokenToOrigMap = new HashMap<>(); + + // Start of generating the features. + tokens.add("[CLS]"); + segmentIds.add(0); + + // For query input. + for (String queryToken : queryTokens) { + tokens.add(queryToken); + segmentIds.add(0); + } + + // For Separation. + tokens.add("[SEP]"); + segmentIds.add(0); + + // For Text Input. + for (int i = 0; i < allDocTokens.size(); i++) { + String docToken = allDocTokens.get(i); + tokens.add(docToken); + segmentIds.add(1); + tokenToOrigMap.put(tokens.size(), tokenToOrigIndex.get(i)); + } + + // For ending mark. + tokens.add("[SEP]"); + segmentIds.add(1); + + List inputIds = tokenizer.convertTokensToIds(tokens); + List inputMask = new ArrayList<>(Collections.nCopies(inputIds.size(), 1)); + + while (inputIds.size() < maxSeqLen) { + inputIds.add(0); + inputMask.add(0); + segmentIds.add(0); + } + + return new Feature(inputIds, inputMask, segmentIds, origTokens, tokenToOrigMap); + } +} diff --git a/lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/ml/LoadDatasetClient.java b/lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/ml/LoadDatasetClient.java new file mode 100644 index 00000000000..b0662dc0acd --- /dev/null +++ b/lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/ml/LoadDatasetClient.java @@ -0,0 +1,89 @@ +/* Copyright 2019 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +package org.tensorflow.lite.examples.bertqa.ml; + +import android.content.Context; +import android.util.Log; + +import com.google.gson.Gson; +import com.google.gson.stream.JsonReader; + +import java.io.IOException; +import java.io.InputStream; +import java.io.InputStreamReader; +import java.util.HashMap; +import java.util.List; + +/** + * Interface to load squad dataset. Provide passages for users to choose from & provide questions + * for autoCompleteTextView. + */ +public class LoadDatasetClient { + private static final String TAG = "BertAppDemo"; + private static final String JSON_DIR = "qa.json"; + private final Context context; + + private String[] contents; + private String[] titles; + private String[][] questions; + + public LoadDatasetClient(Context context) { + this.context = context; + loadJson(); + } + + private void loadJson() { + try { + InputStream is = context.getAssets().open(JSON_DIR); + JsonReader reader = new JsonReader(new InputStreamReader(is)); + HashMap>> map = new Gson().fromJson(reader, HashMap.class); + List> jsonTitles = map.get("titles"); + List> jsonContents = map.get("contents"); + List> jsonQuestions = map.get("questions"); + + titles = listToArray(jsonTitles); + contents = listToArray(jsonContents); + + questions = new String[jsonQuestions.size()][]; + int index = 0; + for (List item : jsonQuestions) { + questions[index++] = item.toArray(new String[item.size()]); + } + } catch (IOException ex) { + Log.e(TAG, ex.toString()); + } + } + + private static String[] listToArray(List> list) { + String[] answer = new String[list.size()]; + int index = 0; + for (List item : list) { + answer[index++] = item.get(0); + } + return answer; + } + + public String[] getTitles() { + return titles; + } + + public String getContent(int index) { + return contents[index]; + } + + public String[] getQuestions(int index) { + return questions[index]; + } +} diff --git a/lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/ml/ModelHelper.java b/lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/ml/ModelHelper.java new file mode 100644 index 00000000000..8b448af9d3a --- /dev/null +++ b/lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/ml/ModelHelper.java @@ -0,0 +1,86 @@ +/* Copyright 2019 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +package org.tensorflow.lite.examples.bertqa.ml; + +import static com.google.common.base.Verify.verify; + +import android.content.Context; +import android.content.res.AssetFileDescriptor; +import android.content.res.AssetManager; +import android.util.Log; + +import org.tensorflow.lite.support.metadata.MetadataExtractor; + +import java.io.BufferedReader; +import java.io.FileInputStream; +import java.io.IOException; +import java.io.InputStream; +import java.io.InputStreamReader; +import java.nio.MappedByteBuffer; +import java.nio.channels.FileChannel; +import java.util.HashMap; +import java.util.Map; + +/** Helper to load TfLite model and dictionary. */ +public class ModelHelper { + private static final String TAG = "BertDemo"; + public static final String MODEL_PATH = "model.tflite"; + public static final String DIC_PATH = "vocab.txt"; + + private ModelHelper() {} + + /** Load tflite model from context. */ + public static MappedByteBuffer loadModelFile(Context context) throws IOException { + return loadModelFile(context.getAssets()); + } + + /** Load tflite model from assets. */ + public static MappedByteBuffer loadModelFile(AssetManager assetManager) throws IOException { + try (AssetFileDescriptor fileDescriptor = assetManager.openFd(MODEL_PATH); + FileInputStream inputStream = new FileInputStream(fileDescriptor.getFileDescriptor())) { + FileChannel fileChannel = inputStream.getChannel(); + long startOffset = fileDescriptor.getStartOffset(); + long declaredLength = fileDescriptor.getDeclaredLength(); + return fileChannel.map(FileChannel.MapMode.READ_ONLY, startOffset, declaredLength); + } + } + + /** Extract dictionary from metadata. */ + public static Map extractDictionary(MetadataExtractor metadataExtractor) { + Map dic = null; + try { + verify(metadataExtractor != null, "metadataExtractor can't be null."); + dic = loadDictionaryFile(metadataExtractor.getAssociatedFile(DIC_PATH)); + Log.v(TAG, "Dictionary loaded."); + } catch (IOException ex) { + Log.e(TAG, ex.getMessage()); + } + return dic; + } + + /** Load dictionary from assets. */ + public static Map loadDictionaryFile(InputStream inputStream) + throws IOException { + Map dic = new HashMap<>(); + try (BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream))) { + int index = 0; + while (reader.ready()) { + String key = reader.readLine(); + dic.put(key, index++); + } + } + return dic; + } +} diff --git a/lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/ml/QaClient.java b/lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/ml/QaClient.java new file mode 100644 index 00000000000..c53e26535bb --- /dev/null +++ b/lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/ml/QaClient.java @@ -0,0 +1,280 @@ +/* Copyright 2019 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +package org.tensorflow.lite.examples.bertqa.ml; + +import static com.google.common.base.Verify.verify; + +import android.content.Context; +import android.util.Log; + + +import androidx.annotation.WorkerThread; + +import com.google.common.base.Joiner; + +import org.tensorflow.lite.Interpreter; +import org.tensorflow.lite.support.metadata.MetadataExtractor; +import org.tensorflow.lite.support.metadata.schema.TensorMetadata; + +import java.io.IOException; +import java.nio.ByteBuffer; +import java.util.ArrayList; +import java.util.Collections; +import java.util.HashMap; +import java.util.List; +import java.util.Map; + +/** Interface to load TfLite model and provide predictions. */ +public class QaClient implements AutoCloseable { + private static final String TAG = "BertDemo"; + + private static final int MAX_ANS_LEN = 32; + private static final int MAX_QUERY_LEN = 64; + private static final int MAX_SEQ_LEN = 384; + private static final boolean DO_LOWER_CASE = true; + private static final int PREDICT_ANS_NUM = 5; + private static final int NUM_LITE_THREADS = 4; + + private static final String IDS_TENSOR_NAME = "ids"; + private static final String MASK_TENSOR_NAME = "mask"; + private static final String SEGMENT_IDS_TENSOR_NAME = "segment_ids"; + private static final String END_LOGITS_TENSOR_NAME = "end_logits"; + private static final String START_LOGITS_TENSOR_NAME = "start_logits"; + + // Need to shift 1 for outputs ([CLS]). + private static final int OUTPUT_OFFSET = 1; + + private final Context context; + private final Map dic = new HashMap<>(); + private final FeatureConverter featureConverter; + private Interpreter tflite; + private MetadataExtractor metadataExtractor = null; + + private static final Joiner SPACE_JOINER = Joiner.on(" "); + + public QaClient(Context context) { + this.context = context; + this.featureConverter = new FeatureConverter(dic, DO_LOWER_CASE, MAX_QUERY_LEN, MAX_SEQ_LEN); + } + + @WorkerThread + public synchronized void loadModel() { + try { + ByteBuffer buffer = ModelHelper.loadModelFile(context); + metadataExtractor = new MetadataExtractor(buffer); + Map loadedDic = ModelHelper.extractDictionary(metadataExtractor); + verify(loadedDic != null, "dic can't be null."); + dic.putAll(loadedDic); + + Interpreter.Options opt = new Interpreter.Options(); + opt.setNumThreads(NUM_LITE_THREADS); + tflite = new Interpreter(buffer, opt); + Log.v(TAG, "TFLite model loaded."); + } catch (IOException ex) { + Log.e(TAG, ex.getMessage()); + } + } + + @WorkerThread + public synchronized void unload() { + close(); + } + + @Override + public void close() { + if (tflite != null) { + tflite.close(); + tflite = null; + } + dic.clear(); + } + + + /** + * Input: Original content and query for the QA task. Later converted to Feature by + * FeatureConverter. Output: A String[] array of answers and a float[] array of corresponding + * logits. + */ + @WorkerThread + public synchronized List predict(String query, String content) { + Log.v(TAG, "TFLite model: " + ModelHelper.MODEL_PATH + " running..."); + Log.v(TAG, "Convert Feature..."); + Feature feature = featureConverter.convert(query, content); + + Log.v(TAG, "Set inputs..."); + int[][] inputIds = new int[1][MAX_SEQ_LEN]; + int[][] inputMask = new int[1][MAX_SEQ_LEN]; + int[][] segmentIds = new int[1][MAX_SEQ_LEN]; + float[][] startLogits = new float[1][MAX_SEQ_LEN]; + float[][] endLogits = new float[1][MAX_SEQ_LEN]; + + for (int j = 0; j < MAX_SEQ_LEN; j++) { + inputIds[0][j] = feature.inputIds[j]; + inputMask[0][j] = feature.inputMask[j]; + segmentIds[0][j] = feature.segmentIds[j]; + } + + Object[] inputs = new Object[3]; + boolean useInputMetadata = false; + if (metadataExtractor != null && metadataExtractor.getInputTensorCount() == 3) { + // If metadata exists and the size of input tensors in metadata is 3, use metadata to treat + // the tensor order. Since the order of input tensors can be different for different models, + // set the inputs according to input tensor names. + useInputMetadata = true; + for (int i = 0; i < 3; i++) { + TensorMetadata inputMetadata = metadataExtractor.getInputTensorMetadata(i); + switch (inputMetadata.name()) { + case IDS_TENSOR_NAME: + inputs[i] = inputIds; + break; + case MASK_TENSOR_NAME: + inputs[i] = inputMask; + break; + case SEGMENT_IDS_TENSOR_NAME: + inputs[i] = segmentIds; + break; + default: + Log.e(TAG, "Input name in metadata doesn't match the default input tensor names."); + useInputMetadata = false; + } + } + } + if (!useInputMetadata) { + // If metadata doesn't exists or doesn't contain the info, fail back to a hard-coded order. + Log.v(TAG, "Use hard-coded order of input tensors."); + inputs[0] = inputIds; + inputs[1] = inputMask; + inputs[2] = segmentIds; + } + + Map output = new HashMap<>(); + // Hard-coded idx for output, maybe changed according to metadata below. + int endLogitsIdx = 0; + int startLogitsIdx = 1; + boolean useOutputMetadata = false; + if (metadataExtractor != null && metadataExtractor.getOutputTensorCount() == 2) { + // If metadata exists and the size of output tensors in metadata is 2, use metadata to treat + // the tensor order. Since the order of output tensors can be different for different models, + // set the indexs of the outputs according to output tensor names. + useOutputMetadata = true; + for (int i = 0; i < 2; i++) { + TensorMetadata outputMetadata = metadataExtractor.getOutputTensorMetadata(i); + switch (outputMetadata.name()) { + case END_LOGITS_TENSOR_NAME: + endLogitsIdx = i; + break; + case START_LOGITS_TENSOR_NAME: + startLogitsIdx = i; + break; + default: + Log.e(TAG, "Output name in metadata doesn't match the default output tensor names."); + useOutputMetadata = false; + } + } + } + if (!useOutputMetadata) { + Log.v(TAG, "Use hard-coded order of output tensors."); + endLogitsIdx = 0; + startLogitsIdx = 1; + } + output.put(endLogitsIdx, endLogits); + output.put(startLogitsIdx, startLogits); + + Log.v(TAG, "Run inference..."); + tflite.runForMultipleInputsOutputs(inputs, output); + + Log.v(TAG, "Convert answers..."); + List answers = getBestAnswers(startLogits[0], endLogits[0], feature); + Log.v(TAG, "Finish."); + return answers; + } + + /** Find the Best N answers & logits from the logits array and input feature. */ + private synchronized List getBestAnswers( + float[] startLogits, float[] endLogits, Feature feature) { + // Model uses the closed interval [start, end] for indices. + int[] startIndexes = getBestIndex(startLogits); + int[] endIndexes = getBestIndex(endLogits); + + List origResults = new ArrayList<>(); + for (int start : startIndexes) { + for (int end : endIndexes) { + if (!feature.tokenToOrigMap.containsKey(start + OUTPUT_OFFSET)) { + continue; + } + if (!feature.tokenToOrigMap.containsKey(end + OUTPUT_OFFSET)) { + continue; + } + if (end < start) { + continue; + } + int length = end - start + 1; + if (length > MAX_ANS_LEN) { + continue; + } + origResults.add(new Answer.Pos(start, end, startLogits[start] + endLogits[end])); + } + } + + Collections.sort(origResults); + + List answers = new ArrayList<>(); + for (int i = 0; i < origResults.size(); i++) { + if (i >= PREDICT_ANS_NUM) { + break; + } + + String convertedText; + if (origResults.get(i).start > 0) { + convertedText = convertBack(feature, origResults.get(i).start, origResults.get(i).end); + } else { + convertedText = ""; + } + Answer ans = new Answer(convertedText, origResults.get(i)); + answers.add(ans); + } + return answers; + } + + /** Get the n-best logits from a list of all the logits. */ + @WorkerThread + private synchronized int[] getBestIndex(float[] logits) { + List tmpList = new ArrayList<>(); + for (int i = 0; i < MAX_SEQ_LEN; i++) { + tmpList.add(new Answer.Pos(i, i, logits[i])); + } + Collections.sort(tmpList); + + int[] indexes = new int[PREDICT_ANS_NUM]; + for (int i = 0; i < PREDICT_ANS_NUM; i++) { + indexes[i] = tmpList.get(i).start; + } + + return indexes; + } + + /** Convert the answer back to original text form. */ + @WorkerThread + private static String convertBack(Feature feature, int start, int end) { + // Shifted index is: index of logits + offset. + int shiftedStart = start + OUTPUT_OFFSET; + int shiftedEnd = end + OUTPUT_OFFSET; + int startIndex = feature.tokenToOrigMap.get(shiftedStart); + int endIndex = feature.tokenToOrigMap.get(shiftedEnd); + // end + 1 for the closed interval. + String ans = SPACE_JOINER.join(feature.origTokens.subList(startIndex, endIndex + 1)); + return ans; + } +} diff --git a/lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/tokenization/BasicTokenizer.java b/lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/tokenization/BasicTokenizer.java new file mode 100644 index 00000000000..70b22de3fe7 --- /dev/null +++ b/lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/tokenization/BasicTokenizer.java @@ -0,0 +1,105 @@ +/* Copyright 2019 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +package org.tensorflow.lite.examples.bertqa.tokenization; + +import com.google.common.base.Ascii; +import com.google.common.collect.Iterables; + +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; + +/** Basic tokenization (punctuation splitting, lower casing, etc.) */ +public final class BasicTokenizer { + private final boolean doLowerCase; + + public BasicTokenizer(boolean doLowerCase) { + this.doLowerCase = doLowerCase; + } + + public List tokenize(String text) { + String cleanedText = cleanText(text); + + List origTokens = whitespaceTokenize(cleanedText); + + StringBuilder stringBuilder = new StringBuilder(); + for (String token : origTokens) { + if (doLowerCase) { + token = Ascii.toLowerCase(token); + } + List list = runSplitOnPunc(token); + for (String subToken : list) { + stringBuilder.append(subToken).append(" "); + } + } + return whitespaceTokenize(stringBuilder.toString()); + } + + /* Performs invalid character removal and whitespace cleanup on text. */ + static String cleanText(String text) { + if (text == null) { + throw new NullPointerException("The input String is null."); + } + + StringBuilder stringBuilder = new StringBuilder(""); + for (int index = 0; index < text.length(); index++) { + char ch = text.charAt(index); + + // Skip the characters that cannot be used. + if (CharChecker.isInvalid(ch) || CharChecker.isControl(ch)) { + continue; + } + if (CharChecker.isWhitespace(ch)) { + stringBuilder.append(" "); + } else { + stringBuilder.append(ch); + } + } + return stringBuilder.toString(); + } + + /* Runs basic whitespace cleaning and splitting on a piece of text. */ + static List whitespaceTokenize(String text) { + if (text == null) { + throw new NullPointerException("The input String is null."); + } + return Arrays.asList(text.split(" ")); + } + + /* Splits punctuation on a piece of text. */ + static List runSplitOnPunc(String text) { + if (text == null) { + throw new NullPointerException("The input String is null."); + } + + List tokens = new ArrayList<>(); + boolean startNewWord = true; + for (int i = 0; i < text.length(); i++) { + char ch = text.charAt(i); + if (CharChecker.isPunctuation(ch)) { + tokens.add(String.valueOf(ch)); + startNewWord = true; + } else { + if (startNewWord) { + tokens.add(""); + startNewWord = false; + } + tokens.set(tokens.size() - 1, Iterables.getLast(tokens) + ch); + } + } + + return tokens; + } +} diff --git a/lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/tokenization/CharChecker.java b/lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/tokenization/CharChecker.java new file mode 100644 index 00000000000..dc08dbe5e70 --- /dev/null +++ b/lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/tokenization/CharChecker.java @@ -0,0 +1,58 @@ +/* Copyright 2019 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +package org.tensorflow.lite.examples.bertqa.tokenization; + +/** To check whether a char is whitespace/control/punctuation. */ +final class CharChecker { + + /** To judge whether it's an empty or unknown character. */ + public static boolean isInvalid(char ch) { + return (ch == 0 || ch == 0xfffd); + } + + /** To judge whether it's a control character(exclude whitespace). */ + public static boolean isControl(char ch) { + if (Character.isWhitespace(ch)) { + return false; + } + int type = Character.getType(ch); + return (type == Character.CONTROL || type == Character.FORMAT); + } + + /** To judge whether it can be regarded as a whitespace. */ + public static boolean isWhitespace(char ch) { + if (Character.isWhitespace(ch)) { + return true; + } + int type = Character.getType(ch); + return (type == Character.SPACE_SEPARATOR + || type == Character.LINE_SEPARATOR + || type == Character.PARAGRAPH_SEPARATOR); + } + + /** To judge whether it's a punctuation. */ + public static boolean isPunctuation(char ch) { + int type = Character.getType(ch); + return (type == Character.CONNECTOR_PUNCTUATION + || type == Character.DASH_PUNCTUATION + || type == Character.START_PUNCTUATION + || type == Character.END_PUNCTUATION + || type == Character.INITIAL_QUOTE_PUNCTUATION + || type == Character.FINAL_QUOTE_PUNCTUATION + || type == Character.OTHER_PUNCTUATION); + } + + private CharChecker() {} +} diff --git a/lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/tokenization/FullTokenizer.java b/lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/tokenization/FullTokenizer.java new file mode 100644 index 00000000000..01c78d9ef6a --- /dev/null +++ b/lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/tokenization/FullTokenizer.java @@ -0,0 +1,52 @@ +/* Copyright 2019 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +package org.tensorflow.lite.examples.bertqa.tokenization; + +import java.util.ArrayList; +import java.util.List; +import java.util.Map; + +/** + * A java realization of Bert tokenization. Original python code: + * https://github.com/google-research/bert/blob/master/tokenization.py runs full tokenization to + * tokenize a String into split subtokens or ids. + */ +public final class FullTokenizer { + private final BasicTokenizer basicTokenizer; + private final WordpieceTokenizer wordpieceTokenizer; + private final Map dic; + + public FullTokenizer(Map inputDic, boolean doLowerCase) { + dic = inputDic; + basicTokenizer = new BasicTokenizer(doLowerCase); + wordpieceTokenizer = new WordpieceTokenizer(inputDic); + } + + public List tokenize(String text) { + List splitTokens = new ArrayList<>(); + for (String token : basicTokenizer.tokenize(text)) { + splitTokens.addAll(wordpieceTokenizer.tokenize(token)); + } + return splitTokens; + } + + public List convertTokensToIds(List tokens) { + List outputIds = new ArrayList<>(); + for (String token : tokens) { + outputIds.add(dic.get(token)); + } + return outputIds; + } +} diff --git a/lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/tokenization/WordpieceTokenizer.java b/lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/tokenization/WordpieceTokenizer.java new file mode 100644 index 00000000000..446f80bcf31 --- /dev/null +++ b/lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/tokenization/WordpieceTokenizer.java @@ -0,0 +1,94 @@ +/* Copyright 2019 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +package org.tensorflow.lite.examples.bertqa.tokenization; + +import java.util.ArrayList; +import java.util.List; +import java.util.Map; + +/** Word piece tokenization to split a piece of text into its word pieces. */ +public final class WordpieceTokenizer { + private final Map dic; + + private static final String UNKNOWN_TOKEN = "[UNK]"; // For unknown words. + private static final int MAX_INPUTCHARS_PER_WORD = 200; + + public WordpieceTokenizer(Map vocab) { + dic = vocab; + } + + /** + * Tokenizes a piece of text into its word pieces. This uses a greedy longest-match-first + * algorithm to perform tokenization using the given vocabulary. For example: input = "unaffable", + * output = ["un", "##aff", "##able"]. + * + * @param text: A single token or whitespace separated tokens. This should have already been + * passed through `BasicTokenizer. + * @return A list of wordpiece tokens. + */ + public List tokenize(String text) { + if (text == null) { + throw new NullPointerException("The input String is null."); + } + + List outputTokens = new ArrayList<>(); + for (String token : BasicTokenizer.whitespaceTokenize(text)) { + + if (token.length() > MAX_INPUTCHARS_PER_WORD) { + outputTokens.add(UNKNOWN_TOKEN); + continue; + } + + boolean isBad = false; // Mark if a word cannot be tokenized into known subwords. + int start = 0; + List subTokens = new ArrayList<>(); + + while (start < token.length()) { + String curSubStr = ""; + + int end = token.length(); // Longer substring matches first. + while (start < end) { + String subStr = + (start == 0) ? token.substring(start, end) : "##" + token.substring(start, end); + if (dic.containsKey(subStr)) { + curSubStr = subStr; + break; + } + end--; + } + + // The word doesn't contain any known subwords. + if ("".equals(curSubStr)) { + isBad = true; + break; + } + + // curSubStr is the longeset subword that can be found. + subTokens.add(curSubStr); + + // Proceed to tokenize the resident string. + start = end; + } + + if (isBad) { + outputTokens.add(UNKNOWN_TOKEN); + } else { + outputTokens.addAll(subTokens); + } + } + + return outputTokens; + } +} diff --git a/lite/examples/bert_qa/android/lib_task_api/.gitignore b/lite/examples/bert_qa/android/lib_task_api/.gitignore new file mode 100644 index 00000000000..42afabfd2ab --- /dev/null +++ b/lite/examples/bert_qa/android/lib_task_api/.gitignore @@ -0,0 +1 @@ +/build \ No newline at end of file diff --git a/lite/examples/bert_qa/android/lib_task_api/build.gradle b/lite/examples/bert_qa/android/lib_task_api/build.gradle new file mode 100644 index 00000000000..024004d37dc --- /dev/null +++ b/lite/examples/bert_qa/android/lib_task_api/build.gradle @@ -0,0 +1,52 @@ +plugins { + id 'com.android.library' +} + +android { + compileSdkVersion 30 + buildToolsVersion "30.0.3" + + defaultConfig { + minSdkVersion 21 + targetSdkVersion 30 + versionCode 1 + versionName "1.0" + + testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner" + consumerProguardFiles "consumer-rules.pro" + } + + buildTypes { + release { + minifyEnabled false + proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'), 'proguard-rules.pro' + } + } + compileOptions { + sourceCompatibility JavaVersion.VERSION_1_8 + targetCompatibility JavaVersion.VERSION_1_8 + } + aaptOptions { + noCompress "tflite" + } + testOptions { + unitTests { + includeAndroidResources = true + } + } +} + +// App assets folder: +project.ext.ASSET_DIR = projectDir.toString() + '/src/main/assets/' + +// Download TF Lite model. +apply from: 'download.gradle' + +dependencies { + implementation 'org.tensorflow:tensorflow-lite-task-text:0.2.0' + + androidTestImplementation 'junit:junit:4.13.2' + androidTestImplementation 'androidx.test.ext:junit:1.1.1' + androidTestImplementation 'com.google.truth:truth:1.0' + androidTestImplementation 'androidx.test.espresso:espresso-core:3.4.0' +} \ No newline at end of file diff --git a/lite/examples/bert_qa/android/lib_task_api/consumer-rules.pro b/lite/examples/bert_qa/android/lib_task_api/consumer-rules.pro new file mode 100644 index 00000000000..e69de29bb2d diff --git a/lite/examples/bert_qa/android/lib_task_api/download.gradle b/lite/examples/bert_qa/android/lib_task_api/download.gradle new file mode 100644 index 00000000000..5a6b670004f --- /dev/null +++ b/lite/examples/bert_qa/android/lib_task_api/download.gradle @@ -0,0 +1,15 @@ +apply plugin: 'de.undercouch.download' + +task downloadLiteModel { + def downloadFiles = [ + 'https://storage.googleapis.com/download.tensorflow.org/models/tflite/bert_qa/contents_from_squad.json': 'qa.json', + 'https://tfhub.dev/tensorflow/lite-model/mobilebert/1/metadata/1?lite-format=tflite': 'model.tflite', + ] + downloadFiles.each { key, value -> + download { + src key + dest project.ext.ASSET_DIR + value + overwrite true + } + } +} \ No newline at end of file diff --git a/lite/examples/bert_qa/android/lib_task_api/proguard-rules.pro b/lite/examples/bert_qa/android/lib_task_api/proguard-rules.pro new file mode 100644 index 00000000000..481bb434814 --- /dev/null +++ b/lite/examples/bert_qa/android/lib_task_api/proguard-rules.pro @@ -0,0 +1,21 @@ +# Add project specific ProGuard rules here. +# You can control the set of applied configuration files using the +# proguardFiles setting in build.gradle. +# +# For more details, see +# http://developer.android.com/guide/developing/tools/proguard.html + +# If your project uses WebView with JS, uncomment the following +# and specify the fully qualified class name to the JavaScript interface +# class: +#-keepclassmembers class fqcn.of.javascript.interface.for.webview { +# public *; +#} + +# Uncomment this to preserve the line number information for +# debugging stack traces. +#-keepattributes SourceFile,LineNumberTable + +# If you keep the line number information, uncomment this to +# hide the original source file name. +#-renamesourcefileattribute SourceFile \ No newline at end of file diff --git a/lite/examples/bert_qa/android/lib_task_api/src/androidTest/java/org/tensorflow/lite/examples/bertqa/QaClientTest.java b/lite/examples/bert_qa/android/lib_task_api/src/androidTest/java/org/tensorflow/lite/examples/bertqa/QaClientTest.java new file mode 100644 index 00000000000..c97518f2229 --- /dev/null +++ b/lite/examples/bert_qa/android/lib_task_api/src/androidTest/java/org/tensorflow/lite/examples/bertqa/QaClientTest.java @@ -0,0 +1,68 @@ +/* Copyright 2019 The TensorFlow Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +==============================================================================*/ +package org.tensorflow.lite.examples.bertqa; + +import static com.google.common.truth.Truth.assertThat; + +import androidx.test.core.app.ApplicationProvider; +import androidx.test.ext.junit.runners.AndroidJUnit4; + +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.tensorflow.lite.examples.bertqa.ml.Answer; +import org.tensorflow.lite.examples.bertqa.ml.QaClient; + +import java.util.ArrayList; +import java.util.List; + +/** Tests of {@link QaClient} */ +@RunWith(AndroidJUnit4.class) +public final class QaClientTest { + private QaClient client; + + @Before + public void setUp() { + client = new QaClient(ApplicationProvider.getApplicationContext()); + client.loadModel(); + } + + @Test + public void testGoldSet() { + final String content = + "Nikola Tesla (Serbian Cyrillic: \u041d\u0438\u043a\u043e\u043b\u0430" + + " \u0422\u0435\u0441\u043b\u0430; 10 July 1856 \u2013 7 January 1943) was a Serbian" + + " American inventor, electrical engineer, mechanical engineer, physicist, and" + + " futurist best known for his contributions to the design of the modern alternating" + + " current (AC) electricity supply system."; + + List predict0 = client.predict("What is Tesla's home country?", content); + assertThat(getTexts(predict0)).contains("Serbian"); + List predict1 = client.predict("What was Nikola Tesla's ethnicity?", content); + assertThat(getTexts(predict1)).contains("Serbian"); + List predict2 = client.predict("What does AC stand for?", content); + assertThat(getTexts(predict2)).contains("alternating current"); + List predict3 = client.predict("When was Tesla born?", content); + assertThat(getTexts(predict3)).contains("10 July 1856"); + } + + private static List getTexts(List answers) { + List texts = new ArrayList<>(); + for (Answer ans : answers) { + texts.add(ans.text); + } + return texts; + } +} diff --git a/lite/examples/bert_qa/android/lib_task_api/src/main/AndroidManifest.xml b/lite/examples/bert_qa/android/lib_task_api/src/main/AndroidManifest.xml new file mode 100644 index 00000000000..d765e70b40c --- /dev/null +++ b/lite/examples/bert_qa/android/lib_task_api/src/main/AndroidManifest.xml @@ -0,0 +1,5 @@ + + + + \ No newline at end of file diff --git a/lite/examples/bert_qa/android/lib_task_api/src/main/assets/qa.json b/lite/examples/bert_qa/android/lib_task_api/src/main/assets/qa.json new file mode 100644 index 00000000000..94b7c9c4b50 --- /dev/null +++ b/lite/examples/bert_qa/android/lib_task_api/src/main/assets/qa.json @@ -0,0 +1 @@ +{"titles": [["Super_Bowl_50"], ["Warsaw"], ["Normans"], ["Nikola_Tesla"], ["Computational_complexity_theory"], ["Teacher"], ["Martin_Luther"], ["Southern_California"], ["Sky_(United_Kingdom)"], ["Victoria_(Australia)"], ["Huguenot"], ["Steam_engine"], ["Oxygen"], ["1973_oil_crisis"], ["Apollo_program"], ["European_Union_law"], ["Amazon_rainforest"], ["Ctenophora"], ["Fresno,_California"], ["Packet_switching"], ["Black_Death"], ["Geology"], ["Newcastle_upon_Tyne"], ["Victoria_and_Albert_Museum"], ["American_Broadcasting_Company"], ["Genghis_Khan"], ["Pharmacy"], ["Immune_system"], ["Civil_disobedience"], ["Construction"], ["Private_school"], ["Harvard_University"], ["Jacksonville,_Florida"], ["Economic_inequality"], ["Doctor_Who"], ["University_of_Chicago"], ["Yuan_dynasty"], ["Kenya"], ["Intergovernmental_Panel_on_Climate_Change"], ["Chloroplast"], ["Prime_number"], ["Rhine"], ["Scottish_Parliament"], ["Islamism"], ["Imperialism"], ["United_Methodist_Church"], ["French_and_Indian_War"], ["Force"]], "contents": [["Super Bowl 50 was an American football game to determine the champion of the National Football League (NFL) for the 2015 season. The American Football Conference (AFC) champion Denver Broncos defeated the National Football Conference (NFC) champion Carolina Panthers 24\u201310 to earn their third Super Bowl title. The game was played on February 7, 2016, at Levi's Stadium in the San Francisco Bay Area at Santa Clara, California. As this was the 50th Super Bowl, the league emphasized the \"golden anniversary\" with various gold-themed initiatives, as well as temporarily suspending the tradition of naming each Super Bowl game with Roman numerals (under which the game would have been known as \"Super Bowl L\"), so that the logo could prominently feature the Arabic numerals 50."], ["One of the most famous people born in Warsaw was Maria Sk\u0142odowska-Curie, who achieved international recognition for her research on radioactivity and was the first female recipient of the Nobel Prize. Famous musicians include W\u0142adys\u0142aw Szpilman and Fr\u00e9d\u00e9ric Chopin. Though Chopin was born in the village of \u017belazowa Wola, about 60 km (37 mi) from Warsaw, he moved to the city with his family when he was seven months old. Casimir Pulaski, a Polish general and hero of the American Revolutionary War, was born here in 1745."], ["The Normans (Norman: Nourmands; French: Normands; Latin: Normanni) were the people who in the 10th and 11th centuries gave their name to Normandy, a region in France. They were descended from Norse (\"Norman\" comes from \"Norseman\") raiders and pirates from Denmark, Iceland and Norway who, under their leader Rollo, agreed to swear fealty to King Charles III of West Francia. Through generations of assimilation and mixing with the native Frankish and Roman-Gaulish populations, their descendants would gradually merge with the Carolingian-based cultures of West Francia. The distinct cultural and ethnic identity of the Normans emerged initially in the first half of the 10th century, and it continued to evolve over the succeeding centuries."], ["Nikola Tesla (Serbian Cyrillic: \u041d\u0438\u043a\u043e\u043b\u0430 \u0422\u0435\u0441\u043b\u0430; 10 July 1856 \u2013 7 January 1943) was a Serbian American inventor, electrical engineer, mechanical engineer, physicist, and futurist best known for his contributions to the design of the modern alternating current (AC) electricity supply system."], ["Computational complexity theory is a branch of the theory of computation in theoretical computer science that focuses on classifying computational problems according to their inherent difficulty, and relating those classes to each other. A computational problem is understood to be a task that is in principle amenable to being solved by a computer, which is equivalent to stating that the problem may be solved by mechanical application of mathematical steps, such as an algorithm."], ["The role of teacher is often formal and ongoing, carried out at a school or other place of formal education. In many countries, a person who wishes to become a teacher must first obtain specified professional qualifications or credentials from a university or college. These professional qualifications may include the study of pedagogy, the science of teaching. Teachers, like other professionals, may have to continue their education after they qualify, a process known as continuing professional development. Teachers may use a lesson plan to facilitate student learning, providing a course of study which is called the curriculum."], ["Martin Luther (/\u02c8lu\u02d0\u03b8\u0259r/ or /\u02c8lu\u02d0\u00f0\u0259r/; German: [\u02c8ma\u0250\u032fti\u02d0n \u02c8l\u028at\u0250] ( listen); 10 November 1483 \u2013 18 February 1546) was a German professor of theology, composer, priest, former monk and a seminal figure in the Protestant Reformation. Luther came to reject several teachings and practices of the Late Medieval Catholic Church. He strongly disputed the claim that freedom from God's punishment for sin could be purchased with money. He proposed an academic discussion of the power and usefulness of indulgences in his Ninety-Five Theses of 1517. His refusal to retract all of his writings at the demand of Pope Leo X in 1520 and the Holy Roman Emperor Charles V at the Diet of Worms in 1521 resulted in his excommunication by the Pope and condemnation as an outlaw by the Emperor."], ["Southern California, often abbreviated SoCal, is a geographic and cultural region that generally comprises California's southernmost 10 counties. The region is traditionally described as \"eight counties\", based on demographics and economic ties: Imperial, Los Angeles, Orange, Riverside, San Bernardino, San Diego, Santa Barbara, and Ventura. The more extensive 10-county definition, including Kern and San Luis Obispo counties, is also used based on historical political divisions. Southern California is a major economic center for the state of California and the United States."], ["Formed in November 1990 by the equal merger of Sky Television and British Satellite Broadcasting, BSkyB became the UK's largest digital subscription television company. Following BSkyB's 2014 acquisition of Sky Italia and a majority 90.04% interest in Sky Deutschland in November 2014, its holding company British Sky Broadcasting Group plc changed its name to Sky plc. The United Kingdom operations also changed the company name from British Sky Broadcasting Limited to Sky UK Limited, still trading as Sky."], ["The economy of Victoria is highly diversified: service sectors including financial and property services, health, education, wholesale, retail, hospitality and manufacturing constitute the majority of employment. Victoria's total gross state product (GSP) is ranked second in Australia, although Victoria is ranked fourth in terms of GSP per capita because of its limited mining activity. Culturally, Melbourne is home to a number of museums, art galleries and theatres and is also described as the \"sporting capital of Australia\". The Melbourne Cricket Ground is the largest stadium in Australia, and the host of the 1956 Summer Olympics and the 2006 Commonwealth Games. The ground is also considered the \"spiritual home\" of Australian cricket and Australian rules football, and hosts the grand final of the Australian Football League (AFL) each year, usually drawing crowds of over 95,000 people. Victoria includes eight public universities, with the oldest, the University of Melbourne, having been founded in 1853."], ["Huguenot numbers peaked near an estimated two million by 1562, concentrated mainly in the southern and central parts of France, about one-eighth the number of French Catholics. As Huguenots gained influence and more openly displayed their faith, Catholic hostility grew, in spite of increasingly liberal political concessions and edicts of toleration from the French crown. A series of religious conflicts followed, known as the Wars of Religion, fought intermittently from 1562 to 1598. The wars finally ended with the granting of the Edict of Nantes, which granted the Huguenots substantial religious, political and military autonomy."], ["Steam engines are external combustion engines, where the working fluid is separate from the combustion products. Non-combustion heat sources such as solar power, nuclear power or geothermal energy may be used. The ideal thermodynamic cycle used to analyze this process is called the Rankine cycle. In the cycle, water is heated and transforms into steam within a boiler operating at a high pressure. When expanded through pistons or turbines, mechanical work is done. The reduced-pressure steam is then condensed and pumped back into the boiler."], ["Oxygen is a chemical element with symbol O and atomic number 8. It is a member of the chalcogen group on the periodic table and is a highly reactive nonmetal and oxidizing agent that readily forms compounds (notably oxides) with most elements. By mass, oxygen is the third-most abundant element in the universe, after hydrogen and helium. At standard temperature and pressure, two atoms of the element bind to form dioxygen, a colorless and odorless diatomic gas with the formula O\n2. Diatomic oxygen gas constitutes 20.8% of the Earth's atmosphere. However, monitoring of atmospheric oxygen levels show a global downward trend, because of fossil-fuel burning. Oxygen is the most abundant element by mass in the Earth's crust as part of oxide compounds such as silicon dioxide, making up almost half of the crust's mass."], ["The 1973 oil crisis began in October 1973 when the members of the Organization of Arab Petroleum Exporting Countries (OAPEC, consisting of the Arab members of OPEC plus Egypt and Syria) proclaimed an oil embargo. By the end of the embargo in March 1974, the price of oil had risen from US$3 per barrel to nearly $12 globally; US prices were significantly higher. The embargo caused an oil crisis, or \"shock\", with many short- and long-term effects on global politics and the global economy. It was later called the \"first oil shock\", followed by the 1979 oil crisis, termed the \"second oil shock.\""], ["The Apollo program, also known as Project Apollo, was the third United States human spaceflight program carried out by the National Aeronautics and Space Administration (NASA), which accomplished landing the first humans on the Moon from 1969 to 1972. First conceived during Dwight D. Eisenhower's administration as a three-man spacecraft to follow the one-man Project Mercury which put the first Americans in space, Apollo was later dedicated to President John F. Kennedy's national goal of \"landing a man on the Moon and returning him safely to the Earth\" by the end of the 1960s, which he proposed in a May 25, 1961, address to Congress. Project Mercury was followed by the two-man Project Gemini (1962\u201366). The first manned flight of Apollo was in 1968."], ["European Union law is a body of treaties and legislation, such as Regulations and Directives, which have direct effect or indirect effect on the laws of European Union member states. The three sources of European Union law are primary law, secondary law and supplementary law. The main sources of primary law are the Treaties establishing the European Union. Secondary sources include regulations and directives which are based on the Treaties. The legislature of the European Union is principally composed of the European Parliament and the Council of the European Union, which under the Treaties may establish secondary law to pursue the objective set out in the Treaties."], ["The Amazon rainforest (Portuguese: Floresta Amaz\u00f4nica or Amaz\u00f4nia; Spanish: Selva Amaz\u00f3nica, Amazon\u00eda or usually Amazonia; French: For\u00eat amazonienne; Dutch: Amazoneregenwoud), also known in English as Amazonia or the Amazon Jungle, is a moist broadleaf forest that covers most of the Amazon basin of South America. This basin encompasses 7,000,000 square kilometres (2,700,000 sq mi), of which 5,500,000 square kilometres (2,100,000 sq mi) are covered by the rainforest. This region includes territory belonging to nine nations. The majority of the forest is contained within Brazil, with 60% of the rainforest, followed by Peru with 13%, Colombia with 10%, and with minor amounts in Venezuela, Ecuador, Bolivia, Guyana, Suriname and French Guiana. States or departments in four nations contain \"Amazonas\" in their names. The Amazon represents over half of the planet's remaining rainforests, and comprises the largest and most biodiverse tract of tropical rainforest in the world, with an estimated 390 billion individual trees divided into 16,000 species."], ["Ctenophora (/t\u1d7b\u02c8n\u0252f\u0259r\u0259/; singular ctenophore, /\u02c8t\u025bn\u0259f\u0254\u02d0r/ or /\u02c8ti\u02d0n\u0259f\u0254\u02d0r/; from the Greek \u03ba\u03c4\u03b5\u03af\u03c2 kteis 'comb' and \u03c6\u03ad\u03c1\u03c9 pher\u014d 'carry'; commonly known as comb jellies) is a phylum of animals that live in marine waters worldwide. Their most distinctive feature is the \u2018combs\u2019 \u2013 groups of cilia which they use for swimming \u2013 they are the largest animals that swim by means of cilia. Adults of various species range from a few millimeters to 1.5 m (4 ft 11 in) in size. Like cnidarians, their bodies consist of a mass of jelly, with one layer of cells on the outside and another lining the internal cavity. In ctenophores, these layers are two cells deep, while those in cnidarians are only one cell deep. Some authors combined ctenophores and cnidarians in one phylum, Coelenterata, as both groups rely on water flow through the body cavity for both digestion and respiration. Increasing awareness of the differences persuaded more recent authors to classify them as separate phyla."], ["Fresno (/\u02c8fr\u025bzno\u028a/ FREZ-noh), the county seat of Fresno County, is a city in the U.S. state of California. As of 2015, the city's population was 520,159, making it the fifth-largest city in California, the largest inland city in California and the 34th-largest in the nation. Fresno is in the center of the San Joaquin Valley and is the largest city in the Central Valley, which contains the San Joaquin Valley. It is approximately 220 miles (350 km) northwest of Los Angeles, 170 miles (270 km) south of the state capital, Sacramento, or 185 miles (300 km) south of San Francisco. The name Fresno means \"ash tree\" in Spanish, and an ash leaf is featured on the city's flag."], ["Starting in the late 1950s, American computer scientist Paul Baran developed the concept Distributed Adaptive Message Block Switching with the goal to provide a fault-tolerant, efficient routing method for telecommunication messages as part of a research program at the RAND Corporation, funded by the US Department of Defense. This concept contrasted and contradicted the theretofore established principles of pre-allocation of network bandwidth, largely fortified by the development of telecommunications in the Bell System. The new concept found little resonance among network implementers until the independent work of Donald Davies at the National Physical Laboratory (United Kingdom) (NPL) in the late 1960s. Davies is credited with coining the modern name packet switching and inspiring numerous packet switching networks in Europe in the decade following, including the incorporation of the concept in the early ARPANET in the United States."], ["The Black Death is thought to have originated in the arid plains of Central Asia, where it then travelled along the Silk Road, reaching Crimea by 1343. From there, it was most likely carried by Oriental rat fleas living on the black rats that were regular passengers on merchant ships. Spreading throughout the Mediterranean and Europe, the Black Death is estimated to have killed 30\u201360% of Europe's total population. In total, the plague reduced the world population from an estimated 450 million down to 350\u2013375 million in the 14th century. The world population as a whole did not recover to pre-plague levels until the 17th century. The plague recurred occasionally in Europe until the 19th century."], ["There are three major types of rock: igneous, sedimentary, and metamorphic. The rock cycle is an important concept in geology which illustrates the relationships between these three types of rock, and magma. When a rock crystallizes from melt (magma and/or lava), it is an igneous rock. This rock can be weathered and eroded, and then redeposited and lithified into a sedimentary rock, or be turned into a metamorphic rock due to heat and pressure that change the mineral content of the rock which gives it a characteristic fabric. The sedimentary rock can then be subsequently turned into a metamorphic rock due to heat and pressure and is then weathered, eroded, deposited, and lithified, ultimately becoming a sedimentary rock. Sedimentary rock may also be re-eroded and redeposited, and metamorphic rock may also undergo additional metamorphism. All three types of rocks may be re-melted; when this happens, a new magma is formed, from which an igneous rock may once again crystallize."], ["Newcastle upon Tyne (RP: i/\u02ccnju\u02d0k\u0251\u02d0s\u0259l \u0259\u02ccp\u0252n \u02c8ta\u026an/; Locally: i/nju\u02d0\u02cck\u00e6s\u0259l \u0259\u02ccp\u0259n \u02c8ta\u026an/), commonly known as Newcastle, is a city in Tyne and Wear, North East England, 103 miles (166 km) south of Edinburgh and 277 miles (446 km) north of London on the northern bank of the River Tyne, 8.5 mi (13.7 km) from the North Sea. Newcastle is the most populous city in the North East and Tyneside the eighth most populous conurbation in the United Kingdom. Newcastle is a member of the English Core Cities Group and is a member of the Eurocities network of European cities. Newcastle was part of the county of Northumberland until 1400, when it became a county itself, a status it retained until becoming part of Tyne and Wear in 1974.[not in citation given] The regional nickname and dialect for people from Newcastle and the surrounding area is Geordie."], ["The Victoria and Albert Museum (often abbreviated as the V&A), London, is the world's largest museum of decorative arts and design, housing a permanent collection of over 4.5 million objects. It was founded in 1852 and named after Queen Victoria and Prince Albert. The V&A is located in the Brompton district of the Royal Borough of Kensington and Chelsea, in an area that has become known as \"Albertopolis\" because of its association with Prince Albert, the Albert Memorial and the major cultural institutions with which he was associated. These include the Natural History Museum, the Science Museum and the Royal Albert Hall. The museum is a non-departmental public body sponsored by the Department for Culture, Media and Sport. Like other national British museums, entrance to the museum has been free since 2001."], ["The American Broadcasting Company (ABC) (stylized in its logo as abc since 1957) is an American commercial broadcast television network that is owned by the Disney\u2013ABC Television Group, a subsidiary of Disney Media Networks division of The Walt Disney Company. The network is part of the Big Three television networks. The network is headquartered on Columbus Avenue and West 66th Street in Manhattan, with additional major offices and production facilities in New York City, Los Angeles and Burbank, California."], ["He came to power by uniting many of the nomadic tribes of Northeast Asia. After founding the Mongol Empire and being proclaimed \"Genghis Khan\", he started the Mongol invasions that resulted in the conquest of most of Eurasia. These included raids or invasions of the Qara Khitai, Caucasus, Khwarezmid Empire, Western Xia and Jin dynasties. These campaigns were often accompanied by wholesale massacres of the civilian populations \u2013 especially in the Khwarezmian and Xia controlled lands. By the end of his life, the Mongol Empire occupied a substantial portion of Central Asia and China."], ["The word pharmacy is derived from its root word pharma which was a term used since the 15th\u201317th centuries. However, the original Greek roots from pharmakos imply sorcery or even poison. In addition to pharma responsibilities, the pharma offered general medical advice and a range of services that are now performed solely by other specialist practitioners, such as surgery and midwifery. The pharma (as it was referred to) often operated through a retail shop which, in addition to ingredients for medicines, sold tobacco and patent medicines. Often the place that did this was called an apothecary and several languages have this as the dominant term, though their practices are more akin to a modern pharmacy, in English the term apothecary would today be seen as outdated or only approproriate if herbal remedies were on offer to a large extent. The pharmas also used many other herbs not listed. The Greek word Pharmakeia (Greek: \u03c6\u03b1\u03c1\u03bc\u03b1\u03ba\u03b5\u03af\u03b1) derives from pharmakon (\u03c6\u03ac\u03c1\u03bc\u03b1\u03ba\u03bf\u03bd), meaning \"drug\", \"medicine\" (or \"poison\").[n 1]"], ["The immune system is a system of many biological structures and processes within an organism that protects against disease. To function properly, an immune system must detect a wide variety of agents, known as pathogens, from viruses to parasitic worms, and distinguish them from the organism's own healthy tissue. In many species, the immune system can be classified into subsystems, such as the innate immune system versus the adaptive immune system, or humoral immunity versus cell-mediated immunity. In humans, the blood\u2013brain barrier, blood\u2013cerebrospinal fluid barrier, and similar fluid\u2013brain barriers separate the peripheral immune system from the neuroimmune system which protects the brain."], ["One of its earliest massive implementations was brought about by Egyptians against the British occupation in the 1919 Revolution. Civil disobedience is one of the many ways people have rebelled against what they deem to be unfair laws. It has been used in many nonviolent resistance movements in India (Gandhi's campaigns for independence from the British Empire), in Czechoslovakia's Velvet Revolution and in East Germany to oust their communist governments, In South Africa in the fight against apartheid, in the American Civil Rights Movement, in the Singing Revolution to bring independence to the Baltic countries from the Soviet Union, recently with the 2003 Rose Revolution in Georgia and the 2004 Orange Revolution in Ukraine, among other various movements worldwide."], ["Construction is the process of constructing a building or infrastructure. Construction differs from manufacturing in that manufacturing typically involves mass production of similar items without a designated purchaser, while construction typically takes place on location for a known client. Construction as an industry comprises six to nine percent of the gross domestic product of developed countries. Construction starts with planning,[citation needed] design, and financing and continues until the project is built and ready for use."], ["Private schools, also known as independent schools, non-governmental, or nonstate schools, are not administered by local, state or national governments; thus, they retain the right to select their students and are funded in whole or in part by charging their students tuition, rather than relying on mandatory taxation through public (government) funding; at some private schools students may be able to get a scholarship, which makes the cost cheaper, depending on a talent the student may have (e.g. sport scholarship, art scholarship, academic scholarship), financial need, or tax credit scholarships that might be available."], ["Established originally by the Massachusetts legislature and soon thereafter named for John Harvard (its first benefactor), Harvard is the United States' oldest institution of higher learning, and the Harvard Corporation (formally, the President and Fellows of Harvard College) is its first chartered corporation. Although never formally affiliated with any denomination, the early College primarily trained Congregationalist and Unitarian clergy. Its curriculum and student body were gradually secularized during the 18th century, and by the 19th century Harvard had emerged as the central cultural establishment among Boston elites. Following the American Civil War, President Charles W. Eliot's long tenure (1869\u20131909) transformed the college and affiliated professional schools into a modern research university; Harvard was a founding member of the Association of American Universities in 1900. James Bryant Conant led the university through the Great Depression and World War II and began to reform the curriculum and liberalize admissions after the war. The undergraduate college became coeducational after its 1977 merger with Radcliffe College."], ["Jacksonville is the largest city by population in the U.S. state of Florida, and the largest city by area in the contiguous United States. It is the county seat of Duval County, with which the city government consolidated in 1968. Consolidation gave Jacksonville its great size and placed most of its metropolitan population within the city limits; with an estimated population of 853,382 in 2014, it is the most populous city proper in Florida and the Southeast, and the 12th most populous in the United States. Jacksonville is the principal city in the Jacksonville metropolitan area, with a population of 1,345,596 in 2010."], ["A study by the World Institute for Development Economics Research at United Nations University reports that the richest 1% of adults alone owned 40% of global assets in the year 2000. The three richest people in the world possess more financial assets than the lowest 48 nations combined. The combined wealth of the \"10 million dollar millionaires\" grew to nearly $41 trillion in 2008. A January 2014 report by Oxfam claims that the 85 wealthiest individuals in the world have a combined wealth equal to that of the bottom 50% of the world's population, or about 3.5 billion people. According to a Los Angeles Times analysis of the report, the wealthiest 1% owns 46% of the world's wealth; the 85 richest people, a small part of the wealthiest 1%, own about 0.7% of the human population's wealth, which is the same as the bottom half of the population. More recently, in January 2015, Oxfam reported that the wealthiest 1 percent will own more than half of the global wealth by 2016. An October 2014 study by Credit Suisse also claims that the top 1% now own nearly half of the world's wealth and that the accelerating disparity could trigger a recession. In October 2015, Credit Suisse published a study which shows global inequality continues to increase, and that half of the world's wealth is now in the hands of those in the top percentile, whose assets each exceed $759,900. A 2016 report by Oxfam claims that the 62 wealthiest individuals own as much wealth as the poorer half of the global population combined. Oxfam's claims have however been questioned on the basis of the methodology used: by using net wealth (adding up assets and subtracting debts), the Oxfam report, for instance, finds that there are more poor people in the United States and Western Europe than in China (due to a greater tendency to take on debts).[unreliable source?][unreliable source?] Anthony Shorrocks, the lead author of the Credit Suisse report which is one of the sources of Oxfam's data, considers the criticism about debt to be a \"silly argument\" and \"a non-issue . . . a diversion.\""], ["Doctor Who is a British science-fiction television programme produced by the BBC since 1963. The programme depicts the adventures of the Doctor, a Time Lord\u2014a space and time-travelling humanoid alien. He explores the universe in his TARDIS, a sentient time-travelling space ship. Its exterior appears as a blue British police box, which was a common sight in Britain in 1963 when the series first aired. Accompanied by companions, the Doctor combats a variety of foes, while working to save civilisations and help people in need."], ["The University of Chicago (UChicago, Chicago, or U of C) is a private research university in Chicago. The university, established in 1890, consists of The College, various graduate programs, interdisciplinary committees organized into four academic research divisions and seven professional schools. Beyond the arts and sciences, Chicago is also well known for its professional schools, which include the Pritzker School of Medicine, the University of Chicago Booth School of Business, the Law School, the School of Social Service Administration, the Harris School of Public Policy Studies, the Graham School of Continuing Liberal and Professional Studies and the Divinity School. The university currently enrolls approximately 5,000 students in the College and around 15,000 students overall."], ["The Yuan dynasty (Chinese: \u5143\u671d; pinyin: Yu\u00e1n Ch\u00e1o), officially the Great Yuan (Chinese: \u5927\u5143; pinyin: D\u00e0 Yu\u00e1n; Mongolian: Yehe Yuan Ulus[a]), was the empire or ruling dynasty of China established by Kublai Khan, leader of the Mongolian Borjigin clan. Although the Mongols had ruled territories including today's North China for decades, it was not until 1271 that Kublai Khan officially proclaimed the dynasty in the traditional Chinese style. His realm was, by this point, isolated from the other khanates and controlled most of present-day China and its surrounding areas, including modern Mongolia and Korea. It was the first foreign dynasty to rule all of China and lasted until 1368, after which its Genghisid rulers returned to their Mongolian homeland and continued to rule the Northern Yuan dynasty. Some of the Mongolian Emperors of the Yuan mastered the Chinese language, while others only used their native language (i.e. Mongolian) and the 'Phags-pa script."], ["Kenya (/\u02c8k\u025bnj\u0259/; locally [\u02c8k\u025b\u0272a] ( listen)), officially the Republic of Kenya, is a country in Africa and a founding member of the East African Community (EAC). Its capital and largest city is Nairobi. Kenya's territory lies on the equator and overlies the East African Rift covering a diverse and expansive terrain that extends roughly from Lake Victoria to Lake Turkana (formerly called Lake Rudolf) and further south-east to the Indian Ocean. It is bordered by Tanzania to the south, Uganda to the west, South Sudan to the north-west, Ethiopia to the north and Somalia to the north-east. Kenya covers 581,309 km2 (224,445 sq mi), and had a population of approximately 45 million people in July 2014."], ["The Intergovernmental Panel on Climate Change (IPCC) is a scientific intergovernmental body under the auspices of the United Nations, set up at the request of member governments. It was first established in 1988 by two United Nations organizations, the World Meteorological Organization (WMO) and the United Nations Environment Programme (UNEP), and later endorsed by the United Nations General Assembly through Resolution 43/53. Membership of the IPCC is open to all members of the WMO and UNEP. The IPCC produces reports that support the United Nations Framework Convention on Climate Change (UNFCCC), which is the main international treaty on climate change. The ultimate objective of the UNFCCC is to \"stabilize greenhouse gas concentrations in the atmosphere at a level that would prevent dangerous anthropogenic [i.e., human-induced] interference with the climate system\". IPCC reports cover \"the scientific, technical and socio-economic information relevant to understanding the scientific basis of risk of human-induced climate change, its potential impacts and options for adaptation and mitigation.\""], ["Chloroplasts' main role is to conduct photosynthesis, where the photosynthetic pigment chlorophyll captures the energy from sunlight and converts it and stores it in the energy-storage molecules ATP and NADPH while freeing oxygen from water. They then use the ATP and NADPH to make organic molecules from carbon dioxide in a process known as the Calvin cycle. Chloroplasts carry out a number of other functions, including fatty acid synthesis, much amino acid synthesis, and the immune response in plants. The number of chloroplasts per cell varies from 1 in algae up to 100 in plants like Arabidopsis and wheat."], ["A prime number (or a prime) is a natural number greater than 1 that has no positive divisors other than 1 and itself. A natural number greater than 1 that is not a prime number is called a composite number. For example, 5 is prime because 1 and 5 are its only positive integer factors, whereas 6 is composite because it has the divisors 2 and 3 in addition to 1 and 6. The fundamental theorem of arithmetic establishes the central role of primes in number theory: any integer greater than 1 can be expressed as a product of primes that is unique up to ordering. The uniqueness in this theorem requires excluding 1 as a prime because one can include arbitrarily many instances of 1 in any factorization, e.g., 3, 1 \u00b7 3, 1 \u00b7 1 \u00b7 3, etc. are all valid factorizations of 3."], ["The Rhine (Romansh: Rein, German: Rhein, French: le Rhin, Dutch: Rijn) is a European river that begins in the Swiss canton of Graub\u00fcnden in the southeastern Swiss Alps, forms part of the Swiss-Austrian, Swiss-Liechtenstein border, Swiss-German and then the Franco-German border, then flows through the Rhineland and eventually empties into the North Sea in the Netherlands. The biggest city on the river Rhine is Cologne, Germany with a population of more than 1,050,000 people. It is the second-longest river in Central and Western Europe (after the Danube), at about 1,230 km (760 mi),[note 2][note 1] with an average discharge of about 2,900 m3/s (100,000 cu ft/s)."], ["Following a referendum in 1997, in which the Scottish electorate voted for devolution, the current Parliament was convened by the Scotland Act 1998, which sets out its powers as a devolved legislature. The Act delineates the legislative competence of the Parliament \u2013 the areas in which it can make laws \u2013 by explicitly specifying powers that are \"reserved\" to the Parliament of the United Kingdom. The Scottish Parliament has the power to legislate in all areas that are not explicitly reserved to Westminster. The British Parliament retains the ability to amend the terms of reference of the Scottish Parliament, and can extend or reduce the areas in which it can make laws. The first meeting of the new Parliament took place on 12 May 1999."], ["Islamism, also known as Political Islam (Arabic: \u0625\u0633\u0644\u0627\u0645 \u0633\u064a\u0627\u0633\u064a\u200e isl\u0101m siy\u0101s\u012b), is an Islamic revival movement often characterized by moral conservatism, literalism, and the attempt \"to implement Islamic values in all spheres of life.\" Islamism favors the reordering of government and society in accordance with the Shari'a. The different Islamist movements have been described as \"oscillating between two poles\": at one end is a strategy of Islamization of society through state power seized by revolution or invasion; at the other \"reformist\" pole Islamists work to Islamize society gradually \"from the bottom up\". The movements have \"arguably altered the Middle East more than any trend since the modern states gained independence\", redefining \"politics and even borders\" according to one journalist (Robin Wright)."], ["Imperialism is a type of advocacy of empire. Its name originated from the Latin word \"imperium\", which means to rule over large territories. Imperialism is \"a policy of extending a country's power and influence through colonization, use of military force, or other means\". Imperialism has greatly shaped the contemporary world. It has also allowed for the rapid spread of technologies and ideas. The term imperialism has been applied to Western (and Japanese) political and economic dominance especially in Asia and Africa in the 19th and 20th centuries. Its precise meaning continues to be debated by scholars. Some writers, such as Edward Said, use the term more broadly to describe any system of domination and subordination organised with an imperial center and a periphery."], ["The United Methodist Church (UMC) is a mainline Protestant Methodist denomination. In the 19th century its main predecessor was a leader in Evangelicalism. Founded in 1968 by the union of the Methodist Church (USA) and the Evangelical United Brethren Church, the UMC traces its roots back to the revival movement of John and Charles Wesley in England as well as the Great Awakening in the United States. As such, the church's theological orientation is decidedly Wesleyan. It embraces both liturgical and evangelical elements."], ["The French and Indian War (1754\u20131763) was the North American theater of the worldwide Seven Years' War. The war was fought between the colonies of British America and New France, with both sides supported by military units from their parent countries of Great Britain and France, as well as Native American allies. At the start of the war, the French North American colonies had a population of roughly 60,000 European settlers, compared with 2 million in the British North American colonies. The outnumbered French particularly depended on the Indians. Long in conflict, the metropole nations declared war on each other in 1756, escalating the war from a regional affair into an intercontinental conflict."], ["Philosophers in antiquity used the concept of force in the study of stationary and moving objects and simple machines, but thinkers such as Aristotle and Archimedes retained fundamental errors in understanding force. In part this was due to an incomplete understanding of the sometimes non-obvious force of friction, and a consequently inadequate view of the nature of natural motion. A fundamental error was the belief that a force is required to maintain motion, even at a constant velocity. Most of the previous misunderstandings about motion and force were eventually corrected by Galileo Galilei and Sir Isaac Newton. With his mathematical insight, Sir Isaac Newton formulated laws of motion that were not improved-on for nearly three hundred years. By the early 20th century, Einstein developed a theory of relativity that correctly predicted the action of forces on objects with increasing momenta near the speed of light, and also provided insight into the forces produced by gravitation and inertia."]], "questions": [["Which NFL team represented the AFC at Super Bowl 50?", "Which NFL team represented the NFC at Super Bowl 50?", "Where did Super Bowl 50 take place?", "Which NFL team won Super Bowl 50?", "What color was used to emphasize the 50th anniversary of the Super Bowl?", "What was the theme of Super Bowl 50?", "What day was the game played on?", "What is the AFC short for?", "What was the theme of Super Bowl 50?", "What does AFC stand for?", "What day was the Super Bowl played on?", "Who won Super Bowl 50?", "What venue did Super Bowl 50 take place in?", "What city did Super Bowl 50 take place in?", "If Roman numerals were used, what would Super Bowl 50 have been called?", "Super Bowl 50 decided the NFL champion for what season?", "What year did the Denver Broncos secure a Super Bowl title for the third time?", "What city did Super Bowl 50 take place in?", "What stadium did Super Bowl 50 take place in?", "What was the final score of Super Bowl 50? ", "What month, day and year did Super Bowl 50 take place? ", "What year was Super Bowl 50?", "What team was the AFC champion?", "What team was the NFC champion?", "Who won Super Bowl 50?", "Super Bowl 50 determined the NFL champion for what season?", "Which team won Super Bowl 50.", "Where was Super Bowl 50 held?", "The name of the NFL championship game is?", "What 2015 NFL team one the AFC playoff?"], ["What was Maria Curie the first female recipient of?", "What year was Casimir Pulaski born in Warsaw?", "Who was one of the most famous people born in Warsaw?", "Who was Fr\u00e9d\u00e9ric Chopin?", "How old was Chopin when he moved to Warsaw with his family?"], ["In what country is Normandy located?", "When were the Normans in Normandy?", "From which countries did the Norse originate?", "Who was the Norse leader?", "What century did the Normans first gain their separate identity?"], ["In what year was Nikola Tesla born? ", "What was Nikola Tesla's ethnicity?", "In what year did Tesla die? ", "When was Nikola Tesla born?", "In what year did Tesla die?", "What is Tesla's home country?", "What does AC stand for?"], ["What branch of theoretical computer science deals with broadly classifying computational problems by difficulty and class of relationship?", "By what main attribute are computational problems classified utilizing computational complexity theory? ", "What is the term for a task that generally lends itself to being solved by a computer?"], ["What is a course of study called?", "What is another name to describe the science of teaching?", "Where do most teachers get their credentials from?", "What can a teacher use to help students learn?", "Where is a teacher most likely to be teaching at?"], ["Of what nationality was Martin Luther?", "When did Martin Luther die?", "What organization's teaching did Luther reject?", "What did the Church claim could be avoided with money?", "What did the Church do when Luther refused to retract his writings?"], ["What is Southern California often abbreviated as?", "Despite being traditionall described as \"eight counties\", how many counties does this region actually have?", "What is a major importance of Southern California in relation to California and the United States?", "What are the ties that best described what the \"eight counties\" are based on?", "The reasons for the las two counties to be added are based on what?"], ["What company was formed by the merger of Sky Television and British Satellite Broadcasting?", "Who is the UK's largest digital subscription television company?", "What year did BSkyB acquire Sky Italia?", "What is the name of the holding company for BSkyB?", "What is the name of the United Kingdom operation for BSkyB?"], ["What kind of economy does Victoria have?", "Where according to gross state product does Victoria rank in Australia?", "At what rank does GPS per capita set Victoria?", "What city in Victoria is called the sporting capital of Australia?", "What is the largest stadium in Australia?"], ["Where was France's Huguenot population largely centered?", "What was the proportion of Huguenots to Catholics at their peak?", "When were the Wars of Religion fought?", "What treaty ended the Wars of Religion?", "What did this agreement do?"], ["Along with geothermal and nuclear, what is a notable non-combustion heat source?", "What ideal thermodynamic cycle analyzes the process by which steam engines work?", "In the Rankine cycle, what does water turn into when heated?", "At what pressure is water heated in the Rankine cycle?", "What types of engines are steam engines?"], ["The atomic number of the periodic table for oxygen?", "What is the second most abundant element?", "How many atoms combine to form dioxygen?", "Roughly, how much oxygen makes up the Earth crust?", "Which gas makes up 20.8% of the Earth's atmosphere?", "How much of the earth's atmosphere is diatomic oxygen?", "What element makes up almost half of the earth's crust by mass?", "What is the atomic number for oxygen?", "Are atmospheric oxygen levels going up, down, or staying the same?", "What are the three most abundent elements of the universe by mass?", "What is the atomic number of the element oxygen?", "Of what group in the periodic table is oxygen a member?", "What type of compounds does oxygen most commonly form?", "Compared to other elements, how abundant does oxygen rank?", "Under normal conditions, what do two atoms of oxygen form?"], ["When did the 1973 oil crisis begin?", "What was the price of oil in March of 1974?", "When was the second oil crisis?", "What was another term used for the oil crisis?", "Who proclaimed the oil embargo?"], ["What project put the first Americans into space?", "What program was created to carry out these projects and missions?", "What year did the first manned Apollo flight occur?", "What President is credited with the original notion of putting Americans in space?", "How many people were on the project that followed Project Mercury?"], ["What are the three sources of European Union law?", "What is European Union Law?", "What are the main sources of primary law?", "What are the secondary sources of primary law?", "What are the two bodies that make up the European Union's legislature?", "What is European Union law?", "What effect does European Union law have on laws of member states?", "What are the three sources of European Union law?", "What are the main legislative bodies of the European Union?", "What are the three main sources of European Union law?", "What are the main sources of primary law?", "What is the legislature of the European Union comprised of?", "How many sources of European Union law are there?"], ["Which name is also used to describe the Amazon rainforest in English?", "How many square kilometers of rainforest is covered in the basin?", "How many nations control this region in total?", "How many nations contain \"Amazonas\" in their names?", "What percentage does the Amazon represents in rainforests on the planet?", "What is the Dutch word for the Amazon rainforest?", "What rainforest covers the majority of the Amazon basin in South America?", "In what country can most of the Amazon rainforest be found?", "The Amazon rainforest makes up what amount of Earth's rainforests?", "How many species of trees can be found in the Amazon rainforest?", "What kind of forest is the Amazon rainforest?", "How many square kilometers is the Amazon Basin?", "How many nations are within the Amazon Basin?", "Which nation contains the majority of the amazon forest?", "What is the estimate for the amount of tree species in the amazon tropical rain forest?"], ["What are Ctenophora commonly known as?", "Where do Ctenophora live?", "What size are adult Ctenophora?", "What is a ctenophora?", "What does the ctenophora use to swim?", "What does ctenophora use for digestion and respiration?", "How large can ctenophora grow?", "What is the most distinctive feature of ctenophora?", "What are ctenophora commonly known as?", "How big can ctenophora grow?", "What does ctenophora rely on for digestion and respiration?", "What does ctenophora mean in Greek?", "Where do ctenophora live?"], ["Which city is the fifth-largest city in California?", "How far is Fresno from Los Angeles?", "What does the name Fresno mean in Spanish?", "What is featured on the city of Fresno's city flag?", "How do you pronounce Fresno?"], ["What did Paul Baran develop ", "What did Distributed Adaptive Message Block Switching do", "What did this concept contradict ", "What is Donald Davies credited with ", "What did Paul Baran develop in the late 1950's", "What was the goal of the system ", "Who is credited with the modern name for this system "], ["Where did the black death originate?", "How did the black death make it to the Mediterranean and Europe?", "How much of the European population did the black death kill?", "When did the world's population finally recover from the black death?", "For how long did the plague stick around?"], ["An igneous rock is a rock that crystallizes from what?", "Sedimentary rock can be turned into which of the three types of rock?", "When the three types of rock are re-melted what is formed? ", "What are the three major types of rock? ", "What changes the mineral content of a rock?"], ["How many miles south of Edinburgh is Newcastle?", "How many miles from the north Sea is Newcastle?", "What network is Newcastle a member of?", "What county was Newcastle a part of until 1400?", "What's the regional nickname for Newcastle and its surrounding area?"], ["where is the Victoria and Albert Museum located?", "how many permanent objects are located there?", "when was the Victoria and Albert museum founded?", "Who is the museum named for?", "In which London borough is the Victoria and Albert Museum located?", "In what year was the Victoria and Albert Museum founded?", "Which monarchs was the Victoria and Albert Museum named after?", "Which department sponsors the Victoria and Albert Museum?", "In which year did the museum started charging free admission fees?"], ["What company owns the American Broadcasting Company?", "In what year did ABC stylize it's logo as abc?", "In what borough of New York City is ABC headquartered?", "On what streets is the ABC headquarters located", "Disney-ABC Television Group is a subsidiary of what division of the Walt Disney Company?"], ["What do we call the empire that Genghis Khan founded?", "Who did Genghis Khan unite before he began conquering the rest of Eurasia?", "In which regions in particular did Genghis Khan's armies massacre civilians?", "What areas did Genghis Khan control at the end of his life?", "Which other empires or dynasties did Genghis Khan conquer?"], ["What word is the word pharmacy taken from?", "What goods were sold in a pharma?", "What did the Greek root pharmakos imply?", "How would the word apothecary be viewed by contemporary English speakers?", "What else was used by pharmas?"], ["What is the immune system?", "What does the immune system protect against?", "What are two of its subsystems?", "What is the subsystem that protects the human brain?", "What is the immune system?", "What does the immune system protect against?", "What are two of its subsystems?", "The immune system protects organisms against what?", "What are the agents the immune system detects known as?", "Which part of the immune system protects the brain?", "What separates the neuroimmune system and peripheral immune system in humans?", "What are the agents detected by the immune system called?", "What are the two major subsystems of the immune system?", "What are the two different types of immunity?", "What is the immune system of the brained known as?"], ["What is it called when people in society rebel against laws they think are unfair?", "What is an example of major civil disobedience in South Africa?", "What was the the movement called that brought Baltic countries independence from the Soviet Union?", "In 2004 the Orange revolution occurred in what country?", "Where was the location of the 2003 Rose revolution?", "Which people brought forward one of the earliest examples of Civil Disobedience?", "Who was one of the earliest examples of Civil Disobedience against?", "Why do people chose civil disobedience to protest?", "What does civil disobedience protest against?", "What civil rights movement in the US was known for it's disobedience?"], ["What is the process of constructing a building or infrastructure?", "What typically involves mass production of similar items without a designated purchaser?", "What percentile of gross domestic product is construction comprised of?", "What three things are needed for construction to take place?", "Construction takes place on location for who?"], ["Along with non-governmental and nonstate schools, what is another name for private schools?", "Along with sport and art, what is a type of talent scholarship?", "Rather than taxation, what are private schools largely funded by?", "What right do private schools have that public schools don't?"], ["What individual is the school named after?", "When did the undergraduate program become coeducational?", "What was the name of the leader through the Great Depression and World War II?", "What organization did Harvard found in 1900?", "What president of the university transformed it into a modern research university?"], ["Which Florida city has the biggest population?", "What was the population Jacksonville city as of 2010?", "Based on population alone, what is Jacksonville's ranking in the United States?", "In which county does Jacksonville reside?", "What year did consolidation cause Jacksonville to become part of Duval County?"], ["What percentage of global assets does the richest 1% of people have?", "According to Oxfam, the 85 richest people have wealth equal to how many average people?", "In order to be considered in the top percentile, a person would need to amass how much money each year?", "What has caused Oxfam's findings to be questioned?", "Why does Oxfam and Credit Suisse believe their findings are being doubted?", "What percent of the global assets in 2000 were owned by just 1% of adults?", "What do the three richest people in the world posses more of than the lowest 48 nations together?", "How much was the combined wealth of the \"10 Million dollar millionaires\" in 2008?", "How much of the global wealth will the wealthiest 1 percent own by 2016?", "Why are there more poor people in the United States and Europe than China?"], ["Who is the producer of Doctor Who?", "What year did Doctor Who first show on TV?", "What is Doctor Who's space ship called?", "What does the outside of the Tardis resemble?", "What type/genre of TV show is Doctor Who?"], ["What kind of university is the University of Chicago?", "When was the University of Chicago established?", "How many professional schools does the University of Chicago have?", "How many academic research divisions does the University of Chicago have?", "How many students does the University of Chicago have enlisted?"], ["What is the Chinese name for the Yuan dynasty?", "What is the Yuan dynasty's official name?", "Who started the Yuan dynasty?", "Who led the Mongolian Borjigin clan?", "When did Khan formally declare the Yuan dynasty?"], ["Where is Kenya located?", "What is Kenya a founding member of?", "What is the capitol of Kenya?", "What country boarders the south of Kenya?", "What was the population of Kenya in 2014?"], ["What organization is the IPCC a part of?", "What UN organizations established the IPCC?", "What does the UN want to stabilize?", "What is the UN's climate change treaty?", "What UN resolution endorsed the IPCC?"], ["What is the primary purpose of chloroplasts?", "What does ATP store?", "What does NADPH store?", "What is the process of turning CO2 into organic molecules called?", "How many chloroplasts per cell does algae have?"], ["What is the only divisor besides 1 that a prime number can have?", "What are numbers greater than 1 that can be divided by 3 or more numbers called?", "What theorem defines the main role of primes in number theory?", "Any number larger than 1 can be represented as a product of what?", "Why must one be excluded in order to preserve the uniqueness of the fundamental theorem?"], ["Where does the Rhine begin? ", "Where does the Rhine empty?", "What is the largest city the Rhine runs through? ", "What river is larger than the Rhine?", "How long is the Rhine?", "Where is the Rhine? ", "What country does the Rhine empty?", "How long is the Rhine? "], ["When was the current parliament of Scotland convened?", "What act set out the Parliament's powers as a devolved legislature?", "The legislative competence of the Parliament species what areas?", "To what body are certain powers explicitly specified as being reserved for?", "The Scottish Parliament may legislate as it pleases as long as the powers aren't already reserved by where?"], ["What is an Islamic revival movement?", "What aspects of life does Islamism seek to integrate itself into?", "What goal does Islamism have when it comes to society and government?", "What have the two different Islamist movements been described as oscillating between?", "One strategy of Islamization is to seize power by what methods?"], ["The word imperialism has it's origins in which ancient language? ", "By what means is imperialism usually administered?", "The term imperialism has been applied to western countries, and which eastern county?", "Imperialism is responsible for the rapid spread of what?"], ["What does UMC stand for?", "What is the United Methodist Church?", "When was the UMC founded?", "Who founded the UMC?", "What is the church's theological orientation?"], ["When was the French and Indian War?", "Who fought in the French and Indian war?", "How many people were in French North American Colonies?", "How many people were in British North American Colonies?"], ["What concept did philosophers in antiquity use to study simple machines?", "What was the belief that maintaining motion required force?", "Who had mathmatical insite?", "How long did it take to improve on Sir Isaac Newton's laws of motion?", "Who develped the theory of relativity?"]]} \ No newline at end of file diff --git a/lite/examples/bert_qa/android/lib_task_api/src/main/java/org/tensorflow/lite/examples/bertqa/ml/Answer.java b/lite/examples/bert_qa/android/lib_task_api/src/main/java/org/tensorflow/lite/examples/bertqa/ml/Answer.java new file mode 100644 index 00000000000..2d5f072276e --- /dev/null +++ b/lite/examples/bert_qa/android/lib_task_api/src/main/java/org/tensorflow/lite/examples/bertqa/ml/Answer.java @@ -0,0 +1,39 @@ +package org.tensorflow.lite.examples.bertqa.ml; + +/** QA Answer class. */ +public class Answer implements Comparable{ + public Pos pos; + public String text; + + public Answer(String text, Pos pos) { + this.text = text; + this.pos = pos; + } + + public Answer(String text, int start, int end, float logit) { + this(text, new Pos(start, end, logit)); + } + + @Override + public int compareTo(Answer other) { + return Float.compare(other.pos.logit, this.pos.logit); + } + + /** Position and related information from the model. */ + public static class Pos implements Comparable { + public int start; + public int end; + public float logit; + + public Pos(int start, int end, float logit) { + this.start = start; + this.end = end; + this.logit = logit; + } + + @Override + public int compareTo(Pos other) { + return Float.compare(other.logit, this.logit); + } + } +} diff --git a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ml/QaClient.java b/lite/examples/bert_qa/android/lib_task_api/src/main/java/org/tensorflow/lite/examples/bertqa/ml/QaClient.java similarity index 71% rename from lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ml/QaClient.java rename to lite/examples/bert_qa/android/lib_task_api/src/main/java/org/tensorflow/lite/examples/bertqa/ml/QaClient.java index 184f079634f..2d8020c8f34 100644 --- a/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ml/QaClient.java +++ b/lite/examples/bert_qa/android/lib_task_api/src/main/java/org/tensorflow/lite/examples/bertqa/ml/QaClient.java @@ -6,8 +6,9 @@ import org.tensorflow.lite.task.text.qa.BertQuestionAnswerer; import org.tensorflow.lite.task.text.qa.QaAnswer; - import java.io.IOException; +import java.util.ArrayList; +import java.util.Collections; import java.util.List; /** @@ -49,9 +50,15 @@ public void unload() { /** * Run inference and predict the possible answers. */ - public List predict(String questionToAsk, String contextOfTheQuestion) { + public List predict(String questionToAsk, String contextOfTheQuestion) { List apiResult = answerer.answer(contextOfTheQuestion, questionToAsk); - return apiResult; + List answers = new ArrayList<>(apiResult.size()); + for (int i = 0; i < apiResult.size(); i++){ + QaAnswer qaAnswer = apiResult.get(i); + answers.add(new Answer(qaAnswer.text, qaAnswer.pos.start, qaAnswer.pos.end, qaAnswer.pos.logit)); + } + Collections.sort(answers); + return answers; } } diff --git a/lite/examples/bert_qa/android/settings.gradle b/lite/examples/bert_qa/android/settings.gradle index 0745a61f07e..2cc90a3ad1b 100644 --- a/lite/examples/bert_qa/android/settings.gradle +++ b/lite/examples/bert_qa/android/settings.gradle @@ -1,2 +1,4 @@ rootProject.name = 'TFLite BERT Q&A Demo App' include ':app' +include ':lib_task_api' +include ':lib_interpreter' From 92dbc3fe2dbd53870890a8d4aab2ba870c72f17c Mon Sep 17 00:00:00 2001 From: Sunit Roy <2703iamsry@gmail.com> Date: Sat, 24 Jul 2021 03:32:57 +0530 Subject: [PATCH 5/7] MinSDK version & Readme update --- lite/examples/bert_qa/android/README.md | 38 +++++++++++++++++-- .../examples/bert_qa/android/app/build.gradle | 2 +- 2 files changed, 36 insertions(+), 4 deletions(-) diff --git a/lite/examples/bert_qa/android/README.md b/lite/examples/bert_qa/android/README.md index 947a24947b7..9d47747730b 100644 --- a/lite/examples/bert_qa/android/README.md +++ b/lite/examples/bert_qa/android/README.md @@ -26,7 +26,7 @@ For more information, refer to the [BERT github page][BERT]. ## Build the demo using Android Studio - + ### Prerequisites * If you don't have already, install @@ -38,7 +38,7 @@ For more information, refer to the [BERT github page][BERT]. - SDK Build Tools 29.0.2 or higher. * You need an Android device or Android emulator and Android development - environment with minimum API 15. + environment with minimum API 21. ### Building @@ -65,6 +65,38 @@ For more information, refer to the [BERT github page][BERT]. * Click `Run` to run the demo app on your Android device. +#### Switch between inference solutions (Task library vs TFLite Interpreter) + +This BERT QA Android reference app demonstrates two implementation +solutions: + +(1) +[`lib_task_api`](https://github.com/SunitRoy2703/examples/tree/bertQa-android-task-lib/lite/examples/bert_qa/android/lib_task_api) +that leverages the out-of-box API from the +[TensorFlow Lite Task Library](https://www.tensorflow.org/lite/inference_with_metadata/task_library/bert_question_answerer); + +(2) +[`lib_interpreter`](https://github.com/SunitRoy2703/examples/tree/bertQa-android-task-lib/lite/examples/bert_qa/android/lib_interpreter) +that creates the custom inference pipleline using the +[TensorFlow Lite Interpreter Java API](https://www.tensorflow.org/lite/guide/inference#load_and_run_a_model_in_java). + +The [`build.gradle`](app/build.gradle) inside `app` folder shows how to change +`flavorDimensions "tfliteInference"` to switch between the two solutions. + +Inside **Android Studio**, you can change the build variant to whichever one you +want to build and run—just go to `Build > Select Build Variant` and select one +from the drop-down menu. See +[configure product flavors in Android Studio](https://developer.android.com/studio/build/build-variants#product-flavors) +for more details. + +For gradle CLI, running `./gradlew build` can create APKs for both solutions +under `app/build/outputs/apk`. + +*Note: If you simply want the out-of-box API to run the app, we recommend +`lib_task_api`for inference. If you want to customize your own models and +control the detail of inputs and outputs, it might be easier to adapt your model +inputs and outputs by using `lib_interpreter`.* + ## Build the demo using gradle (command line) ### Building and Installing @@ -91,4 +123,4 @@ folder. [BERT]: https://github.com/google-research/bert "Bert" [SQuAD]: https://rajpurkar.github.io/SQuAD-explorer/ "SQuAD" -[MobileBERT]:https://tfhub.dev/tensorflow/tfjs-model/mobilebert/1 "MobileBERT" \ No newline at end of file +[MobileBERT]:https://tfhub.dev/tensorflow/tfjs-model/mobilebert/1 "MobileBERT" diff --git a/lite/examples/bert_qa/android/app/build.gradle b/lite/examples/bert_qa/android/app/build.gradle index ae563af28a0..1695318e544 100644 --- a/lite/examples/bert_qa/android/app/build.gradle +++ b/lite/examples/bert_qa/android/app/build.gradle @@ -5,7 +5,7 @@ android { buildToolsVersion "29.0.2" defaultConfig { applicationId "org.tensorflow.lite.examples.bertapp" - minSdkVersion 26 + minSdkVersion 21 targetSdkVersion 29 versionCode 1 versionName "1.0" From fe8632b8b99ca27a4b9b5e35471e9d00cd936b36 Mon Sep 17 00:00:00 2001 From: Sunit Roy <2703iamsry@gmail.com> Date: Sun, 25 Jul 2021 02:35:20 +0530 Subject: [PATCH 6/7] Update Explore_The_Code & gitignore --- lite/examples/bert_qa/android/.gitignore | 4 +- .../bert_qa/android/EXPLORE_THE_CODE.md | 169 ++++++++++++++++++ .../lib_interpreter/src/main/assets/qa.json | 1 - .../lib_task_api/src/main/assets/qa.json | 1 - .../lite/examples/bertqa/ml/QaClient.java | 2 +- 5 files changed, 173 insertions(+), 4 deletions(-) delete mode 100644 lite/examples/bert_qa/android/lib_interpreter/src/main/assets/qa.json delete mode 100644 lite/examples/bert_qa/android/lib_task_api/src/main/assets/qa.json diff --git a/lite/examples/bert_qa/android/.gitignore b/lite/examples/bert_qa/android/.gitignore index bbacc76e892..e29db5dadda 100644 --- a/lite/examples/bert_qa/android/.gitignore +++ b/lite/examples/bert_qa/android/.gitignore @@ -1 +1,3 @@ -/app/src/main/assets/ \ No newline at end of file +/app/src/main/assets/ +/lib_interpreter/src/main/assets/ +/lib_task_api/src/main/assets/ \ No newline at end of file diff --git a/lite/examples/bert_qa/android/EXPLORE_THE_CODE.md b/lite/examples/bert_qa/android/EXPLORE_THE_CODE.md index e69de29bb2d..e23c9ea1187 100644 --- a/lite/examples/bert_qa/android/EXPLORE_THE_CODE.md +++ b/lite/examples/bert_qa/android/EXPLORE_THE_CODE.md @@ -0,0 +1,169 @@ +# TensorFlow Lite BERT QA Android example + +This document walks through the code of a simple Android mobile application that +demonstrates +[BERT Question and Answer](https://www.tensorflow.org/lite/examples/bert_qa/overview). + +## Explore the code + +The app is written entirely in Java and uses the TensorFlow Lite +[Java library](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/java) +for performing BERT Question and Answer. + +We're now going to walk through the most important parts of the sample code. + +### Get the question and the context of the question + +This mobile application gets the question and the context of the question using the functions defined in the +file +[`QaActivity.java`](https://github.com/tensorflow/examples/blob/master/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/ui/QaActivity.java). + + +### Answerer + +This BERT QA Android reference app demonstrates two implementation +solutions, +[`lib_task_api`](https://github.com/SunitRoy2703/examples/tree/bertQa-android-task-lib/lite/examples/bert_qa/android/lib_task_api) +that leverages the out-of-box API from the +[TensorFlow Lite Task Library](https://www.tensorflow.org/lite/inference_with_metadata/task_library/bert_question_answerer), +and +[`lib_interpreter`](https://github.com/SunitRoy2703/examples/tree/bertQa-android-task-lib/lite/examples/bert_qa/android/lib_interpreter) +that creates the custom inference pipleline using the +[TensorFlow Lite Interpreter Java API](https://www.tensorflow.org/lite/guide/inference#load_and_run_a_model_in_java). + +Both solutions implement the file `QaClient.java` (see +[the one in lib_task_api](https://github.com/SunitRoy2703/examples/blob/bertQa-android-task-lib/lite/examples/bert_qa/android/lib_task_api/src/main/java/org/tensorflow/lite/examples/bertqa/ml/QaClient.java) +and +[the one in lib_interpreter](https://github.com/SunitRoy2703/examples/blob/bertQa-android-task-lib/lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/ml/QaClient.java) +that contains most of the complex logic for processing the text input and +running inference. + +#### Using the TensorFlow Lite Task Library + +Inference can be done using just a few lines of code with the +[`BertQuestionAnswerer`](https://www.tensorflow.org/lite/inference_with_metadata/task_library/bert_question_answerer) +in the TensorFlow Lite Task Library. + +##### Load model and create BertQuestionAnswerer + +`BertQuestionAnswerer` expects a model populated with the +[model metadata](https://www.tensorflow.org/lite/convert/metadata) and the label +file. See the +[model compatibility requirements](https://www.tensorflow.org/lite/inference_with_metadata/task_library/bert_question_answerer#model_compatibility_requirements) +for more details. + + +```java +/** + * Load TFLite model and create BertQuestionAnswerer instance. + */ + public void loadModel() { + try { + answerer = BertQuestionAnswerer.createFromFile(context, MODEL_PATH); + } catch (IOException e) { + Log.e(TAG, e.getMessage()); + } + } +``` + +`BertQuestionAnswerer` currently does not support configuring delegates and +multithread, but those are on our roadmap. Please stay tuned! + +##### Run inference + +The following code runs inference using `BertQuestionAnswerer` and predicts the possible answers + +```java + /** + * Run inference and predict the possible answers. + */ + List apiResult = answerer.answer(contextOfTheQuestion, questionToAsk); + +``` + +The output of `BertQuestionAnswerer` is a list of [`QaAnswer`](https://github.com/tensorflow/tflite-support/blob/master/tensorflow_lite_support/java/src/java/org/tensorflow/lite/task/text/qa/QaAnswer.java) instance, where +each `QaAnswer` element is a single head classification result. All the +demo models are single head models, therefore, `results` only contains one +`QaAnswer` object. + +To match the implementation of +[`lib_interpreter`](https://github.com/SunitRoy2703/examples/tree/bertQa-android-task-lib/lite/examples/bert_qa/android/lib_interpreter), +`results` is converted into List<[`Answer`](https://github.com/SunitRoy2703/examples/blob/bertQa-android-task-lib/lite/examples/bert_qa/android/lib_task_api/src/main/java/org/tensorflow/lite/examples/bertqa/ml/Answer.java)>. + +#### Using the TensorFlow Lite Interpreter + +##### Load model and create interpreter + +To perform inference, we need to load a model file and instantiate an +`Interpreter`. This happens in the `loadModel` method of the `QaClient` class. Information about number of threads is used to configure the `Interpreter` via the +`Interpreter.Options` instance passed into its constructor. + +```java +Interpreter.Options opt = new Interpreter.Options(); + opt.setNumThreads(NUM_LITE_THREADS); + tflite = new Interpreter(buffer, opt); +... +``` + +##### Pre-process query & content + +Next in the `predict` method of the `QaClient` class, we take the input of query & content, +convert it to a `Feature` format for efficient processing and pre-process +it. The steps are shown in the public 'FeatureConverter.convert()' method: + +```java + +public Feature convert(String query, String context) { + List queryTokens = tokenizer.tokenize(query); + if (queryTokens.size() > maxQueryLen) { + queryTokens = queryTokens.subList(0, maxQueryLen); + } + + List origTokens = Arrays.asList(context.trim().split("\\s+")); + List tokenToOrigIndex = new ArrayList<>(); + List allDocTokens = new ArrayList<>(); + for (int i = 0; i < origTokens.size(); i++) { + String token = origTokens.get(i); + List subTokens = tokenizer.tokenize(token); + for (String subToken : subTokens) { + tokenToOrigIndex.add(i); + allDocTokens.add(subToken); + } + } + +``` + +##### Run inference + +Inference is performed using the following in `QaClient` class: + +```java +tflite.runForMultipleInputsOutputs(inputs, output); +``` + +### Display results + +The QaClient is invoked and inference results are displayed by the +`presentAnswer()` function in +[`QaActivity.java`](https://github.com/SunitRoy2703/examples/blob/bertQa-android-task-lib/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/QaActivity.java). + +```java +private void presentAnswer(Answer answer) { + // Highlight answer. + Spannable spanText = new SpannableString(content); + int offset = content.indexOf(answer.text, 0); + if (offset >= 0) { + spanText.setSpan( + new BackgroundColorSpan(getColor(R.color.tfe_qa_color_highlight)), + offset, + offset + answer.text.length(), + Spannable.SPAN_EXCLUSIVE_EXCLUSIVE); + } + contentTextView.setText(spanText); + + // Use TTS to speak out the answer. + if (textToSpeech != null) { + textToSpeech.speak(answer.text, TextToSpeech.QUEUE_FLUSH, null, answer.text); + } + } +``` diff --git a/lite/examples/bert_qa/android/lib_interpreter/src/main/assets/qa.json b/lite/examples/bert_qa/android/lib_interpreter/src/main/assets/qa.json deleted file mode 100644 index 94b7c9c4b50..00000000000 --- a/lite/examples/bert_qa/android/lib_interpreter/src/main/assets/qa.json +++ /dev/null @@ -1 +0,0 @@ -{"titles": [["Super_Bowl_50"], ["Warsaw"], ["Normans"], ["Nikola_Tesla"], ["Computational_complexity_theory"], ["Teacher"], ["Martin_Luther"], ["Southern_California"], ["Sky_(United_Kingdom)"], ["Victoria_(Australia)"], ["Huguenot"], ["Steam_engine"], ["Oxygen"], ["1973_oil_crisis"], ["Apollo_program"], ["European_Union_law"], ["Amazon_rainforest"], ["Ctenophora"], ["Fresno,_California"], ["Packet_switching"], ["Black_Death"], ["Geology"], ["Newcastle_upon_Tyne"], ["Victoria_and_Albert_Museum"], ["American_Broadcasting_Company"], ["Genghis_Khan"], ["Pharmacy"], ["Immune_system"], ["Civil_disobedience"], ["Construction"], ["Private_school"], ["Harvard_University"], ["Jacksonville,_Florida"], ["Economic_inequality"], ["Doctor_Who"], ["University_of_Chicago"], ["Yuan_dynasty"], ["Kenya"], ["Intergovernmental_Panel_on_Climate_Change"], ["Chloroplast"], ["Prime_number"], ["Rhine"], ["Scottish_Parliament"], ["Islamism"], ["Imperialism"], ["United_Methodist_Church"], ["French_and_Indian_War"], ["Force"]], "contents": [["Super Bowl 50 was an American football game to determine the champion of the National Football League (NFL) for the 2015 season. The American Football Conference (AFC) champion Denver Broncos defeated the National Football Conference (NFC) champion Carolina Panthers 24\u201310 to earn their third Super Bowl title. The game was played on February 7, 2016, at Levi's Stadium in the San Francisco Bay Area at Santa Clara, California. As this was the 50th Super Bowl, the league emphasized the \"golden anniversary\" with various gold-themed initiatives, as well as temporarily suspending the tradition of naming each Super Bowl game with Roman numerals (under which the game would have been known as \"Super Bowl L\"), so that the logo could prominently feature the Arabic numerals 50."], ["One of the most famous people born in Warsaw was Maria Sk\u0142odowska-Curie, who achieved international recognition for her research on radioactivity and was the first female recipient of the Nobel Prize. Famous musicians include W\u0142adys\u0142aw Szpilman and Fr\u00e9d\u00e9ric Chopin. Though Chopin was born in the village of \u017belazowa Wola, about 60 km (37 mi) from Warsaw, he moved to the city with his family when he was seven months old. Casimir Pulaski, a Polish general and hero of the American Revolutionary War, was born here in 1745."], ["The Normans (Norman: Nourmands; French: Normands; Latin: Normanni) were the people who in the 10th and 11th centuries gave their name to Normandy, a region in France. They were descended from Norse (\"Norman\" comes from \"Norseman\") raiders and pirates from Denmark, Iceland and Norway who, under their leader Rollo, agreed to swear fealty to King Charles III of West Francia. Through generations of assimilation and mixing with the native Frankish and Roman-Gaulish populations, their descendants would gradually merge with the Carolingian-based cultures of West Francia. The distinct cultural and ethnic identity of the Normans emerged initially in the first half of the 10th century, and it continued to evolve over the succeeding centuries."], ["Nikola Tesla (Serbian Cyrillic: \u041d\u0438\u043a\u043e\u043b\u0430 \u0422\u0435\u0441\u043b\u0430; 10 July 1856 \u2013 7 January 1943) was a Serbian American inventor, electrical engineer, mechanical engineer, physicist, and futurist best known for his contributions to the design of the modern alternating current (AC) electricity supply system."], ["Computational complexity theory is a branch of the theory of computation in theoretical computer science that focuses on classifying computational problems according to their inherent difficulty, and relating those classes to each other. A computational problem is understood to be a task that is in principle amenable to being solved by a computer, which is equivalent to stating that the problem may be solved by mechanical application of mathematical steps, such as an algorithm."], ["The role of teacher is often formal and ongoing, carried out at a school or other place of formal education. In many countries, a person who wishes to become a teacher must first obtain specified professional qualifications or credentials from a university or college. These professional qualifications may include the study of pedagogy, the science of teaching. Teachers, like other professionals, may have to continue their education after they qualify, a process known as continuing professional development. Teachers may use a lesson plan to facilitate student learning, providing a course of study which is called the curriculum."], ["Martin Luther (/\u02c8lu\u02d0\u03b8\u0259r/ or /\u02c8lu\u02d0\u00f0\u0259r/; German: [\u02c8ma\u0250\u032fti\u02d0n \u02c8l\u028at\u0250] ( listen); 10 November 1483 \u2013 18 February 1546) was a German professor of theology, composer, priest, former monk and a seminal figure in the Protestant Reformation. Luther came to reject several teachings and practices of the Late Medieval Catholic Church. He strongly disputed the claim that freedom from God's punishment for sin could be purchased with money. He proposed an academic discussion of the power and usefulness of indulgences in his Ninety-Five Theses of 1517. His refusal to retract all of his writings at the demand of Pope Leo X in 1520 and the Holy Roman Emperor Charles V at the Diet of Worms in 1521 resulted in his excommunication by the Pope and condemnation as an outlaw by the Emperor."], ["Southern California, often abbreviated SoCal, is a geographic and cultural region that generally comprises California's southernmost 10 counties. The region is traditionally described as \"eight counties\", based on demographics and economic ties: Imperial, Los Angeles, Orange, Riverside, San Bernardino, San Diego, Santa Barbara, and Ventura. The more extensive 10-county definition, including Kern and San Luis Obispo counties, is also used based on historical political divisions. Southern California is a major economic center for the state of California and the United States."], ["Formed in November 1990 by the equal merger of Sky Television and British Satellite Broadcasting, BSkyB became the UK's largest digital subscription television company. Following BSkyB's 2014 acquisition of Sky Italia and a majority 90.04% interest in Sky Deutschland in November 2014, its holding company British Sky Broadcasting Group plc changed its name to Sky plc. The United Kingdom operations also changed the company name from British Sky Broadcasting Limited to Sky UK Limited, still trading as Sky."], ["The economy of Victoria is highly diversified: service sectors including financial and property services, health, education, wholesale, retail, hospitality and manufacturing constitute the majority of employment. Victoria's total gross state product (GSP) is ranked second in Australia, although Victoria is ranked fourth in terms of GSP per capita because of its limited mining activity. Culturally, Melbourne is home to a number of museums, art galleries and theatres and is also described as the \"sporting capital of Australia\". The Melbourne Cricket Ground is the largest stadium in Australia, and the host of the 1956 Summer Olympics and the 2006 Commonwealth Games. The ground is also considered the \"spiritual home\" of Australian cricket and Australian rules football, and hosts the grand final of the Australian Football League (AFL) each year, usually drawing crowds of over 95,000 people. Victoria includes eight public universities, with the oldest, the University of Melbourne, having been founded in 1853."], ["Huguenot numbers peaked near an estimated two million by 1562, concentrated mainly in the southern and central parts of France, about one-eighth the number of French Catholics. As Huguenots gained influence and more openly displayed their faith, Catholic hostility grew, in spite of increasingly liberal political concessions and edicts of toleration from the French crown. A series of religious conflicts followed, known as the Wars of Religion, fought intermittently from 1562 to 1598. The wars finally ended with the granting of the Edict of Nantes, which granted the Huguenots substantial religious, political and military autonomy."], ["Steam engines are external combustion engines, where the working fluid is separate from the combustion products. Non-combustion heat sources such as solar power, nuclear power or geothermal energy may be used. The ideal thermodynamic cycle used to analyze this process is called the Rankine cycle. In the cycle, water is heated and transforms into steam within a boiler operating at a high pressure. When expanded through pistons or turbines, mechanical work is done. The reduced-pressure steam is then condensed and pumped back into the boiler."], ["Oxygen is a chemical element with symbol O and atomic number 8. It is a member of the chalcogen group on the periodic table and is a highly reactive nonmetal and oxidizing agent that readily forms compounds (notably oxides) with most elements. By mass, oxygen is the third-most abundant element in the universe, after hydrogen and helium. At standard temperature and pressure, two atoms of the element bind to form dioxygen, a colorless and odorless diatomic gas with the formula O\n2. Diatomic oxygen gas constitutes 20.8% of the Earth's atmosphere. However, monitoring of atmospheric oxygen levels show a global downward trend, because of fossil-fuel burning. Oxygen is the most abundant element by mass in the Earth's crust as part of oxide compounds such as silicon dioxide, making up almost half of the crust's mass."], ["The 1973 oil crisis began in October 1973 when the members of the Organization of Arab Petroleum Exporting Countries (OAPEC, consisting of the Arab members of OPEC plus Egypt and Syria) proclaimed an oil embargo. By the end of the embargo in March 1974, the price of oil had risen from US$3 per barrel to nearly $12 globally; US prices were significantly higher. The embargo caused an oil crisis, or \"shock\", with many short- and long-term effects on global politics and the global economy. It was later called the \"first oil shock\", followed by the 1979 oil crisis, termed the \"second oil shock.\""], ["The Apollo program, also known as Project Apollo, was the third United States human spaceflight program carried out by the National Aeronautics and Space Administration (NASA), which accomplished landing the first humans on the Moon from 1969 to 1972. First conceived during Dwight D. Eisenhower's administration as a three-man spacecraft to follow the one-man Project Mercury which put the first Americans in space, Apollo was later dedicated to President John F. Kennedy's national goal of \"landing a man on the Moon and returning him safely to the Earth\" by the end of the 1960s, which he proposed in a May 25, 1961, address to Congress. Project Mercury was followed by the two-man Project Gemini (1962\u201366). The first manned flight of Apollo was in 1968."], ["European Union law is a body of treaties and legislation, such as Regulations and Directives, which have direct effect or indirect effect on the laws of European Union member states. The three sources of European Union law are primary law, secondary law and supplementary law. The main sources of primary law are the Treaties establishing the European Union. Secondary sources include regulations and directives which are based on the Treaties. The legislature of the European Union is principally composed of the European Parliament and the Council of the European Union, which under the Treaties may establish secondary law to pursue the objective set out in the Treaties."], ["The Amazon rainforest (Portuguese: Floresta Amaz\u00f4nica or Amaz\u00f4nia; Spanish: Selva Amaz\u00f3nica, Amazon\u00eda or usually Amazonia; French: For\u00eat amazonienne; Dutch: Amazoneregenwoud), also known in English as Amazonia or the Amazon Jungle, is a moist broadleaf forest that covers most of the Amazon basin of South America. This basin encompasses 7,000,000 square kilometres (2,700,000 sq mi), of which 5,500,000 square kilometres (2,100,000 sq mi) are covered by the rainforest. This region includes territory belonging to nine nations. The majority of the forest is contained within Brazil, with 60% of the rainforest, followed by Peru with 13%, Colombia with 10%, and with minor amounts in Venezuela, Ecuador, Bolivia, Guyana, Suriname and French Guiana. States or departments in four nations contain \"Amazonas\" in their names. The Amazon represents over half of the planet's remaining rainforests, and comprises the largest and most biodiverse tract of tropical rainforest in the world, with an estimated 390 billion individual trees divided into 16,000 species."], ["Ctenophora (/t\u1d7b\u02c8n\u0252f\u0259r\u0259/; singular ctenophore, /\u02c8t\u025bn\u0259f\u0254\u02d0r/ or /\u02c8ti\u02d0n\u0259f\u0254\u02d0r/; from the Greek \u03ba\u03c4\u03b5\u03af\u03c2 kteis 'comb' and \u03c6\u03ad\u03c1\u03c9 pher\u014d 'carry'; commonly known as comb jellies) is a phylum of animals that live in marine waters worldwide. Their most distinctive feature is the \u2018combs\u2019 \u2013 groups of cilia which they use for swimming \u2013 they are the largest animals that swim by means of cilia. Adults of various species range from a few millimeters to 1.5 m (4 ft 11 in) in size. Like cnidarians, their bodies consist of a mass of jelly, with one layer of cells on the outside and another lining the internal cavity. In ctenophores, these layers are two cells deep, while those in cnidarians are only one cell deep. Some authors combined ctenophores and cnidarians in one phylum, Coelenterata, as both groups rely on water flow through the body cavity for both digestion and respiration. Increasing awareness of the differences persuaded more recent authors to classify them as separate phyla."], ["Fresno (/\u02c8fr\u025bzno\u028a/ FREZ-noh), the county seat of Fresno County, is a city in the U.S. state of California. As of 2015, the city's population was 520,159, making it the fifth-largest city in California, the largest inland city in California and the 34th-largest in the nation. Fresno is in the center of the San Joaquin Valley and is the largest city in the Central Valley, which contains the San Joaquin Valley. It is approximately 220 miles (350 km) northwest of Los Angeles, 170 miles (270 km) south of the state capital, Sacramento, or 185 miles (300 km) south of San Francisco. The name Fresno means \"ash tree\" in Spanish, and an ash leaf is featured on the city's flag."], ["Starting in the late 1950s, American computer scientist Paul Baran developed the concept Distributed Adaptive Message Block Switching with the goal to provide a fault-tolerant, efficient routing method for telecommunication messages as part of a research program at the RAND Corporation, funded by the US Department of Defense. This concept contrasted and contradicted the theretofore established principles of pre-allocation of network bandwidth, largely fortified by the development of telecommunications in the Bell System. The new concept found little resonance among network implementers until the independent work of Donald Davies at the National Physical Laboratory (United Kingdom) (NPL) in the late 1960s. Davies is credited with coining the modern name packet switching and inspiring numerous packet switching networks in Europe in the decade following, including the incorporation of the concept in the early ARPANET in the United States."], ["The Black Death is thought to have originated in the arid plains of Central Asia, where it then travelled along the Silk Road, reaching Crimea by 1343. From there, it was most likely carried by Oriental rat fleas living on the black rats that were regular passengers on merchant ships. Spreading throughout the Mediterranean and Europe, the Black Death is estimated to have killed 30\u201360% of Europe's total population. In total, the plague reduced the world population from an estimated 450 million down to 350\u2013375 million in the 14th century. The world population as a whole did not recover to pre-plague levels until the 17th century. The plague recurred occasionally in Europe until the 19th century."], ["There are three major types of rock: igneous, sedimentary, and metamorphic. The rock cycle is an important concept in geology which illustrates the relationships between these three types of rock, and magma. When a rock crystallizes from melt (magma and/or lava), it is an igneous rock. This rock can be weathered and eroded, and then redeposited and lithified into a sedimentary rock, or be turned into a metamorphic rock due to heat and pressure that change the mineral content of the rock which gives it a characteristic fabric. The sedimentary rock can then be subsequently turned into a metamorphic rock due to heat and pressure and is then weathered, eroded, deposited, and lithified, ultimately becoming a sedimentary rock. Sedimentary rock may also be re-eroded and redeposited, and metamorphic rock may also undergo additional metamorphism. All three types of rocks may be re-melted; when this happens, a new magma is formed, from which an igneous rock may once again crystallize."], ["Newcastle upon Tyne (RP: i/\u02ccnju\u02d0k\u0251\u02d0s\u0259l \u0259\u02ccp\u0252n \u02c8ta\u026an/; Locally: i/nju\u02d0\u02cck\u00e6s\u0259l \u0259\u02ccp\u0259n \u02c8ta\u026an/), commonly known as Newcastle, is a city in Tyne and Wear, North East England, 103 miles (166 km) south of Edinburgh and 277 miles (446 km) north of London on the northern bank of the River Tyne, 8.5 mi (13.7 km) from the North Sea. Newcastle is the most populous city in the North East and Tyneside the eighth most populous conurbation in the United Kingdom. Newcastle is a member of the English Core Cities Group and is a member of the Eurocities network of European cities. Newcastle was part of the county of Northumberland until 1400, when it became a county itself, a status it retained until becoming part of Tyne and Wear in 1974.[not in citation given] The regional nickname and dialect for people from Newcastle and the surrounding area is Geordie."], ["The Victoria and Albert Museum (often abbreviated as the V&A), London, is the world's largest museum of decorative arts and design, housing a permanent collection of over 4.5 million objects. It was founded in 1852 and named after Queen Victoria and Prince Albert. The V&A is located in the Brompton district of the Royal Borough of Kensington and Chelsea, in an area that has become known as \"Albertopolis\" because of its association with Prince Albert, the Albert Memorial and the major cultural institutions with which he was associated. These include the Natural History Museum, the Science Museum and the Royal Albert Hall. The museum is a non-departmental public body sponsored by the Department for Culture, Media and Sport. Like other national British museums, entrance to the museum has been free since 2001."], ["The American Broadcasting Company (ABC) (stylized in its logo as abc since 1957) is an American commercial broadcast television network that is owned by the Disney\u2013ABC Television Group, a subsidiary of Disney Media Networks division of The Walt Disney Company. The network is part of the Big Three television networks. The network is headquartered on Columbus Avenue and West 66th Street in Manhattan, with additional major offices and production facilities in New York City, Los Angeles and Burbank, California."], ["He came to power by uniting many of the nomadic tribes of Northeast Asia. After founding the Mongol Empire and being proclaimed \"Genghis Khan\", he started the Mongol invasions that resulted in the conquest of most of Eurasia. These included raids or invasions of the Qara Khitai, Caucasus, Khwarezmid Empire, Western Xia and Jin dynasties. These campaigns were often accompanied by wholesale massacres of the civilian populations \u2013 especially in the Khwarezmian and Xia controlled lands. By the end of his life, the Mongol Empire occupied a substantial portion of Central Asia and China."], ["The word pharmacy is derived from its root word pharma which was a term used since the 15th\u201317th centuries. However, the original Greek roots from pharmakos imply sorcery or even poison. In addition to pharma responsibilities, the pharma offered general medical advice and a range of services that are now performed solely by other specialist practitioners, such as surgery and midwifery. The pharma (as it was referred to) often operated through a retail shop which, in addition to ingredients for medicines, sold tobacco and patent medicines. Often the place that did this was called an apothecary and several languages have this as the dominant term, though their practices are more akin to a modern pharmacy, in English the term apothecary would today be seen as outdated or only approproriate if herbal remedies were on offer to a large extent. The pharmas also used many other herbs not listed. The Greek word Pharmakeia (Greek: \u03c6\u03b1\u03c1\u03bc\u03b1\u03ba\u03b5\u03af\u03b1) derives from pharmakon (\u03c6\u03ac\u03c1\u03bc\u03b1\u03ba\u03bf\u03bd), meaning \"drug\", \"medicine\" (or \"poison\").[n 1]"], ["The immune system is a system of many biological structures and processes within an organism that protects against disease. To function properly, an immune system must detect a wide variety of agents, known as pathogens, from viruses to parasitic worms, and distinguish them from the organism's own healthy tissue. In many species, the immune system can be classified into subsystems, such as the innate immune system versus the adaptive immune system, or humoral immunity versus cell-mediated immunity. In humans, the blood\u2013brain barrier, blood\u2013cerebrospinal fluid barrier, and similar fluid\u2013brain barriers separate the peripheral immune system from the neuroimmune system which protects the brain."], ["One of its earliest massive implementations was brought about by Egyptians against the British occupation in the 1919 Revolution. Civil disobedience is one of the many ways people have rebelled against what they deem to be unfair laws. It has been used in many nonviolent resistance movements in India (Gandhi's campaigns for independence from the British Empire), in Czechoslovakia's Velvet Revolution and in East Germany to oust their communist governments, In South Africa in the fight against apartheid, in the American Civil Rights Movement, in the Singing Revolution to bring independence to the Baltic countries from the Soviet Union, recently with the 2003 Rose Revolution in Georgia and the 2004 Orange Revolution in Ukraine, among other various movements worldwide."], ["Construction is the process of constructing a building or infrastructure. Construction differs from manufacturing in that manufacturing typically involves mass production of similar items without a designated purchaser, while construction typically takes place on location for a known client. Construction as an industry comprises six to nine percent of the gross domestic product of developed countries. Construction starts with planning,[citation needed] design, and financing and continues until the project is built and ready for use."], ["Private schools, also known as independent schools, non-governmental, or nonstate schools, are not administered by local, state or national governments; thus, they retain the right to select their students and are funded in whole or in part by charging their students tuition, rather than relying on mandatory taxation through public (government) funding; at some private schools students may be able to get a scholarship, which makes the cost cheaper, depending on a talent the student may have (e.g. sport scholarship, art scholarship, academic scholarship), financial need, or tax credit scholarships that might be available."], ["Established originally by the Massachusetts legislature and soon thereafter named for John Harvard (its first benefactor), Harvard is the United States' oldest institution of higher learning, and the Harvard Corporation (formally, the President and Fellows of Harvard College) is its first chartered corporation. Although never formally affiliated with any denomination, the early College primarily trained Congregationalist and Unitarian clergy. Its curriculum and student body were gradually secularized during the 18th century, and by the 19th century Harvard had emerged as the central cultural establishment among Boston elites. Following the American Civil War, President Charles W. Eliot's long tenure (1869\u20131909) transformed the college and affiliated professional schools into a modern research university; Harvard was a founding member of the Association of American Universities in 1900. James Bryant Conant led the university through the Great Depression and World War II and began to reform the curriculum and liberalize admissions after the war. The undergraduate college became coeducational after its 1977 merger with Radcliffe College."], ["Jacksonville is the largest city by population in the U.S. state of Florida, and the largest city by area in the contiguous United States. It is the county seat of Duval County, with which the city government consolidated in 1968. Consolidation gave Jacksonville its great size and placed most of its metropolitan population within the city limits; with an estimated population of 853,382 in 2014, it is the most populous city proper in Florida and the Southeast, and the 12th most populous in the United States. Jacksonville is the principal city in the Jacksonville metropolitan area, with a population of 1,345,596 in 2010."], ["A study by the World Institute for Development Economics Research at United Nations University reports that the richest 1% of adults alone owned 40% of global assets in the year 2000. The three richest people in the world possess more financial assets than the lowest 48 nations combined. The combined wealth of the \"10 million dollar millionaires\" grew to nearly $41 trillion in 2008. A January 2014 report by Oxfam claims that the 85 wealthiest individuals in the world have a combined wealth equal to that of the bottom 50% of the world's population, or about 3.5 billion people. According to a Los Angeles Times analysis of the report, the wealthiest 1% owns 46% of the world's wealth; the 85 richest people, a small part of the wealthiest 1%, own about 0.7% of the human population's wealth, which is the same as the bottom half of the population. More recently, in January 2015, Oxfam reported that the wealthiest 1 percent will own more than half of the global wealth by 2016. An October 2014 study by Credit Suisse also claims that the top 1% now own nearly half of the world's wealth and that the accelerating disparity could trigger a recession. In October 2015, Credit Suisse published a study which shows global inequality continues to increase, and that half of the world's wealth is now in the hands of those in the top percentile, whose assets each exceed $759,900. A 2016 report by Oxfam claims that the 62 wealthiest individuals own as much wealth as the poorer half of the global population combined. Oxfam's claims have however been questioned on the basis of the methodology used: by using net wealth (adding up assets and subtracting debts), the Oxfam report, for instance, finds that there are more poor people in the United States and Western Europe than in China (due to a greater tendency to take on debts).[unreliable source?][unreliable source?] Anthony Shorrocks, the lead author of the Credit Suisse report which is one of the sources of Oxfam's data, considers the criticism about debt to be a \"silly argument\" and \"a non-issue . . . a diversion.\""], ["Doctor Who is a British science-fiction television programme produced by the BBC since 1963. The programme depicts the adventures of the Doctor, a Time Lord\u2014a space and time-travelling humanoid alien. He explores the universe in his TARDIS, a sentient time-travelling space ship. Its exterior appears as a blue British police box, which was a common sight in Britain in 1963 when the series first aired. Accompanied by companions, the Doctor combats a variety of foes, while working to save civilisations and help people in need."], ["The University of Chicago (UChicago, Chicago, or U of C) is a private research university in Chicago. The university, established in 1890, consists of The College, various graduate programs, interdisciplinary committees organized into four academic research divisions and seven professional schools. Beyond the arts and sciences, Chicago is also well known for its professional schools, which include the Pritzker School of Medicine, the University of Chicago Booth School of Business, the Law School, the School of Social Service Administration, the Harris School of Public Policy Studies, the Graham School of Continuing Liberal and Professional Studies and the Divinity School. The university currently enrolls approximately 5,000 students in the College and around 15,000 students overall."], ["The Yuan dynasty (Chinese: \u5143\u671d; pinyin: Yu\u00e1n Ch\u00e1o), officially the Great Yuan (Chinese: \u5927\u5143; pinyin: D\u00e0 Yu\u00e1n; Mongolian: Yehe Yuan Ulus[a]), was the empire or ruling dynasty of China established by Kublai Khan, leader of the Mongolian Borjigin clan. Although the Mongols had ruled territories including today's North China for decades, it was not until 1271 that Kublai Khan officially proclaimed the dynasty in the traditional Chinese style. His realm was, by this point, isolated from the other khanates and controlled most of present-day China and its surrounding areas, including modern Mongolia and Korea. It was the first foreign dynasty to rule all of China and lasted until 1368, after which its Genghisid rulers returned to their Mongolian homeland and continued to rule the Northern Yuan dynasty. Some of the Mongolian Emperors of the Yuan mastered the Chinese language, while others only used their native language (i.e. Mongolian) and the 'Phags-pa script."], ["Kenya (/\u02c8k\u025bnj\u0259/; locally [\u02c8k\u025b\u0272a] ( listen)), officially the Republic of Kenya, is a country in Africa and a founding member of the East African Community (EAC). Its capital and largest city is Nairobi. Kenya's territory lies on the equator and overlies the East African Rift covering a diverse and expansive terrain that extends roughly from Lake Victoria to Lake Turkana (formerly called Lake Rudolf) and further south-east to the Indian Ocean. It is bordered by Tanzania to the south, Uganda to the west, South Sudan to the north-west, Ethiopia to the north and Somalia to the north-east. Kenya covers 581,309 km2 (224,445 sq mi), and had a population of approximately 45 million people in July 2014."], ["The Intergovernmental Panel on Climate Change (IPCC) is a scientific intergovernmental body under the auspices of the United Nations, set up at the request of member governments. It was first established in 1988 by two United Nations organizations, the World Meteorological Organization (WMO) and the United Nations Environment Programme (UNEP), and later endorsed by the United Nations General Assembly through Resolution 43/53. Membership of the IPCC is open to all members of the WMO and UNEP. The IPCC produces reports that support the United Nations Framework Convention on Climate Change (UNFCCC), which is the main international treaty on climate change. The ultimate objective of the UNFCCC is to \"stabilize greenhouse gas concentrations in the atmosphere at a level that would prevent dangerous anthropogenic [i.e., human-induced] interference with the climate system\". IPCC reports cover \"the scientific, technical and socio-economic information relevant to understanding the scientific basis of risk of human-induced climate change, its potential impacts and options for adaptation and mitigation.\""], ["Chloroplasts' main role is to conduct photosynthesis, where the photosynthetic pigment chlorophyll captures the energy from sunlight and converts it and stores it in the energy-storage molecules ATP and NADPH while freeing oxygen from water. They then use the ATP and NADPH to make organic molecules from carbon dioxide in a process known as the Calvin cycle. Chloroplasts carry out a number of other functions, including fatty acid synthesis, much amino acid synthesis, and the immune response in plants. The number of chloroplasts per cell varies from 1 in algae up to 100 in plants like Arabidopsis and wheat."], ["A prime number (or a prime) is a natural number greater than 1 that has no positive divisors other than 1 and itself. A natural number greater than 1 that is not a prime number is called a composite number. For example, 5 is prime because 1 and 5 are its only positive integer factors, whereas 6 is composite because it has the divisors 2 and 3 in addition to 1 and 6. The fundamental theorem of arithmetic establishes the central role of primes in number theory: any integer greater than 1 can be expressed as a product of primes that is unique up to ordering. The uniqueness in this theorem requires excluding 1 as a prime because one can include arbitrarily many instances of 1 in any factorization, e.g., 3, 1 \u00b7 3, 1 \u00b7 1 \u00b7 3, etc. are all valid factorizations of 3."], ["The Rhine (Romansh: Rein, German: Rhein, French: le Rhin, Dutch: Rijn) is a European river that begins in the Swiss canton of Graub\u00fcnden in the southeastern Swiss Alps, forms part of the Swiss-Austrian, Swiss-Liechtenstein border, Swiss-German and then the Franco-German border, then flows through the Rhineland and eventually empties into the North Sea in the Netherlands. The biggest city on the river Rhine is Cologne, Germany with a population of more than 1,050,000 people. It is the second-longest river in Central and Western Europe (after the Danube), at about 1,230 km (760 mi),[note 2][note 1] with an average discharge of about 2,900 m3/s (100,000 cu ft/s)."], ["Following a referendum in 1997, in which the Scottish electorate voted for devolution, the current Parliament was convened by the Scotland Act 1998, which sets out its powers as a devolved legislature. The Act delineates the legislative competence of the Parliament \u2013 the areas in which it can make laws \u2013 by explicitly specifying powers that are \"reserved\" to the Parliament of the United Kingdom. The Scottish Parliament has the power to legislate in all areas that are not explicitly reserved to Westminster. The British Parliament retains the ability to amend the terms of reference of the Scottish Parliament, and can extend or reduce the areas in which it can make laws. The first meeting of the new Parliament took place on 12 May 1999."], ["Islamism, also known as Political Islam (Arabic: \u0625\u0633\u0644\u0627\u0645 \u0633\u064a\u0627\u0633\u064a\u200e isl\u0101m siy\u0101s\u012b), is an Islamic revival movement often characterized by moral conservatism, literalism, and the attempt \"to implement Islamic values in all spheres of life.\" Islamism favors the reordering of government and society in accordance with the Shari'a. The different Islamist movements have been described as \"oscillating between two poles\": at one end is a strategy of Islamization of society through state power seized by revolution or invasion; at the other \"reformist\" pole Islamists work to Islamize society gradually \"from the bottom up\". The movements have \"arguably altered the Middle East more than any trend since the modern states gained independence\", redefining \"politics and even borders\" according to one journalist (Robin Wright)."], ["Imperialism is a type of advocacy of empire. Its name originated from the Latin word \"imperium\", which means to rule over large territories. Imperialism is \"a policy of extending a country's power and influence through colonization, use of military force, or other means\". Imperialism has greatly shaped the contemporary world. It has also allowed for the rapid spread of technologies and ideas. The term imperialism has been applied to Western (and Japanese) political and economic dominance especially in Asia and Africa in the 19th and 20th centuries. Its precise meaning continues to be debated by scholars. Some writers, such as Edward Said, use the term more broadly to describe any system of domination and subordination organised with an imperial center and a periphery."], ["The United Methodist Church (UMC) is a mainline Protestant Methodist denomination. In the 19th century its main predecessor was a leader in Evangelicalism. Founded in 1968 by the union of the Methodist Church (USA) and the Evangelical United Brethren Church, the UMC traces its roots back to the revival movement of John and Charles Wesley in England as well as the Great Awakening in the United States. As such, the church's theological orientation is decidedly Wesleyan. It embraces both liturgical and evangelical elements."], ["The French and Indian War (1754\u20131763) was the North American theater of the worldwide Seven Years' War. The war was fought between the colonies of British America and New France, with both sides supported by military units from their parent countries of Great Britain and France, as well as Native American allies. At the start of the war, the French North American colonies had a population of roughly 60,000 European settlers, compared with 2 million in the British North American colonies. The outnumbered French particularly depended on the Indians. Long in conflict, the metropole nations declared war on each other in 1756, escalating the war from a regional affair into an intercontinental conflict."], ["Philosophers in antiquity used the concept of force in the study of stationary and moving objects and simple machines, but thinkers such as Aristotle and Archimedes retained fundamental errors in understanding force. In part this was due to an incomplete understanding of the sometimes non-obvious force of friction, and a consequently inadequate view of the nature of natural motion. A fundamental error was the belief that a force is required to maintain motion, even at a constant velocity. Most of the previous misunderstandings about motion and force were eventually corrected by Galileo Galilei and Sir Isaac Newton. With his mathematical insight, Sir Isaac Newton formulated laws of motion that were not improved-on for nearly three hundred years. By the early 20th century, Einstein developed a theory of relativity that correctly predicted the action of forces on objects with increasing momenta near the speed of light, and also provided insight into the forces produced by gravitation and inertia."]], "questions": [["Which NFL team represented the AFC at Super Bowl 50?", "Which NFL team represented the NFC at Super Bowl 50?", "Where did Super Bowl 50 take place?", "Which NFL team won Super Bowl 50?", "What color was used to emphasize the 50th anniversary of the Super Bowl?", "What was the theme of Super Bowl 50?", "What day was the game played on?", "What is the AFC short for?", "What was the theme of Super Bowl 50?", "What does AFC stand for?", "What day was the Super Bowl played on?", "Who won Super Bowl 50?", "What venue did Super Bowl 50 take place in?", "What city did Super Bowl 50 take place in?", "If Roman numerals were used, what would Super Bowl 50 have been called?", "Super Bowl 50 decided the NFL champion for what season?", "What year did the Denver Broncos secure a Super Bowl title for the third time?", "What city did Super Bowl 50 take place in?", "What stadium did Super Bowl 50 take place in?", "What was the final score of Super Bowl 50? ", "What month, day and year did Super Bowl 50 take place? ", "What year was Super Bowl 50?", "What team was the AFC champion?", "What team was the NFC champion?", "Who won Super Bowl 50?", "Super Bowl 50 determined the NFL champion for what season?", "Which team won Super Bowl 50.", "Where was Super Bowl 50 held?", "The name of the NFL championship game is?", "What 2015 NFL team one the AFC playoff?"], ["What was Maria Curie the first female recipient of?", "What year was Casimir Pulaski born in Warsaw?", "Who was one of the most famous people born in Warsaw?", "Who was Fr\u00e9d\u00e9ric Chopin?", "How old was Chopin when he moved to Warsaw with his family?"], ["In what country is Normandy located?", "When were the Normans in Normandy?", "From which countries did the Norse originate?", "Who was the Norse leader?", "What century did the Normans first gain their separate identity?"], ["In what year was Nikola Tesla born? ", "What was Nikola Tesla's ethnicity?", "In what year did Tesla die? ", "When was Nikola Tesla born?", "In what year did Tesla die?", "What is Tesla's home country?", "What does AC stand for?"], ["What branch of theoretical computer science deals with broadly classifying computational problems by difficulty and class of relationship?", "By what main attribute are computational problems classified utilizing computational complexity theory? ", "What is the term for a task that generally lends itself to being solved by a computer?"], ["What is a course of study called?", "What is another name to describe the science of teaching?", "Where do most teachers get their credentials from?", "What can a teacher use to help students learn?", "Where is a teacher most likely to be teaching at?"], ["Of what nationality was Martin Luther?", "When did Martin Luther die?", "What organization's teaching did Luther reject?", "What did the Church claim could be avoided with money?", "What did the Church do when Luther refused to retract his writings?"], ["What is Southern California often abbreviated as?", "Despite being traditionall described as \"eight counties\", how many counties does this region actually have?", "What is a major importance of Southern California in relation to California and the United States?", "What are the ties that best described what the \"eight counties\" are based on?", "The reasons for the las two counties to be added are based on what?"], ["What company was formed by the merger of Sky Television and British Satellite Broadcasting?", "Who is the UK's largest digital subscription television company?", "What year did BSkyB acquire Sky Italia?", "What is the name of the holding company for BSkyB?", "What is the name of the United Kingdom operation for BSkyB?"], ["What kind of economy does Victoria have?", "Where according to gross state product does Victoria rank in Australia?", "At what rank does GPS per capita set Victoria?", "What city in Victoria is called the sporting capital of Australia?", "What is the largest stadium in Australia?"], ["Where was France's Huguenot population largely centered?", "What was the proportion of Huguenots to Catholics at their peak?", "When were the Wars of Religion fought?", "What treaty ended the Wars of Religion?", "What did this agreement do?"], ["Along with geothermal and nuclear, what is a notable non-combustion heat source?", "What ideal thermodynamic cycle analyzes the process by which steam engines work?", "In the Rankine cycle, what does water turn into when heated?", "At what pressure is water heated in the Rankine cycle?", "What types of engines are steam engines?"], ["The atomic number of the periodic table for oxygen?", "What is the second most abundant element?", "How many atoms combine to form dioxygen?", "Roughly, how much oxygen makes up the Earth crust?", "Which gas makes up 20.8% of the Earth's atmosphere?", "How much of the earth's atmosphere is diatomic oxygen?", "What element makes up almost half of the earth's crust by mass?", "What is the atomic number for oxygen?", "Are atmospheric oxygen levels going up, down, or staying the same?", "What are the three most abundent elements of the universe by mass?", "What is the atomic number of the element oxygen?", "Of what group in the periodic table is oxygen a member?", "What type of compounds does oxygen most commonly form?", "Compared to other elements, how abundant does oxygen rank?", "Under normal conditions, what do two atoms of oxygen form?"], ["When did the 1973 oil crisis begin?", "What was the price of oil in March of 1974?", "When was the second oil crisis?", "What was another term used for the oil crisis?", "Who proclaimed the oil embargo?"], ["What project put the first Americans into space?", "What program was created to carry out these projects and missions?", "What year did the first manned Apollo flight occur?", "What President is credited with the original notion of putting Americans in space?", "How many people were on the project that followed Project Mercury?"], ["What are the three sources of European Union law?", "What is European Union Law?", "What are the main sources of primary law?", "What are the secondary sources of primary law?", "What are the two bodies that make up the European Union's legislature?", "What is European Union law?", "What effect does European Union law have on laws of member states?", "What are the three sources of European Union law?", "What are the main legislative bodies of the European Union?", "What are the three main sources of European Union law?", "What are the main sources of primary law?", "What is the legislature of the European Union comprised of?", "How many sources of European Union law are there?"], ["Which name is also used to describe the Amazon rainforest in English?", "How many square kilometers of rainforest is covered in the basin?", "How many nations control this region in total?", "How many nations contain \"Amazonas\" in their names?", "What percentage does the Amazon represents in rainforests on the planet?", "What is the Dutch word for the Amazon rainforest?", "What rainforest covers the majority of the Amazon basin in South America?", "In what country can most of the Amazon rainforest be found?", "The Amazon rainforest makes up what amount of Earth's rainforests?", "How many species of trees can be found in the Amazon rainforest?", "What kind of forest is the Amazon rainforest?", "How many square kilometers is the Amazon Basin?", "How many nations are within the Amazon Basin?", "Which nation contains the majority of the amazon forest?", "What is the estimate for the amount of tree species in the amazon tropical rain forest?"], ["What are Ctenophora commonly known as?", "Where do Ctenophora live?", "What size are adult Ctenophora?", "What is a ctenophora?", "What does the ctenophora use to swim?", "What does ctenophora use for digestion and respiration?", "How large can ctenophora grow?", "What is the most distinctive feature of ctenophora?", "What are ctenophora commonly known as?", "How big can ctenophora grow?", "What does ctenophora rely on for digestion and respiration?", "What does ctenophora mean in Greek?", "Where do ctenophora live?"], ["Which city is the fifth-largest city in California?", "How far is Fresno from Los Angeles?", "What does the name Fresno mean in Spanish?", "What is featured on the city of Fresno's city flag?", "How do you pronounce Fresno?"], ["What did Paul Baran develop ", "What did Distributed Adaptive Message Block Switching do", "What did this concept contradict ", "What is Donald Davies credited with ", "What did Paul Baran develop in the late 1950's", "What was the goal of the system ", "Who is credited with the modern name for this system "], ["Where did the black death originate?", "How did the black death make it to the Mediterranean and Europe?", "How much of the European population did the black death kill?", "When did the world's population finally recover from the black death?", "For how long did the plague stick around?"], ["An igneous rock is a rock that crystallizes from what?", "Sedimentary rock can be turned into which of the three types of rock?", "When the three types of rock are re-melted what is formed? ", "What are the three major types of rock? ", "What changes the mineral content of a rock?"], ["How many miles south of Edinburgh is Newcastle?", "How many miles from the north Sea is Newcastle?", "What network is Newcastle a member of?", "What county was Newcastle a part of until 1400?", "What's the regional nickname for Newcastle and its surrounding area?"], ["where is the Victoria and Albert Museum located?", "how many permanent objects are located there?", "when was the Victoria and Albert museum founded?", "Who is the museum named for?", "In which London borough is the Victoria and Albert Museum located?", "In what year was the Victoria and Albert Museum founded?", "Which monarchs was the Victoria and Albert Museum named after?", "Which department sponsors the Victoria and Albert Museum?", "In which year did the museum started charging free admission fees?"], ["What company owns the American Broadcasting Company?", "In what year did ABC stylize it's logo as abc?", "In what borough of New York City is ABC headquartered?", "On what streets is the ABC headquarters located", "Disney-ABC Television Group is a subsidiary of what division of the Walt Disney Company?"], ["What do we call the empire that Genghis Khan founded?", "Who did Genghis Khan unite before he began conquering the rest of Eurasia?", "In which regions in particular did Genghis Khan's armies massacre civilians?", "What areas did Genghis Khan control at the end of his life?", "Which other empires or dynasties did Genghis Khan conquer?"], ["What word is the word pharmacy taken from?", "What goods were sold in a pharma?", "What did the Greek root pharmakos imply?", "How would the word apothecary be viewed by contemporary English speakers?", "What else was used by pharmas?"], ["What is the immune system?", "What does the immune system protect against?", "What are two of its subsystems?", "What is the subsystem that protects the human brain?", "What is the immune system?", "What does the immune system protect against?", "What are two of its subsystems?", "The immune system protects organisms against what?", "What are the agents the immune system detects known as?", "Which part of the immune system protects the brain?", "What separates the neuroimmune system and peripheral immune system in humans?", "What are the agents detected by the immune system called?", "What are the two major subsystems of the immune system?", "What are the two different types of immunity?", "What is the immune system of the brained known as?"], ["What is it called when people in society rebel against laws they think are unfair?", "What is an example of major civil disobedience in South Africa?", "What was the the movement called that brought Baltic countries independence from the Soviet Union?", "In 2004 the Orange revolution occurred in what country?", "Where was the location of the 2003 Rose revolution?", "Which people brought forward one of the earliest examples of Civil Disobedience?", "Who was one of the earliest examples of Civil Disobedience against?", "Why do people chose civil disobedience to protest?", "What does civil disobedience protest against?", "What civil rights movement in the US was known for it's disobedience?"], ["What is the process of constructing a building or infrastructure?", "What typically involves mass production of similar items without a designated purchaser?", "What percentile of gross domestic product is construction comprised of?", "What three things are needed for construction to take place?", "Construction takes place on location for who?"], ["Along with non-governmental and nonstate schools, what is another name for private schools?", "Along with sport and art, what is a type of talent scholarship?", "Rather than taxation, what are private schools largely funded by?", "What right do private schools have that public schools don't?"], ["What individual is the school named after?", "When did the undergraduate program become coeducational?", "What was the name of the leader through the Great Depression and World War II?", "What organization did Harvard found in 1900?", "What president of the university transformed it into a modern research university?"], ["Which Florida city has the biggest population?", "What was the population Jacksonville city as of 2010?", "Based on population alone, what is Jacksonville's ranking in the United States?", "In which county does Jacksonville reside?", "What year did consolidation cause Jacksonville to become part of Duval County?"], ["What percentage of global assets does the richest 1% of people have?", "According to Oxfam, the 85 richest people have wealth equal to how many average people?", "In order to be considered in the top percentile, a person would need to amass how much money each year?", "What has caused Oxfam's findings to be questioned?", "Why does Oxfam and Credit Suisse believe their findings are being doubted?", "What percent of the global assets in 2000 were owned by just 1% of adults?", "What do the three richest people in the world posses more of than the lowest 48 nations together?", "How much was the combined wealth of the \"10 Million dollar millionaires\" in 2008?", "How much of the global wealth will the wealthiest 1 percent own by 2016?", "Why are there more poor people in the United States and Europe than China?"], ["Who is the producer of Doctor Who?", "What year did Doctor Who first show on TV?", "What is Doctor Who's space ship called?", "What does the outside of the Tardis resemble?", "What type/genre of TV show is Doctor Who?"], ["What kind of university is the University of Chicago?", "When was the University of Chicago established?", "How many professional schools does the University of Chicago have?", "How many academic research divisions does the University of Chicago have?", "How many students does the University of Chicago have enlisted?"], ["What is the Chinese name for the Yuan dynasty?", "What is the Yuan dynasty's official name?", "Who started the Yuan dynasty?", "Who led the Mongolian Borjigin clan?", "When did Khan formally declare the Yuan dynasty?"], ["Where is Kenya located?", "What is Kenya a founding member of?", "What is the capitol of Kenya?", "What country boarders the south of Kenya?", "What was the population of Kenya in 2014?"], ["What organization is the IPCC a part of?", "What UN organizations established the IPCC?", "What does the UN want to stabilize?", "What is the UN's climate change treaty?", "What UN resolution endorsed the IPCC?"], ["What is the primary purpose of chloroplasts?", "What does ATP store?", "What does NADPH store?", "What is the process of turning CO2 into organic molecules called?", "How many chloroplasts per cell does algae have?"], ["What is the only divisor besides 1 that a prime number can have?", "What are numbers greater than 1 that can be divided by 3 or more numbers called?", "What theorem defines the main role of primes in number theory?", "Any number larger than 1 can be represented as a product of what?", "Why must one be excluded in order to preserve the uniqueness of the fundamental theorem?"], ["Where does the Rhine begin? ", "Where does the Rhine empty?", "What is the largest city the Rhine runs through? ", "What river is larger than the Rhine?", "How long is the Rhine?", "Where is the Rhine? ", "What country does the Rhine empty?", "How long is the Rhine? "], ["When was the current parliament of Scotland convened?", "What act set out the Parliament's powers as a devolved legislature?", "The legislative competence of the Parliament species what areas?", "To what body are certain powers explicitly specified as being reserved for?", "The Scottish Parliament may legislate as it pleases as long as the powers aren't already reserved by where?"], ["What is an Islamic revival movement?", "What aspects of life does Islamism seek to integrate itself into?", "What goal does Islamism have when it comes to society and government?", "What have the two different Islamist movements been described as oscillating between?", "One strategy of Islamization is to seize power by what methods?"], ["The word imperialism has it's origins in which ancient language? ", "By what means is imperialism usually administered?", "The term imperialism has been applied to western countries, and which eastern county?", "Imperialism is responsible for the rapid spread of what?"], ["What does UMC stand for?", "What is the United Methodist Church?", "When was the UMC founded?", "Who founded the UMC?", "What is the church's theological orientation?"], ["When was the French and Indian War?", "Who fought in the French and Indian war?", "How many people were in French North American Colonies?", "How many people were in British North American Colonies?"], ["What concept did philosophers in antiquity use to study simple machines?", "What was the belief that maintaining motion required force?", "Who had mathmatical insite?", "How long did it take to improve on Sir Isaac Newton's laws of motion?", "Who develped the theory of relativity?"]]} \ No newline at end of file diff --git a/lite/examples/bert_qa/android/lib_task_api/src/main/assets/qa.json b/lite/examples/bert_qa/android/lib_task_api/src/main/assets/qa.json deleted file mode 100644 index 94b7c9c4b50..00000000000 --- a/lite/examples/bert_qa/android/lib_task_api/src/main/assets/qa.json +++ /dev/null @@ -1 +0,0 @@ -{"titles": [["Super_Bowl_50"], ["Warsaw"], ["Normans"], ["Nikola_Tesla"], ["Computational_complexity_theory"], ["Teacher"], ["Martin_Luther"], ["Southern_California"], ["Sky_(United_Kingdom)"], ["Victoria_(Australia)"], ["Huguenot"], ["Steam_engine"], ["Oxygen"], ["1973_oil_crisis"], ["Apollo_program"], ["European_Union_law"], ["Amazon_rainforest"], ["Ctenophora"], ["Fresno,_California"], ["Packet_switching"], ["Black_Death"], ["Geology"], ["Newcastle_upon_Tyne"], ["Victoria_and_Albert_Museum"], ["American_Broadcasting_Company"], ["Genghis_Khan"], ["Pharmacy"], ["Immune_system"], ["Civil_disobedience"], ["Construction"], ["Private_school"], ["Harvard_University"], ["Jacksonville,_Florida"], ["Economic_inequality"], ["Doctor_Who"], ["University_of_Chicago"], ["Yuan_dynasty"], ["Kenya"], ["Intergovernmental_Panel_on_Climate_Change"], ["Chloroplast"], ["Prime_number"], ["Rhine"], ["Scottish_Parliament"], ["Islamism"], ["Imperialism"], ["United_Methodist_Church"], ["French_and_Indian_War"], ["Force"]], "contents": [["Super Bowl 50 was an American football game to determine the champion of the National Football League (NFL) for the 2015 season. The American Football Conference (AFC) champion Denver Broncos defeated the National Football Conference (NFC) champion Carolina Panthers 24\u201310 to earn their third Super Bowl title. The game was played on February 7, 2016, at Levi's Stadium in the San Francisco Bay Area at Santa Clara, California. As this was the 50th Super Bowl, the league emphasized the \"golden anniversary\" with various gold-themed initiatives, as well as temporarily suspending the tradition of naming each Super Bowl game with Roman numerals (under which the game would have been known as \"Super Bowl L\"), so that the logo could prominently feature the Arabic numerals 50."], ["One of the most famous people born in Warsaw was Maria Sk\u0142odowska-Curie, who achieved international recognition for her research on radioactivity and was the first female recipient of the Nobel Prize. Famous musicians include W\u0142adys\u0142aw Szpilman and Fr\u00e9d\u00e9ric Chopin. Though Chopin was born in the village of \u017belazowa Wola, about 60 km (37 mi) from Warsaw, he moved to the city with his family when he was seven months old. Casimir Pulaski, a Polish general and hero of the American Revolutionary War, was born here in 1745."], ["The Normans (Norman: Nourmands; French: Normands; Latin: Normanni) were the people who in the 10th and 11th centuries gave their name to Normandy, a region in France. They were descended from Norse (\"Norman\" comes from \"Norseman\") raiders and pirates from Denmark, Iceland and Norway who, under their leader Rollo, agreed to swear fealty to King Charles III of West Francia. Through generations of assimilation and mixing with the native Frankish and Roman-Gaulish populations, their descendants would gradually merge with the Carolingian-based cultures of West Francia. The distinct cultural and ethnic identity of the Normans emerged initially in the first half of the 10th century, and it continued to evolve over the succeeding centuries."], ["Nikola Tesla (Serbian Cyrillic: \u041d\u0438\u043a\u043e\u043b\u0430 \u0422\u0435\u0441\u043b\u0430; 10 July 1856 \u2013 7 January 1943) was a Serbian American inventor, electrical engineer, mechanical engineer, physicist, and futurist best known for his contributions to the design of the modern alternating current (AC) electricity supply system."], ["Computational complexity theory is a branch of the theory of computation in theoretical computer science that focuses on classifying computational problems according to their inherent difficulty, and relating those classes to each other. A computational problem is understood to be a task that is in principle amenable to being solved by a computer, which is equivalent to stating that the problem may be solved by mechanical application of mathematical steps, such as an algorithm."], ["The role of teacher is often formal and ongoing, carried out at a school or other place of formal education. In many countries, a person who wishes to become a teacher must first obtain specified professional qualifications or credentials from a university or college. These professional qualifications may include the study of pedagogy, the science of teaching. Teachers, like other professionals, may have to continue their education after they qualify, a process known as continuing professional development. Teachers may use a lesson plan to facilitate student learning, providing a course of study which is called the curriculum."], ["Martin Luther (/\u02c8lu\u02d0\u03b8\u0259r/ or /\u02c8lu\u02d0\u00f0\u0259r/; German: [\u02c8ma\u0250\u032fti\u02d0n \u02c8l\u028at\u0250] ( listen); 10 November 1483 \u2013 18 February 1546) was a German professor of theology, composer, priest, former monk and a seminal figure in the Protestant Reformation. Luther came to reject several teachings and practices of the Late Medieval Catholic Church. He strongly disputed the claim that freedom from God's punishment for sin could be purchased with money. He proposed an academic discussion of the power and usefulness of indulgences in his Ninety-Five Theses of 1517. His refusal to retract all of his writings at the demand of Pope Leo X in 1520 and the Holy Roman Emperor Charles V at the Diet of Worms in 1521 resulted in his excommunication by the Pope and condemnation as an outlaw by the Emperor."], ["Southern California, often abbreviated SoCal, is a geographic and cultural region that generally comprises California's southernmost 10 counties. The region is traditionally described as \"eight counties\", based on demographics and economic ties: Imperial, Los Angeles, Orange, Riverside, San Bernardino, San Diego, Santa Barbara, and Ventura. The more extensive 10-county definition, including Kern and San Luis Obispo counties, is also used based on historical political divisions. Southern California is a major economic center for the state of California and the United States."], ["Formed in November 1990 by the equal merger of Sky Television and British Satellite Broadcasting, BSkyB became the UK's largest digital subscription television company. Following BSkyB's 2014 acquisition of Sky Italia and a majority 90.04% interest in Sky Deutschland in November 2014, its holding company British Sky Broadcasting Group plc changed its name to Sky plc. The United Kingdom operations also changed the company name from British Sky Broadcasting Limited to Sky UK Limited, still trading as Sky."], ["The economy of Victoria is highly diversified: service sectors including financial and property services, health, education, wholesale, retail, hospitality and manufacturing constitute the majority of employment. Victoria's total gross state product (GSP) is ranked second in Australia, although Victoria is ranked fourth in terms of GSP per capita because of its limited mining activity. Culturally, Melbourne is home to a number of museums, art galleries and theatres and is also described as the \"sporting capital of Australia\". The Melbourne Cricket Ground is the largest stadium in Australia, and the host of the 1956 Summer Olympics and the 2006 Commonwealth Games. The ground is also considered the \"spiritual home\" of Australian cricket and Australian rules football, and hosts the grand final of the Australian Football League (AFL) each year, usually drawing crowds of over 95,000 people. Victoria includes eight public universities, with the oldest, the University of Melbourne, having been founded in 1853."], ["Huguenot numbers peaked near an estimated two million by 1562, concentrated mainly in the southern and central parts of France, about one-eighth the number of French Catholics. As Huguenots gained influence and more openly displayed their faith, Catholic hostility grew, in spite of increasingly liberal political concessions and edicts of toleration from the French crown. A series of religious conflicts followed, known as the Wars of Religion, fought intermittently from 1562 to 1598. The wars finally ended with the granting of the Edict of Nantes, which granted the Huguenots substantial religious, political and military autonomy."], ["Steam engines are external combustion engines, where the working fluid is separate from the combustion products. Non-combustion heat sources such as solar power, nuclear power or geothermal energy may be used. The ideal thermodynamic cycle used to analyze this process is called the Rankine cycle. In the cycle, water is heated and transforms into steam within a boiler operating at a high pressure. When expanded through pistons or turbines, mechanical work is done. The reduced-pressure steam is then condensed and pumped back into the boiler."], ["Oxygen is a chemical element with symbol O and atomic number 8. It is a member of the chalcogen group on the periodic table and is a highly reactive nonmetal and oxidizing agent that readily forms compounds (notably oxides) with most elements. By mass, oxygen is the third-most abundant element in the universe, after hydrogen and helium. At standard temperature and pressure, two atoms of the element bind to form dioxygen, a colorless and odorless diatomic gas with the formula O\n2. Diatomic oxygen gas constitutes 20.8% of the Earth's atmosphere. However, monitoring of atmospheric oxygen levels show a global downward trend, because of fossil-fuel burning. Oxygen is the most abundant element by mass in the Earth's crust as part of oxide compounds such as silicon dioxide, making up almost half of the crust's mass."], ["The 1973 oil crisis began in October 1973 when the members of the Organization of Arab Petroleum Exporting Countries (OAPEC, consisting of the Arab members of OPEC plus Egypt and Syria) proclaimed an oil embargo. By the end of the embargo in March 1974, the price of oil had risen from US$3 per barrel to nearly $12 globally; US prices were significantly higher. The embargo caused an oil crisis, or \"shock\", with many short- and long-term effects on global politics and the global economy. It was later called the \"first oil shock\", followed by the 1979 oil crisis, termed the \"second oil shock.\""], ["The Apollo program, also known as Project Apollo, was the third United States human spaceflight program carried out by the National Aeronautics and Space Administration (NASA), which accomplished landing the first humans on the Moon from 1969 to 1972. First conceived during Dwight D. Eisenhower's administration as a three-man spacecraft to follow the one-man Project Mercury which put the first Americans in space, Apollo was later dedicated to President John F. Kennedy's national goal of \"landing a man on the Moon and returning him safely to the Earth\" by the end of the 1960s, which he proposed in a May 25, 1961, address to Congress. Project Mercury was followed by the two-man Project Gemini (1962\u201366). The first manned flight of Apollo was in 1968."], ["European Union law is a body of treaties and legislation, such as Regulations and Directives, which have direct effect or indirect effect on the laws of European Union member states. The three sources of European Union law are primary law, secondary law and supplementary law. The main sources of primary law are the Treaties establishing the European Union. Secondary sources include regulations and directives which are based on the Treaties. The legislature of the European Union is principally composed of the European Parliament and the Council of the European Union, which under the Treaties may establish secondary law to pursue the objective set out in the Treaties."], ["The Amazon rainforest (Portuguese: Floresta Amaz\u00f4nica or Amaz\u00f4nia; Spanish: Selva Amaz\u00f3nica, Amazon\u00eda or usually Amazonia; French: For\u00eat amazonienne; Dutch: Amazoneregenwoud), also known in English as Amazonia or the Amazon Jungle, is a moist broadleaf forest that covers most of the Amazon basin of South America. This basin encompasses 7,000,000 square kilometres (2,700,000 sq mi), of which 5,500,000 square kilometres (2,100,000 sq mi) are covered by the rainforest. This region includes territory belonging to nine nations. The majority of the forest is contained within Brazil, with 60% of the rainforest, followed by Peru with 13%, Colombia with 10%, and with minor amounts in Venezuela, Ecuador, Bolivia, Guyana, Suriname and French Guiana. States or departments in four nations contain \"Amazonas\" in their names. The Amazon represents over half of the planet's remaining rainforests, and comprises the largest and most biodiverse tract of tropical rainforest in the world, with an estimated 390 billion individual trees divided into 16,000 species."], ["Ctenophora (/t\u1d7b\u02c8n\u0252f\u0259r\u0259/; singular ctenophore, /\u02c8t\u025bn\u0259f\u0254\u02d0r/ or /\u02c8ti\u02d0n\u0259f\u0254\u02d0r/; from the Greek \u03ba\u03c4\u03b5\u03af\u03c2 kteis 'comb' and \u03c6\u03ad\u03c1\u03c9 pher\u014d 'carry'; commonly known as comb jellies) is a phylum of animals that live in marine waters worldwide. Their most distinctive feature is the \u2018combs\u2019 \u2013 groups of cilia which they use for swimming \u2013 they are the largest animals that swim by means of cilia. Adults of various species range from a few millimeters to 1.5 m (4 ft 11 in) in size. Like cnidarians, their bodies consist of a mass of jelly, with one layer of cells on the outside and another lining the internal cavity. In ctenophores, these layers are two cells deep, while those in cnidarians are only one cell deep. Some authors combined ctenophores and cnidarians in one phylum, Coelenterata, as both groups rely on water flow through the body cavity for both digestion and respiration. Increasing awareness of the differences persuaded more recent authors to classify them as separate phyla."], ["Fresno (/\u02c8fr\u025bzno\u028a/ FREZ-noh), the county seat of Fresno County, is a city in the U.S. state of California. As of 2015, the city's population was 520,159, making it the fifth-largest city in California, the largest inland city in California and the 34th-largest in the nation. Fresno is in the center of the San Joaquin Valley and is the largest city in the Central Valley, which contains the San Joaquin Valley. It is approximately 220 miles (350 km) northwest of Los Angeles, 170 miles (270 km) south of the state capital, Sacramento, or 185 miles (300 km) south of San Francisco. The name Fresno means \"ash tree\" in Spanish, and an ash leaf is featured on the city's flag."], ["Starting in the late 1950s, American computer scientist Paul Baran developed the concept Distributed Adaptive Message Block Switching with the goal to provide a fault-tolerant, efficient routing method for telecommunication messages as part of a research program at the RAND Corporation, funded by the US Department of Defense. This concept contrasted and contradicted the theretofore established principles of pre-allocation of network bandwidth, largely fortified by the development of telecommunications in the Bell System. The new concept found little resonance among network implementers until the independent work of Donald Davies at the National Physical Laboratory (United Kingdom) (NPL) in the late 1960s. Davies is credited with coining the modern name packet switching and inspiring numerous packet switching networks in Europe in the decade following, including the incorporation of the concept in the early ARPANET in the United States."], ["The Black Death is thought to have originated in the arid plains of Central Asia, where it then travelled along the Silk Road, reaching Crimea by 1343. From there, it was most likely carried by Oriental rat fleas living on the black rats that were regular passengers on merchant ships. Spreading throughout the Mediterranean and Europe, the Black Death is estimated to have killed 30\u201360% of Europe's total population. In total, the plague reduced the world population from an estimated 450 million down to 350\u2013375 million in the 14th century. The world population as a whole did not recover to pre-plague levels until the 17th century. The plague recurred occasionally in Europe until the 19th century."], ["There are three major types of rock: igneous, sedimentary, and metamorphic. The rock cycle is an important concept in geology which illustrates the relationships between these three types of rock, and magma. When a rock crystallizes from melt (magma and/or lava), it is an igneous rock. This rock can be weathered and eroded, and then redeposited and lithified into a sedimentary rock, or be turned into a metamorphic rock due to heat and pressure that change the mineral content of the rock which gives it a characteristic fabric. The sedimentary rock can then be subsequently turned into a metamorphic rock due to heat and pressure and is then weathered, eroded, deposited, and lithified, ultimately becoming a sedimentary rock. Sedimentary rock may also be re-eroded and redeposited, and metamorphic rock may also undergo additional metamorphism. All three types of rocks may be re-melted; when this happens, a new magma is formed, from which an igneous rock may once again crystallize."], ["Newcastle upon Tyne (RP: i/\u02ccnju\u02d0k\u0251\u02d0s\u0259l \u0259\u02ccp\u0252n \u02c8ta\u026an/; Locally: i/nju\u02d0\u02cck\u00e6s\u0259l \u0259\u02ccp\u0259n \u02c8ta\u026an/), commonly known as Newcastle, is a city in Tyne and Wear, North East England, 103 miles (166 km) south of Edinburgh and 277 miles (446 km) north of London on the northern bank of the River Tyne, 8.5 mi (13.7 km) from the North Sea. Newcastle is the most populous city in the North East and Tyneside the eighth most populous conurbation in the United Kingdom. Newcastle is a member of the English Core Cities Group and is a member of the Eurocities network of European cities. Newcastle was part of the county of Northumberland until 1400, when it became a county itself, a status it retained until becoming part of Tyne and Wear in 1974.[not in citation given] The regional nickname and dialect for people from Newcastle and the surrounding area is Geordie."], ["The Victoria and Albert Museum (often abbreviated as the V&A), London, is the world's largest museum of decorative arts and design, housing a permanent collection of over 4.5 million objects. It was founded in 1852 and named after Queen Victoria and Prince Albert. The V&A is located in the Brompton district of the Royal Borough of Kensington and Chelsea, in an area that has become known as \"Albertopolis\" because of its association with Prince Albert, the Albert Memorial and the major cultural institutions with which he was associated. These include the Natural History Museum, the Science Museum and the Royal Albert Hall. The museum is a non-departmental public body sponsored by the Department for Culture, Media and Sport. Like other national British museums, entrance to the museum has been free since 2001."], ["The American Broadcasting Company (ABC) (stylized in its logo as abc since 1957) is an American commercial broadcast television network that is owned by the Disney\u2013ABC Television Group, a subsidiary of Disney Media Networks division of The Walt Disney Company. The network is part of the Big Three television networks. The network is headquartered on Columbus Avenue and West 66th Street in Manhattan, with additional major offices and production facilities in New York City, Los Angeles and Burbank, California."], ["He came to power by uniting many of the nomadic tribes of Northeast Asia. After founding the Mongol Empire and being proclaimed \"Genghis Khan\", he started the Mongol invasions that resulted in the conquest of most of Eurasia. These included raids or invasions of the Qara Khitai, Caucasus, Khwarezmid Empire, Western Xia and Jin dynasties. These campaigns were often accompanied by wholesale massacres of the civilian populations \u2013 especially in the Khwarezmian and Xia controlled lands. By the end of his life, the Mongol Empire occupied a substantial portion of Central Asia and China."], ["The word pharmacy is derived from its root word pharma which was a term used since the 15th\u201317th centuries. However, the original Greek roots from pharmakos imply sorcery or even poison. In addition to pharma responsibilities, the pharma offered general medical advice and a range of services that are now performed solely by other specialist practitioners, such as surgery and midwifery. The pharma (as it was referred to) often operated through a retail shop which, in addition to ingredients for medicines, sold tobacco and patent medicines. Often the place that did this was called an apothecary and several languages have this as the dominant term, though their practices are more akin to a modern pharmacy, in English the term apothecary would today be seen as outdated or only approproriate if herbal remedies were on offer to a large extent. The pharmas also used many other herbs not listed. The Greek word Pharmakeia (Greek: \u03c6\u03b1\u03c1\u03bc\u03b1\u03ba\u03b5\u03af\u03b1) derives from pharmakon (\u03c6\u03ac\u03c1\u03bc\u03b1\u03ba\u03bf\u03bd), meaning \"drug\", \"medicine\" (or \"poison\").[n 1]"], ["The immune system is a system of many biological structures and processes within an organism that protects against disease. To function properly, an immune system must detect a wide variety of agents, known as pathogens, from viruses to parasitic worms, and distinguish them from the organism's own healthy tissue. In many species, the immune system can be classified into subsystems, such as the innate immune system versus the adaptive immune system, or humoral immunity versus cell-mediated immunity. In humans, the blood\u2013brain barrier, blood\u2013cerebrospinal fluid barrier, and similar fluid\u2013brain barriers separate the peripheral immune system from the neuroimmune system which protects the brain."], ["One of its earliest massive implementations was brought about by Egyptians against the British occupation in the 1919 Revolution. Civil disobedience is one of the many ways people have rebelled against what they deem to be unfair laws. It has been used in many nonviolent resistance movements in India (Gandhi's campaigns for independence from the British Empire), in Czechoslovakia's Velvet Revolution and in East Germany to oust their communist governments, In South Africa in the fight against apartheid, in the American Civil Rights Movement, in the Singing Revolution to bring independence to the Baltic countries from the Soviet Union, recently with the 2003 Rose Revolution in Georgia and the 2004 Orange Revolution in Ukraine, among other various movements worldwide."], ["Construction is the process of constructing a building or infrastructure. Construction differs from manufacturing in that manufacturing typically involves mass production of similar items without a designated purchaser, while construction typically takes place on location for a known client. Construction as an industry comprises six to nine percent of the gross domestic product of developed countries. Construction starts with planning,[citation needed] design, and financing and continues until the project is built and ready for use."], ["Private schools, also known as independent schools, non-governmental, or nonstate schools, are not administered by local, state or national governments; thus, they retain the right to select their students and are funded in whole or in part by charging their students tuition, rather than relying on mandatory taxation through public (government) funding; at some private schools students may be able to get a scholarship, which makes the cost cheaper, depending on a talent the student may have (e.g. sport scholarship, art scholarship, academic scholarship), financial need, or tax credit scholarships that might be available."], ["Established originally by the Massachusetts legislature and soon thereafter named for John Harvard (its first benefactor), Harvard is the United States' oldest institution of higher learning, and the Harvard Corporation (formally, the President and Fellows of Harvard College) is its first chartered corporation. Although never formally affiliated with any denomination, the early College primarily trained Congregationalist and Unitarian clergy. Its curriculum and student body were gradually secularized during the 18th century, and by the 19th century Harvard had emerged as the central cultural establishment among Boston elites. Following the American Civil War, President Charles W. Eliot's long tenure (1869\u20131909) transformed the college and affiliated professional schools into a modern research university; Harvard was a founding member of the Association of American Universities in 1900. James Bryant Conant led the university through the Great Depression and World War II and began to reform the curriculum and liberalize admissions after the war. The undergraduate college became coeducational after its 1977 merger with Radcliffe College."], ["Jacksonville is the largest city by population in the U.S. state of Florida, and the largest city by area in the contiguous United States. It is the county seat of Duval County, with which the city government consolidated in 1968. Consolidation gave Jacksonville its great size and placed most of its metropolitan population within the city limits; with an estimated population of 853,382 in 2014, it is the most populous city proper in Florida and the Southeast, and the 12th most populous in the United States. Jacksonville is the principal city in the Jacksonville metropolitan area, with a population of 1,345,596 in 2010."], ["A study by the World Institute for Development Economics Research at United Nations University reports that the richest 1% of adults alone owned 40% of global assets in the year 2000. The three richest people in the world possess more financial assets than the lowest 48 nations combined. The combined wealth of the \"10 million dollar millionaires\" grew to nearly $41 trillion in 2008. A January 2014 report by Oxfam claims that the 85 wealthiest individuals in the world have a combined wealth equal to that of the bottom 50% of the world's population, or about 3.5 billion people. According to a Los Angeles Times analysis of the report, the wealthiest 1% owns 46% of the world's wealth; the 85 richest people, a small part of the wealthiest 1%, own about 0.7% of the human population's wealth, which is the same as the bottom half of the population. More recently, in January 2015, Oxfam reported that the wealthiest 1 percent will own more than half of the global wealth by 2016. An October 2014 study by Credit Suisse also claims that the top 1% now own nearly half of the world's wealth and that the accelerating disparity could trigger a recession. In October 2015, Credit Suisse published a study which shows global inequality continues to increase, and that half of the world's wealth is now in the hands of those in the top percentile, whose assets each exceed $759,900. A 2016 report by Oxfam claims that the 62 wealthiest individuals own as much wealth as the poorer half of the global population combined. Oxfam's claims have however been questioned on the basis of the methodology used: by using net wealth (adding up assets and subtracting debts), the Oxfam report, for instance, finds that there are more poor people in the United States and Western Europe than in China (due to a greater tendency to take on debts).[unreliable source?][unreliable source?] Anthony Shorrocks, the lead author of the Credit Suisse report which is one of the sources of Oxfam's data, considers the criticism about debt to be a \"silly argument\" and \"a non-issue . . . a diversion.\""], ["Doctor Who is a British science-fiction television programme produced by the BBC since 1963. The programme depicts the adventures of the Doctor, a Time Lord\u2014a space and time-travelling humanoid alien. He explores the universe in his TARDIS, a sentient time-travelling space ship. Its exterior appears as a blue British police box, which was a common sight in Britain in 1963 when the series first aired. Accompanied by companions, the Doctor combats a variety of foes, while working to save civilisations and help people in need."], ["The University of Chicago (UChicago, Chicago, or U of C) is a private research university in Chicago. The university, established in 1890, consists of The College, various graduate programs, interdisciplinary committees organized into four academic research divisions and seven professional schools. Beyond the arts and sciences, Chicago is also well known for its professional schools, which include the Pritzker School of Medicine, the University of Chicago Booth School of Business, the Law School, the School of Social Service Administration, the Harris School of Public Policy Studies, the Graham School of Continuing Liberal and Professional Studies and the Divinity School. The university currently enrolls approximately 5,000 students in the College and around 15,000 students overall."], ["The Yuan dynasty (Chinese: \u5143\u671d; pinyin: Yu\u00e1n Ch\u00e1o), officially the Great Yuan (Chinese: \u5927\u5143; pinyin: D\u00e0 Yu\u00e1n; Mongolian: Yehe Yuan Ulus[a]), was the empire or ruling dynasty of China established by Kublai Khan, leader of the Mongolian Borjigin clan. Although the Mongols had ruled territories including today's North China for decades, it was not until 1271 that Kublai Khan officially proclaimed the dynasty in the traditional Chinese style. His realm was, by this point, isolated from the other khanates and controlled most of present-day China and its surrounding areas, including modern Mongolia and Korea. It was the first foreign dynasty to rule all of China and lasted until 1368, after which its Genghisid rulers returned to their Mongolian homeland and continued to rule the Northern Yuan dynasty. Some of the Mongolian Emperors of the Yuan mastered the Chinese language, while others only used their native language (i.e. Mongolian) and the 'Phags-pa script."], ["Kenya (/\u02c8k\u025bnj\u0259/; locally [\u02c8k\u025b\u0272a] ( listen)), officially the Republic of Kenya, is a country in Africa and a founding member of the East African Community (EAC). Its capital and largest city is Nairobi. Kenya's territory lies on the equator and overlies the East African Rift covering a diverse and expansive terrain that extends roughly from Lake Victoria to Lake Turkana (formerly called Lake Rudolf) and further south-east to the Indian Ocean. It is bordered by Tanzania to the south, Uganda to the west, South Sudan to the north-west, Ethiopia to the north and Somalia to the north-east. Kenya covers 581,309 km2 (224,445 sq mi), and had a population of approximately 45 million people in July 2014."], ["The Intergovernmental Panel on Climate Change (IPCC) is a scientific intergovernmental body under the auspices of the United Nations, set up at the request of member governments. It was first established in 1988 by two United Nations organizations, the World Meteorological Organization (WMO) and the United Nations Environment Programme (UNEP), and later endorsed by the United Nations General Assembly through Resolution 43/53. Membership of the IPCC is open to all members of the WMO and UNEP. The IPCC produces reports that support the United Nations Framework Convention on Climate Change (UNFCCC), which is the main international treaty on climate change. The ultimate objective of the UNFCCC is to \"stabilize greenhouse gas concentrations in the atmosphere at a level that would prevent dangerous anthropogenic [i.e., human-induced] interference with the climate system\". IPCC reports cover \"the scientific, technical and socio-economic information relevant to understanding the scientific basis of risk of human-induced climate change, its potential impacts and options for adaptation and mitigation.\""], ["Chloroplasts' main role is to conduct photosynthesis, where the photosynthetic pigment chlorophyll captures the energy from sunlight and converts it and stores it in the energy-storage molecules ATP and NADPH while freeing oxygen from water. They then use the ATP and NADPH to make organic molecules from carbon dioxide in a process known as the Calvin cycle. Chloroplasts carry out a number of other functions, including fatty acid synthesis, much amino acid synthesis, and the immune response in plants. The number of chloroplasts per cell varies from 1 in algae up to 100 in plants like Arabidopsis and wheat."], ["A prime number (or a prime) is a natural number greater than 1 that has no positive divisors other than 1 and itself. A natural number greater than 1 that is not a prime number is called a composite number. For example, 5 is prime because 1 and 5 are its only positive integer factors, whereas 6 is composite because it has the divisors 2 and 3 in addition to 1 and 6. The fundamental theorem of arithmetic establishes the central role of primes in number theory: any integer greater than 1 can be expressed as a product of primes that is unique up to ordering. The uniqueness in this theorem requires excluding 1 as a prime because one can include arbitrarily many instances of 1 in any factorization, e.g., 3, 1 \u00b7 3, 1 \u00b7 1 \u00b7 3, etc. are all valid factorizations of 3."], ["The Rhine (Romansh: Rein, German: Rhein, French: le Rhin, Dutch: Rijn) is a European river that begins in the Swiss canton of Graub\u00fcnden in the southeastern Swiss Alps, forms part of the Swiss-Austrian, Swiss-Liechtenstein border, Swiss-German and then the Franco-German border, then flows through the Rhineland and eventually empties into the North Sea in the Netherlands. The biggest city on the river Rhine is Cologne, Germany with a population of more than 1,050,000 people. It is the second-longest river in Central and Western Europe (after the Danube), at about 1,230 km (760 mi),[note 2][note 1] with an average discharge of about 2,900 m3/s (100,000 cu ft/s)."], ["Following a referendum in 1997, in which the Scottish electorate voted for devolution, the current Parliament was convened by the Scotland Act 1998, which sets out its powers as a devolved legislature. The Act delineates the legislative competence of the Parliament \u2013 the areas in which it can make laws \u2013 by explicitly specifying powers that are \"reserved\" to the Parliament of the United Kingdom. The Scottish Parliament has the power to legislate in all areas that are not explicitly reserved to Westminster. The British Parliament retains the ability to amend the terms of reference of the Scottish Parliament, and can extend or reduce the areas in which it can make laws. The first meeting of the new Parliament took place on 12 May 1999."], ["Islamism, also known as Political Islam (Arabic: \u0625\u0633\u0644\u0627\u0645 \u0633\u064a\u0627\u0633\u064a\u200e isl\u0101m siy\u0101s\u012b), is an Islamic revival movement often characterized by moral conservatism, literalism, and the attempt \"to implement Islamic values in all spheres of life.\" Islamism favors the reordering of government and society in accordance with the Shari'a. The different Islamist movements have been described as \"oscillating between two poles\": at one end is a strategy of Islamization of society through state power seized by revolution or invasion; at the other \"reformist\" pole Islamists work to Islamize society gradually \"from the bottom up\". The movements have \"arguably altered the Middle East more than any trend since the modern states gained independence\", redefining \"politics and even borders\" according to one journalist (Robin Wright)."], ["Imperialism is a type of advocacy of empire. Its name originated from the Latin word \"imperium\", which means to rule over large territories. Imperialism is \"a policy of extending a country's power and influence through colonization, use of military force, or other means\". Imperialism has greatly shaped the contemporary world. It has also allowed for the rapid spread of technologies and ideas. The term imperialism has been applied to Western (and Japanese) political and economic dominance especially in Asia and Africa in the 19th and 20th centuries. Its precise meaning continues to be debated by scholars. Some writers, such as Edward Said, use the term more broadly to describe any system of domination and subordination organised with an imperial center and a periphery."], ["The United Methodist Church (UMC) is a mainline Protestant Methodist denomination. In the 19th century its main predecessor was a leader in Evangelicalism. Founded in 1968 by the union of the Methodist Church (USA) and the Evangelical United Brethren Church, the UMC traces its roots back to the revival movement of John and Charles Wesley in England as well as the Great Awakening in the United States. As such, the church's theological orientation is decidedly Wesleyan. It embraces both liturgical and evangelical elements."], ["The French and Indian War (1754\u20131763) was the North American theater of the worldwide Seven Years' War. The war was fought between the colonies of British America and New France, with both sides supported by military units from their parent countries of Great Britain and France, as well as Native American allies. At the start of the war, the French North American colonies had a population of roughly 60,000 European settlers, compared with 2 million in the British North American colonies. The outnumbered French particularly depended on the Indians. Long in conflict, the metropole nations declared war on each other in 1756, escalating the war from a regional affair into an intercontinental conflict."], ["Philosophers in antiquity used the concept of force in the study of stationary and moving objects and simple machines, but thinkers such as Aristotle and Archimedes retained fundamental errors in understanding force. In part this was due to an incomplete understanding of the sometimes non-obvious force of friction, and a consequently inadequate view of the nature of natural motion. A fundamental error was the belief that a force is required to maintain motion, even at a constant velocity. Most of the previous misunderstandings about motion and force were eventually corrected by Galileo Galilei and Sir Isaac Newton. With his mathematical insight, Sir Isaac Newton formulated laws of motion that were not improved-on for nearly three hundred years. By the early 20th century, Einstein developed a theory of relativity that correctly predicted the action of forces on objects with increasing momenta near the speed of light, and also provided insight into the forces produced by gravitation and inertia."]], "questions": [["Which NFL team represented the AFC at Super Bowl 50?", "Which NFL team represented the NFC at Super Bowl 50?", "Where did Super Bowl 50 take place?", "Which NFL team won Super Bowl 50?", "What color was used to emphasize the 50th anniversary of the Super Bowl?", "What was the theme of Super Bowl 50?", "What day was the game played on?", "What is the AFC short for?", "What was the theme of Super Bowl 50?", "What does AFC stand for?", "What day was the Super Bowl played on?", "Who won Super Bowl 50?", "What venue did Super Bowl 50 take place in?", "What city did Super Bowl 50 take place in?", "If Roman numerals were used, what would Super Bowl 50 have been called?", "Super Bowl 50 decided the NFL champion for what season?", "What year did the Denver Broncos secure a Super Bowl title for the third time?", "What city did Super Bowl 50 take place in?", "What stadium did Super Bowl 50 take place in?", "What was the final score of Super Bowl 50? ", "What month, day and year did Super Bowl 50 take place? ", "What year was Super Bowl 50?", "What team was the AFC champion?", "What team was the NFC champion?", "Who won Super Bowl 50?", "Super Bowl 50 determined the NFL champion for what season?", "Which team won Super Bowl 50.", "Where was Super Bowl 50 held?", "The name of the NFL championship game is?", "What 2015 NFL team one the AFC playoff?"], ["What was Maria Curie the first female recipient of?", "What year was Casimir Pulaski born in Warsaw?", "Who was one of the most famous people born in Warsaw?", "Who was Fr\u00e9d\u00e9ric Chopin?", "How old was Chopin when he moved to Warsaw with his family?"], ["In what country is Normandy located?", "When were the Normans in Normandy?", "From which countries did the Norse originate?", "Who was the Norse leader?", "What century did the Normans first gain their separate identity?"], ["In what year was Nikola Tesla born? ", "What was Nikola Tesla's ethnicity?", "In what year did Tesla die? ", "When was Nikola Tesla born?", "In what year did Tesla die?", "What is Tesla's home country?", "What does AC stand for?"], ["What branch of theoretical computer science deals with broadly classifying computational problems by difficulty and class of relationship?", "By what main attribute are computational problems classified utilizing computational complexity theory? ", "What is the term for a task that generally lends itself to being solved by a computer?"], ["What is a course of study called?", "What is another name to describe the science of teaching?", "Where do most teachers get their credentials from?", "What can a teacher use to help students learn?", "Where is a teacher most likely to be teaching at?"], ["Of what nationality was Martin Luther?", "When did Martin Luther die?", "What organization's teaching did Luther reject?", "What did the Church claim could be avoided with money?", "What did the Church do when Luther refused to retract his writings?"], ["What is Southern California often abbreviated as?", "Despite being traditionall described as \"eight counties\", how many counties does this region actually have?", "What is a major importance of Southern California in relation to California and the United States?", "What are the ties that best described what the \"eight counties\" are based on?", "The reasons for the las two counties to be added are based on what?"], ["What company was formed by the merger of Sky Television and British Satellite Broadcasting?", "Who is the UK's largest digital subscription television company?", "What year did BSkyB acquire Sky Italia?", "What is the name of the holding company for BSkyB?", "What is the name of the United Kingdom operation for BSkyB?"], ["What kind of economy does Victoria have?", "Where according to gross state product does Victoria rank in Australia?", "At what rank does GPS per capita set Victoria?", "What city in Victoria is called the sporting capital of Australia?", "What is the largest stadium in Australia?"], ["Where was France's Huguenot population largely centered?", "What was the proportion of Huguenots to Catholics at their peak?", "When were the Wars of Religion fought?", "What treaty ended the Wars of Religion?", "What did this agreement do?"], ["Along with geothermal and nuclear, what is a notable non-combustion heat source?", "What ideal thermodynamic cycle analyzes the process by which steam engines work?", "In the Rankine cycle, what does water turn into when heated?", "At what pressure is water heated in the Rankine cycle?", "What types of engines are steam engines?"], ["The atomic number of the periodic table for oxygen?", "What is the second most abundant element?", "How many atoms combine to form dioxygen?", "Roughly, how much oxygen makes up the Earth crust?", "Which gas makes up 20.8% of the Earth's atmosphere?", "How much of the earth's atmosphere is diatomic oxygen?", "What element makes up almost half of the earth's crust by mass?", "What is the atomic number for oxygen?", "Are atmospheric oxygen levels going up, down, or staying the same?", "What are the three most abundent elements of the universe by mass?", "What is the atomic number of the element oxygen?", "Of what group in the periodic table is oxygen a member?", "What type of compounds does oxygen most commonly form?", "Compared to other elements, how abundant does oxygen rank?", "Under normal conditions, what do two atoms of oxygen form?"], ["When did the 1973 oil crisis begin?", "What was the price of oil in March of 1974?", "When was the second oil crisis?", "What was another term used for the oil crisis?", "Who proclaimed the oil embargo?"], ["What project put the first Americans into space?", "What program was created to carry out these projects and missions?", "What year did the first manned Apollo flight occur?", "What President is credited with the original notion of putting Americans in space?", "How many people were on the project that followed Project Mercury?"], ["What are the three sources of European Union law?", "What is European Union Law?", "What are the main sources of primary law?", "What are the secondary sources of primary law?", "What are the two bodies that make up the European Union's legislature?", "What is European Union law?", "What effect does European Union law have on laws of member states?", "What are the three sources of European Union law?", "What are the main legislative bodies of the European Union?", "What are the three main sources of European Union law?", "What are the main sources of primary law?", "What is the legislature of the European Union comprised of?", "How many sources of European Union law are there?"], ["Which name is also used to describe the Amazon rainforest in English?", "How many square kilometers of rainforest is covered in the basin?", "How many nations control this region in total?", "How many nations contain \"Amazonas\" in their names?", "What percentage does the Amazon represents in rainforests on the planet?", "What is the Dutch word for the Amazon rainforest?", "What rainforest covers the majority of the Amazon basin in South America?", "In what country can most of the Amazon rainforest be found?", "The Amazon rainforest makes up what amount of Earth's rainforests?", "How many species of trees can be found in the Amazon rainforest?", "What kind of forest is the Amazon rainforest?", "How many square kilometers is the Amazon Basin?", "How many nations are within the Amazon Basin?", "Which nation contains the majority of the amazon forest?", "What is the estimate for the amount of tree species in the amazon tropical rain forest?"], ["What are Ctenophora commonly known as?", "Where do Ctenophora live?", "What size are adult Ctenophora?", "What is a ctenophora?", "What does the ctenophora use to swim?", "What does ctenophora use for digestion and respiration?", "How large can ctenophora grow?", "What is the most distinctive feature of ctenophora?", "What are ctenophora commonly known as?", "How big can ctenophora grow?", "What does ctenophora rely on for digestion and respiration?", "What does ctenophora mean in Greek?", "Where do ctenophora live?"], ["Which city is the fifth-largest city in California?", "How far is Fresno from Los Angeles?", "What does the name Fresno mean in Spanish?", "What is featured on the city of Fresno's city flag?", "How do you pronounce Fresno?"], ["What did Paul Baran develop ", "What did Distributed Adaptive Message Block Switching do", "What did this concept contradict ", "What is Donald Davies credited with ", "What did Paul Baran develop in the late 1950's", "What was the goal of the system ", "Who is credited with the modern name for this system "], ["Where did the black death originate?", "How did the black death make it to the Mediterranean and Europe?", "How much of the European population did the black death kill?", "When did the world's population finally recover from the black death?", "For how long did the plague stick around?"], ["An igneous rock is a rock that crystallizes from what?", "Sedimentary rock can be turned into which of the three types of rock?", "When the three types of rock are re-melted what is formed? ", "What are the three major types of rock? ", "What changes the mineral content of a rock?"], ["How many miles south of Edinburgh is Newcastle?", "How many miles from the north Sea is Newcastle?", "What network is Newcastle a member of?", "What county was Newcastle a part of until 1400?", "What's the regional nickname for Newcastle and its surrounding area?"], ["where is the Victoria and Albert Museum located?", "how many permanent objects are located there?", "when was the Victoria and Albert museum founded?", "Who is the museum named for?", "In which London borough is the Victoria and Albert Museum located?", "In what year was the Victoria and Albert Museum founded?", "Which monarchs was the Victoria and Albert Museum named after?", "Which department sponsors the Victoria and Albert Museum?", "In which year did the museum started charging free admission fees?"], ["What company owns the American Broadcasting Company?", "In what year did ABC stylize it's logo as abc?", "In what borough of New York City is ABC headquartered?", "On what streets is the ABC headquarters located", "Disney-ABC Television Group is a subsidiary of what division of the Walt Disney Company?"], ["What do we call the empire that Genghis Khan founded?", "Who did Genghis Khan unite before he began conquering the rest of Eurasia?", "In which regions in particular did Genghis Khan's armies massacre civilians?", "What areas did Genghis Khan control at the end of his life?", "Which other empires or dynasties did Genghis Khan conquer?"], ["What word is the word pharmacy taken from?", "What goods were sold in a pharma?", "What did the Greek root pharmakos imply?", "How would the word apothecary be viewed by contemporary English speakers?", "What else was used by pharmas?"], ["What is the immune system?", "What does the immune system protect against?", "What are two of its subsystems?", "What is the subsystem that protects the human brain?", "What is the immune system?", "What does the immune system protect against?", "What are two of its subsystems?", "The immune system protects organisms against what?", "What are the agents the immune system detects known as?", "Which part of the immune system protects the brain?", "What separates the neuroimmune system and peripheral immune system in humans?", "What are the agents detected by the immune system called?", "What are the two major subsystems of the immune system?", "What are the two different types of immunity?", "What is the immune system of the brained known as?"], ["What is it called when people in society rebel against laws they think are unfair?", "What is an example of major civil disobedience in South Africa?", "What was the the movement called that brought Baltic countries independence from the Soviet Union?", "In 2004 the Orange revolution occurred in what country?", "Where was the location of the 2003 Rose revolution?", "Which people brought forward one of the earliest examples of Civil Disobedience?", "Who was one of the earliest examples of Civil Disobedience against?", "Why do people chose civil disobedience to protest?", "What does civil disobedience protest against?", "What civil rights movement in the US was known for it's disobedience?"], ["What is the process of constructing a building or infrastructure?", "What typically involves mass production of similar items without a designated purchaser?", "What percentile of gross domestic product is construction comprised of?", "What three things are needed for construction to take place?", "Construction takes place on location for who?"], ["Along with non-governmental and nonstate schools, what is another name for private schools?", "Along with sport and art, what is a type of talent scholarship?", "Rather than taxation, what are private schools largely funded by?", "What right do private schools have that public schools don't?"], ["What individual is the school named after?", "When did the undergraduate program become coeducational?", "What was the name of the leader through the Great Depression and World War II?", "What organization did Harvard found in 1900?", "What president of the university transformed it into a modern research university?"], ["Which Florida city has the biggest population?", "What was the population Jacksonville city as of 2010?", "Based on population alone, what is Jacksonville's ranking in the United States?", "In which county does Jacksonville reside?", "What year did consolidation cause Jacksonville to become part of Duval County?"], ["What percentage of global assets does the richest 1% of people have?", "According to Oxfam, the 85 richest people have wealth equal to how many average people?", "In order to be considered in the top percentile, a person would need to amass how much money each year?", "What has caused Oxfam's findings to be questioned?", "Why does Oxfam and Credit Suisse believe their findings are being doubted?", "What percent of the global assets in 2000 were owned by just 1% of adults?", "What do the three richest people in the world posses more of than the lowest 48 nations together?", "How much was the combined wealth of the \"10 Million dollar millionaires\" in 2008?", "How much of the global wealth will the wealthiest 1 percent own by 2016?", "Why are there more poor people in the United States and Europe than China?"], ["Who is the producer of Doctor Who?", "What year did Doctor Who first show on TV?", "What is Doctor Who's space ship called?", "What does the outside of the Tardis resemble?", "What type/genre of TV show is Doctor Who?"], ["What kind of university is the University of Chicago?", "When was the University of Chicago established?", "How many professional schools does the University of Chicago have?", "How many academic research divisions does the University of Chicago have?", "How many students does the University of Chicago have enlisted?"], ["What is the Chinese name for the Yuan dynasty?", "What is the Yuan dynasty's official name?", "Who started the Yuan dynasty?", "Who led the Mongolian Borjigin clan?", "When did Khan formally declare the Yuan dynasty?"], ["Where is Kenya located?", "What is Kenya a founding member of?", "What is the capitol of Kenya?", "What country boarders the south of Kenya?", "What was the population of Kenya in 2014?"], ["What organization is the IPCC a part of?", "What UN organizations established the IPCC?", "What does the UN want to stabilize?", "What is the UN's climate change treaty?", "What UN resolution endorsed the IPCC?"], ["What is the primary purpose of chloroplasts?", "What does ATP store?", "What does NADPH store?", "What is the process of turning CO2 into organic molecules called?", "How many chloroplasts per cell does algae have?"], ["What is the only divisor besides 1 that a prime number can have?", "What are numbers greater than 1 that can be divided by 3 or more numbers called?", "What theorem defines the main role of primes in number theory?", "Any number larger than 1 can be represented as a product of what?", "Why must one be excluded in order to preserve the uniqueness of the fundamental theorem?"], ["Where does the Rhine begin? ", "Where does the Rhine empty?", "What is the largest city the Rhine runs through? ", "What river is larger than the Rhine?", "How long is the Rhine?", "Where is the Rhine? ", "What country does the Rhine empty?", "How long is the Rhine? "], ["When was the current parliament of Scotland convened?", "What act set out the Parliament's powers as a devolved legislature?", "The legislative competence of the Parliament species what areas?", "To what body are certain powers explicitly specified as being reserved for?", "The Scottish Parliament may legislate as it pleases as long as the powers aren't already reserved by where?"], ["What is an Islamic revival movement?", "What aspects of life does Islamism seek to integrate itself into?", "What goal does Islamism have when it comes to society and government?", "What have the two different Islamist movements been described as oscillating between?", "One strategy of Islamization is to seize power by what methods?"], ["The word imperialism has it's origins in which ancient language? ", "By what means is imperialism usually administered?", "The term imperialism has been applied to western countries, and which eastern county?", "Imperialism is responsible for the rapid spread of what?"], ["What does UMC stand for?", "What is the United Methodist Church?", "When was the UMC founded?", "Who founded the UMC?", "What is the church's theological orientation?"], ["When was the French and Indian War?", "Who fought in the French and Indian war?", "How many people were in French North American Colonies?", "How many people were in British North American Colonies?"], ["What concept did philosophers in antiquity use to study simple machines?", "What was the belief that maintaining motion required force?", "Who had mathmatical insite?", "How long did it take to improve on Sir Isaac Newton's laws of motion?", "Who develped the theory of relativity?"]]} \ No newline at end of file diff --git a/lite/examples/bert_qa/android/lib_task_api/src/main/java/org/tensorflow/lite/examples/bertqa/ml/QaClient.java b/lite/examples/bert_qa/android/lib_task_api/src/main/java/org/tensorflow/lite/examples/bertqa/ml/QaClient.java index 2d8020c8f34..67df0779fae 100644 --- a/lite/examples/bert_qa/android/lib_task_api/src/main/java/org/tensorflow/lite/examples/bertqa/ml/QaClient.java +++ b/lite/examples/bert_qa/android/lib_task_api/src/main/java/org/tensorflow/lite/examples/bertqa/ml/QaClient.java @@ -12,7 +12,7 @@ import java.util.List; /** - * Load TfLite model and provide predictions with task api. + * Load TFLite model and create BertQuestionAnswerer instance. */ public class QaClient { From 1b9c3aef492d9d239fd9271a86c7e8410caa995d Mon Sep 17 00:00:00 2001 From: Sunit Roy <2703iamsry@gmail.com> Date: Mon, 2 Aug 2021 23:12:33 +0530 Subject: [PATCH 7/7] replaced the links with relative links --- .../examples/bert_qa/android/EXPLORE_THE_CODE.md | 16 ++++++++-------- lite/examples/bert_qa/android/README.md | 4 ++-- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/lite/examples/bert_qa/android/EXPLORE_THE_CODE.md b/lite/examples/bert_qa/android/EXPLORE_THE_CODE.md index e23c9ea1187..21a6d3159ac 100644 --- a/lite/examples/bert_qa/android/EXPLORE_THE_CODE.md +++ b/lite/examples/bert_qa/android/EXPLORE_THE_CODE.md @@ -23,18 +23,18 @@ file This BERT QA Android reference app demonstrates two implementation solutions, -[`lib_task_api`](https://github.com/SunitRoy2703/examples/tree/bertQa-android-task-lib/lite/examples/bert_qa/android/lib_task_api) +[`lib_task_api`](/lite/examples/bert_qa/android/lib_task_api) that leverages the out-of-box API from the [TensorFlow Lite Task Library](https://www.tensorflow.org/lite/inference_with_metadata/task_library/bert_question_answerer), and -[`lib_interpreter`](https://github.com/SunitRoy2703/examples/tree/bertQa-android-task-lib/lite/examples/bert_qa/android/lib_interpreter) +[`lib_interpreter`](/lite/examples/bert_qa/android/lib_interpreter) that creates the custom inference pipleline using the [TensorFlow Lite Interpreter Java API](https://www.tensorflow.org/lite/guide/inference#load_and_run_a_model_in_java). Both solutions implement the file `QaClient.java` (see -[the one in lib_task_api](https://github.com/SunitRoy2703/examples/blob/bertQa-android-task-lib/lite/examples/bert_qa/android/lib_task_api/src/main/java/org/tensorflow/lite/examples/bertqa/ml/QaClient.java) +[the one in lib_task_api](/lite/examples/bert_qa/android/lib_task_api/src/main/java/org/tensorflow/lite/examples/bertqa/ml/QaClient.java) and -[the one in lib_interpreter](https://github.com/SunitRoy2703/examples/blob/bertQa-android-task-lib/lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/ml/QaClient.java) +[the one in lib_interpreter](/lite/examples/bert_qa/android/lib_interpreter/src/main/java/org/tensorflow/lite/examples/bertqa/ml/QaClient.java) that contains most of the complex logic for processing the text input and running inference. @@ -87,8 +87,8 @@ demo models are single head models, therefore, `results` only contains one `QaAnswer` object. To match the implementation of -[`lib_interpreter`](https://github.com/SunitRoy2703/examples/tree/bertQa-android-task-lib/lite/examples/bert_qa/android/lib_interpreter), -`results` is converted into List<[`Answer`](https://github.com/SunitRoy2703/examples/blob/bertQa-android-task-lib/lite/examples/bert_qa/android/lib_task_api/src/main/java/org/tensorflow/lite/examples/bertqa/ml/Answer.java)>. +[`lib_interpreter`](/lite/examples/bert_qa/android/lib_interpreter), +`results` is converted into List<[`Answer`](/lite/examples/bert_qa/android/lib_task_api/src/main/java/org/tensorflow/lite/examples/bertqa/ml/Answer.java)>. #### Using the TensorFlow Lite Interpreter @@ -145,7 +145,7 @@ tflite.runForMultipleInputsOutputs(inputs, output); The QaClient is invoked and inference results are displayed by the `presentAnswer()` function in -[`QaActivity.java`](https://github.com/SunitRoy2703/examples/blob/bertQa-android-task-lib/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/QaActivity.java). +[`QaActivity.java`](/lite/examples/bert_qa/android/app/src/main/java/org/tensorflow/lite/examples/bertqa/QaActivity.java). ```java private void presentAnswer(Answer answer) { @@ -166,4 +166,4 @@ private void presentAnswer(Answer answer) { textToSpeech.speak(answer.text, TextToSpeech.QUEUE_FLUSH, null, answer.text); } } -``` +``` \ No newline at end of file diff --git a/lite/examples/bert_qa/android/README.md b/lite/examples/bert_qa/android/README.md index 9d47747730b..8fe06715cd7 100644 --- a/lite/examples/bert_qa/android/README.md +++ b/lite/examples/bert_qa/android/README.md @@ -71,12 +71,12 @@ This BERT QA Android reference app demonstrates two implementation solutions: (1) -[`lib_task_api`](https://github.com/SunitRoy2703/examples/tree/bertQa-android-task-lib/lite/examples/bert_qa/android/lib_task_api) +[`lib_task_api`](/lite/examples/bert_qa/android/lib_task_api) that leverages the out-of-box API from the [TensorFlow Lite Task Library](https://www.tensorflow.org/lite/inference_with_metadata/task_library/bert_question_answerer); (2) -[`lib_interpreter`](https://github.com/SunitRoy2703/examples/tree/bertQa-android-task-lib/lite/examples/bert_qa/android/lib_interpreter) +[`lib_interpreter`](/lite/examples/bert_qa/android/lib_interpreter) that creates the custom inference pipleline using the [TensorFlow Lite Interpreter Java API](https://www.tensorflow.org/lite/guide/inference#load_and_run_a_model_in_java).