|
2 | 2 |
|
3 | 3 |
|
4 | 4 |
|
5 |
| -# Natural Language Representation Model |
| 5 | +# Prebuilt Language Models |
6 | 6 |
|
7 |
| - |
8 |
| -Natural Language Representation models are generic language representation models which have been trained towards more sophisticated pretraining tasks for both monolingual as well as multilingual scenarios. Turing NLR models are used as a natural replacement for BERT-like models. |
| 7 | +Prebuilt language models have been trained towards more sophisticated tasks for both monolingual as well as multilingual scenarios. In public preview only English models are made available. |
9 | 8 |
|
10 | 9 | ## Models
|
| 10 | +The public preview of Orchestrator includes the following prebuilt language models available in [versions repository][2]. |
| 11 | + |
| 12 | +### pretrained.20200924.microsoft.dte.00.03.en.onnx |
| 13 | +This is a fast and small base model with sufficient accuracy but if the accuracy and not speed and memory size is critical then consider other options. It is a 3-layer pretrained BERT model optimized for conversation for example-based use ([KNN][3]). |
11 | 14 |
|
12 |
| -**TBD**: Describe models in [versions repository][2]. |
| 15 | +### pretrained.20200924.microsoft.dte.00.06.en.onnx |
| 16 | +This is a high quality base model that strikes the balance between size, speed and accuracy. It is a 6-layer pretrained BERT model optimized for conversation for example-based use ([KNN][3]). This is the default model used if none explicitly specified. |
13 | 17 |
|
| 18 | +### pretrained.20200924.microsoft.dte.00.12.en.onnx |
| 19 | +This is a highest quality base model but is larger and slower than other options. It is a 12-layer pretrained BERT model optimized for conversation for example-based use (KNN). |
14 | 20 |
|
| 21 | +### pretrained.20200924.microsoft.dte.00.12.roberta.en.onnx |
| 22 | +This is a high quality base model but it is larger and slower than some other options. It is a 12-layer pretrained RoBERTa model optimized for conversation for example-based use ([KNN][3]). |
15 | 23 |
|
16 | 24 | ## References
|
17 | 25 |
|
18 | 26 | * [UniLMv2 Paper][1]
|
19 | 27 |
|
20 |
| -* [NLR Versions Repository][2] |
| 28 | +* [Base Models Versions Repository][2] |
21 | 29 |
|
22 |
| -* |
| 30 | +* [KNN (K nearest neighbors algorithm)][3] |
23 | 31 |
|
24 | 32 | [1]: https://arxiv.org/abs/2002.12804 "UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training"
|
25 | 33 | [2]: https://aka.ms/nlrversions
|
| 34 | +[3]: https://en.wikipedia.org/wiki/K-nearest_neighbors_algorithm |
26 | 35 |
|
0 commit comments