Skip to content

Commit 77213c4

Browse files
CezaryMarcjanEyal
and
Eyal
authored
Update NLRModels.md (#6102)
* Update NLRModels.md * Remove extra bullet Remove extra bullet and added link to KNN (verify it is the right one) * Update NLRModels.md Co-authored-by: Eyal <[email protected]>
1 parent 75ff4f0 commit 77213c4

File tree

1 file changed

+15
-6
lines changed

1 file changed

+15
-6
lines changed

Orchestrator/docs/NLRModels.md

Lines changed: 15 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -2,25 +2,34 @@
22

33

44

5-
# Natural Language Representation Model
5+
# Prebuilt Language Models
66

7-
8-
Natural Language Representation models are generic language representation models which have been trained towards more sophisticated pretraining tasks for both monolingual as well as multilingual scenarios. Turing NLR models are used as a natural replacement for BERT-like models.
7+
Prebuilt language models have been trained towards more sophisticated tasks for both monolingual as well as multilingual scenarios. In public preview only English models are made available.
98

109
## Models
10+
The public preview of Orchestrator includes the following prebuilt language models available in [versions repository][2].
11+
12+
### pretrained.20200924.microsoft.dte.00.03.en.onnx
13+
This is a fast and small base model with sufficient accuracy but if the accuracy and not speed and memory size is critical then consider other options. It is a 3-layer pretrained BERT model optimized for conversation for example-based use ([KNN][3]).
1114

12-
**TBD**: Describe models in [versions repository][2].
15+
### pretrained.20200924.microsoft.dte.00.06.en.onnx
16+
This is a high quality base model that strikes the balance between size, speed and accuracy. It is a 6-layer pretrained BERT model optimized for conversation for example-based use ([KNN][3]). This is the default model used if none explicitly specified.
1317

18+
### pretrained.20200924.microsoft.dte.00.12.en.onnx
19+
This is a highest quality base model but is larger and slower than other options. It is a 12-layer pretrained BERT model optimized for conversation for example-based use (KNN).
1420

21+
### pretrained.20200924.microsoft.dte.00.12.roberta.en.onnx
22+
This is a high quality base model but it is larger and slower than some other options. It is a 12-layer pretrained RoBERTa model optimized for conversation for example-based use ([KNN][3]).
1523

1624
## References
1725

1826
* [UniLMv2 Paper][1]
1927

20-
* [NLR Versions Repository][2]
28+
* [Base Models Versions Repository][2]
2129

22-
*
30+
* [KNN (K nearest neighbors algorithm)][3]
2331

2432
[1]: https://arxiv.org/abs/2002.12804 "UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training"
2533
[2]: https://aka.ms/nlrversions
34+
[3]: https://en.wikipedia.org/wiki/K-nearest_neighbors_algorithm
2635

0 commit comments

Comments
 (0)