Adding new Databricks and EMR supported versions

Author: maziyarpanahi

README.md

@@ -271,6 +271,8 @@ Spark NLP 4.2.5 has been tested and is compatible with the following runtimes:
 - 11.1 ML
 - 11.2
 - 11.2 ML
+- 11.3
+- 11.3 ML
 
 **GPU:**
 
@@ -283,6 +285,7 @@ Spark NLP 4.2.5 has been tested and is compatible with the following runtimes:
 - 11.0 ML & GPU
 - 11.1 ML & GPU
 - 11.2 ML & GPU
+- 11.3 ML & GPU
 
 NOTE: Spark NLP 4.0.x is based on TensorFlow 2.7.x which is compatible with CUDA11 and cuDNN 8.0.2. The only Databricks runtimes supporting CUDA 11 are 9.x and above as listed under GPU.
 

docs/en/transformers.md

@@ -27,7 +27,7 @@ sidebar:
 
 ### Overview
 
-We have extended support for `HuggingFace` 🤗   and `TF Hub` exported models since `3.1.0` to equivalent Spark NLP 🚀 annotators. Starting this release, you can easily use the `saved_model` feature in HuggingFace within a few lines of codes and import any `BERT`, `DistilBERT`, `CamemBERT`, `RoBERTa`, `DeBERTa`, `XLM-RoBERTa`, `Longformer`, `BertForTokenClassification`, `DistilBertForTokenClassification`, `AlbertForTokenClassification`, `RoBertaForTokenClassification`, `DeBertaForTokenClassification`, `XlmRoBertaForTokenClassification`, `XlnetForTokenClassification`,  `LongformerForTokenClassification`, `CamemBertForTokenClassification`, `BertForSequenceClassification`, `DistilBertForSequenceClassification`, `AlbertForSequenceClassification`, `RoBertaForSequenceClassification`, `DeBertaForSequenceClassification`, `XlmRoBertaForSequenceClassification`, `XlnetForSequenceClassification`,  `LongformerForSequenceClassification`, `AlbertForQuestionAnswering`, `BertForQuestionAnswering`,  `DeBertaForQuestionAnswering`, `DistilBertForQuestionAnswering`, `LongformerForQuestionAnswering`, `RoBertaForQuestionAnswering`, `XlmRoBertaForQuestionAnswering`, `TapasForQuestionAnswering`, and `Vision Transformers (ViT)` models to Spark NLP. We will work on the remaining annotators and extend this support to the rest with each release 😊
+We have extended support for `HuggingFace` 🤗   and `TF Hub` exported models since `3.1.0` to equivalent Spark NLP 🚀 annotators. Starting this release, you can easily use the `saved_model` feature in HuggingFace within a few lines of codes and import any `BERT`, `DistilBERT`, `CamemBERT`, `RoBERTa`, `DeBERTa`, `XLM-RoBERTa`, `Longformer`, `BertForTokenClassification`, `DistilBertForTokenClassification`, `AlbertForTokenClassification`, `RoBertaForTokenClassification`, `DeBertaForTokenClassification`, `XlmRoBertaForTokenClassification`, `XlnetForTokenClassification`,  `LongformerForTokenClassification`, `CamemBertForTokenClassification`, `CamemBertForSequenceClassification`, `BertForSequenceClassification`, `DistilBertForSequenceClassification`, `AlbertForSequenceClassification`, `RoBertaForSequenceClassification`, `DeBertaForSequenceClassification`, `XlmRoBertaForSequenceClassification`, `XlnetForSequenceClassification`,  `LongformerForSequenceClassification`, `AlbertForQuestionAnswering`, `BertForQuestionAnswering`,  `DeBertaForQuestionAnswering`, `DistilBertForQuestionAnswering`, `LongformerForQuestionAnswering`, `RoBertaForQuestionAnswering`, `XlmRoBertaForQuestionAnswering`, `TapasForQuestionAnswering`, and `Vision Transformers (ViT)` models to Spark NLP. We will work on the remaining annotators and extend this support to the rest with each release 😊
 
 ### Compatibility
 
@@ -65,6 +65,7 @@ XlmRoBertaForTokenClassification |   | ✅  |  [TFXLMRobertaForTokenClassificati
 XlnetForTokenClassification |   | ✅  |  [TFXLNetForTokenClassificationet](https://huggingface.co/docs/transformers/model_doc/xlnet#transformers.TFXLNetForTokenClassificationet)
 LongformerForTokenClassification |   | ✅  |  [TFLongformerForTokenClassification](https://huggingface.co/docs/transformers/model_doc/longformer#transformers.TFLongformerForTokenClassification)
 CamemBertForTokenClassification |   | ✅  |  [TFCamemBertForTokenClassification](https://huggingface.co/docs/transformers/model_doc/camembert#transformers.TFCamembertForTokenClassification)
+CamemBertForSequenceClassification |   | ✅  |  [TFCamemBertForSequenceClassification](https://huggingface.co/docs/transformers/model_doc/camembert#transformers.TFCamembertForSequenceClassification)
 BertForSequenceClassification |   | ✅  |  [TFBertForSequenceClassification](https://huggingface.co/docs/transformers/model_doc/bert#transformers.TFBertForSequenceClassification)
 DistilBertForSequenceClassification |   | ✅  |  [TFDistilBertForSequenceClassification](https://huggingface.co/docs/transformers/model_doc/distilbert#transformers.TFDistilBertForSequenceClassification)
 AlbertForSequenceClassification |   | ✅  |  [TFAlbertForSequenceClassification](https://huggingface.co/docs/transformers/model_doc/albert#transformers.TFAlbertForSequenceClassification)
@@ -81,11 +82,11 @@ LongformerForSequenceClassification |   | ✅  |  [TFLongformerForSequenceClassi
 | RoBertaForQuestionAnswering  |   |✅  |  [TFRobertaForQuestionAnswering](https://huggingface.co/docs/transformers/model_doc/roberta#transformers.TFRobertaForQuestionAnswering)
 | XlmRoBertaForQuestionAnswering  |   |✅  |  [TFXLMRobertaForQuestionAnswering](https://huggingface.co/docs/transformers/model_doc/xlm-roberta#transformers.TFXLMRobertaForQuestionAnswering)
 | TapasForQuestionAnswering  |   | ❎ |  [TFTapasForQuestionAnswering](https://huggingface.co/docs/transformers/model_doc/tapas#transformers.TFTapasForQuestionAnswering)
-ViTForImageClassification | ❌  | ✅ | [TFViTForImageClassification](https://huggingface.co/docs/transformers/model_doc/vit#transformers.TFViTForImageClassification)
-Automatic Speech Recognition (Wav2Vec2ForCTC)|   | ❎ | [TFWav2Vec2ForCTC](https://huggingface.co/docs/transformers/model_doc/wav2vec2#transformers.TFWav2Vec2ForCTC)
-T5Transformer |   |  ❌ |
-MarianTransformer|   | ❌  |
-OpenAI GPT2|   | ❌  |
+| ViTForImageClassification | ❌  | ✅ | [TFViTForImageClassification](https://huggingface.co/docs/transformers/model_doc/vit#transformers.TFViTForImageClassification)
+| Automatic Speech Recognition (Wav2Vec2ForCTC)|   | ❎ | [TFWav2Vec2ForCTC](https://huggingface.co/docs/transformers/model_doc/wav2vec2#transformers.TFWav2Vec2ForCTC)
+| T5Transformer |   |  ❌ |
+| MarianTransformer|   | ❌  |
+| OpenAI GPT2|   | ❌  |
 
 ### Example Notebooks
 
@@ -109,6 +110,7 @@ AlbertForTokenClassification|[HuggingFace in Spark NLP - AlbertForTokenClassific
 RoBertaForTokenClassification|[HuggingFace in Spark NLP - RoBertaForTokenClassification](https://github.com/JohnSnowLabs/spark-nlp-workshop/blob/master/jupyter/transformers/HuggingFace%20in%20Spark%20NLP%20-%20RoBertaForTokenClassification.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/jupyter/transformers/HuggingFace%20in%20Spark%20NLP%20-%20RoBertaForTokenClassification.ipynb)
 XlmRoBertaForTokenClassification|[HuggingFace in Spark NLP - XlmRoBertaForTokenClassification](https://github.com/JohnSnowLabs/spark-nlp-workshop/blob/master/jupyter/transformers/HuggingFace%20in%20Spark%20NLP%20-%20XlmRoBertaForTokenClassification.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/jupyter/transformers/HuggingFace%20in%20Spark%20NLP%20-%20XlmRoBertaForTokenClassification.ipynb)
 CamemBertForTokenClassification|[HuggingFace in Spark NLP - CamemBertForTokenClassification](https://github.com/JohnSnowLabs/spark-nlp-workshop/blob/master/jupyter/transformers/HuggingFace%20in%20Spark%20NLP%20-%20CamemBertForTokenClassification.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/jupyter/transformers/HuggingFace%20in%20Spark%20NLP%20-%20CamemBertForTokenClassification.ipynb)
+CamemBertForTokenClassification|[HuggingFace in Spark NLP - CamemBertForSequenceClassification](https://github.com/JohnSnowLabs/spark-nlp-workshop/blob/master/jupyter/transformers/HuggingFace%20in%20Spark%20NLP%20-%20CamemBertForSequenceClassification.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/jupyter/transformers/HuggingFace%20in%20Spark%20NLP%20-%20CamemBertForSequenceClassification.ipynb)
 BertForSequenceClassification |[HuggingFace in Spark NLP - BertForSequenceClassification](https://github.com/JohnSnowLabs/spark-nlp-workshop/blob/master/jupyter/transformers/HuggingFace%20in%20Spark%20NLP%20-%20BertForSequenceClassification.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/jupyter/transformers/HuggingFace%20in%20Spark%20NLP%20-%20BertForSequenceClassification.ipynb)
 DistilBertForSequenceClassification |[HuggingFace in Spark NLP - DistilBertForSequenceClassification](https://github.com/JohnSnowLabs/spark-nlp-workshop/blob/master/jupyter/transformers/HuggingFace%20in%20Spark%20NLP%20-%20DistilBertForSequenceClassification.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/jupyter/transformers/HuggingFace%20in%20Spark%20NLP%20-%20DistilBertForSequenceClassification.ipynb)
 AlbertForSequenceClassification |[HuggingFace in Spark NLP - AlbertForSequenceClassification](https://github.com/JohnSnowLabs/spark-nlp-workshop/blob/master/jupyter/transformers/HuggingFace%20in%20Spark%20NLP%20-%20BertForSequenceClassification.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/jupyter/transformers/HuggingFace%20in%20Spark%20NLP%20-%20AlbertForSequenceClassification.ipynb)

python/README.md

@@ -271,6 +271,8 @@ Spark NLP 4.2.5 has been tested and is compatible with the following runtimes:
 - 11.1 ML
 - 11.2
 - 11.2 ML
+- 11.3
+- 11.3 ML
 
 **GPU:**
 
@@ -283,6 +285,7 @@ Spark NLP 4.2.5 has been tested and is compatible with the following runtimes:
 - 11.0 ML & GPU
 - 11.1 ML & GPU
 - 11.2 ML & GPU
+- 11.3 ML & GPU
 
 NOTE: Spark NLP 4.0.x is based on TensorFlow 2.7.x which is compatible with CUDA11 and cuDNN 8.0.2. The only Databricks runtimes supporting CUDA 11 are 9.x and above as listed under GPU.