Skip to content

Commit 6887947

Browse files
sguggerjulien-c
authored and
LysandreJik
committed
Big model table (#8774)
* First draft * Styling * With all changes staged * Update docs/source/index.rst Co-authored-by: Julien Chaumond <[email protected]> * Styling Co-authored-by: Julien Chaumond <[email protected]>
1 parent 8c5a2b8 commit 6887947

File tree

8 files changed

+257
-19
lines changed

8 files changed

+257
-19
lines changed

‎docs/source/_static/css/huggingface.css

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,15 @@
22

33
/* Colab dropdown */
44

5+
table.center-aligned-table td {
6+
text-align: center;
7+
}
8+
9+
table.center-aligned-table th {
10+
text-align: center;
11+
vertical-align: middle;
12+
}
13+
514
.colab-dropdown {
615
position: relative;
716
display: inline-block;

‎docs/source/index.rst

Lines changed: 93 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,8 @@ Choose the right framework for every part of a model's lifetime:
3535
- Move a single model between TF2.0/PyTorch frameworks at will
3636
- Seamlessly pick the right framework for training, evaluation, production
3737

38+
Experimental support for Flax with a few models right now, expected to grow in the coming months.
39+
3840
Contents
3941
-----------------------------------------------------------------------------------------------------------------------
4042

@@ -52,8 +54,8 @@ The documentation is organized in five parts:
5254
- **MODELS** for the classes and functions related to each model implemented in the library.
5355
- **INTERNAL HELPERS** for the classes and functions we use internally.
5456

55-
The library currently contains PyTorch and Tensorflow implementations, pre-trained model weights, usage scripts and
56-
conversion utilities for the following models:
57+
The library currently contains PyTorch, Tensorflow and Flax implementations, pretrained model weights, usage scripts
58+
and conversion utilities for the following models:
5759

5860
..
5961
This list is updated automatically from the README with `make fix-copies`. Do not update manually!
@@ -166,6 +168,95 @@ conversion utilities for the following models:
166168
34. `Other community models <https://huggingface.co/models>`__, contributed by the `community
167169
<https://huggingface.co/users>`__.
168170

171+
172+
The table below represents the current support in the library for each of those models, whether they have a Python
173+
tokenizer (called "slow"). A "fast" tokenizer backed by the 🤗 Tokenizers library, whether they have support in PyTorch,
174+
TensorFlow and/or Flax.
175+
176+
..
177+
This table is updated automatically from the auto modules with `make fix-copies`. Do not update manually!
178+
179+
.. rst-class:: center-aligned-table
180+
181+
+-----------------------------+----------------+----------------+-----------------+--------------------+--------------+
182+
| Model | Tokenizer slow | Tokenizer fast | PyTorch support | TensorFlow support | Flax Support |
183+
+=============================+================+================+=================+====================+==============+
184+
| ALBERT | ✅ | ✅ | ✅ | ✅ | ❌ |
185+
+-----------------------------+----------------+----------------+-----------------+--------------------+--------------+
186+
| BART | ✅ | ✅ | ✅ | ✅ | ❌ |
187+
+-----------------------------+----------------+----------------+-----------------+--------------------+--------------+
188+
| BERT | ✅ | ✅ | ✅ | ✅ | ✅ |
189+
+-----------------------------+----------------+----------------+-----------------+--------------------+--------------+
190+
| Bert Generation | ✅ | ❌ | ✅ | ❌ | ❌ |
191+
+-----------------------------+----------------+----------------+-----------------+--------------------+--------------+
192+
| Blenderbot | ✅ | ❌ | ✅ | ✅ | ❌ |
193+
+-----------------------------+----------------+----------------+-----------------+--------------------+--------------+
194+
| CTRL | ✅ | ❌ | ✅ | ✅ | ❌ |
195+
+-----------------------------+----------------+----------------+-----------------+--------------------+--------------+
196+
| CamemBERT | ✅ | ✅ | ✅ | ✅ | ❌ |
197+
+-----------------------------+----------------+----------------+-----------------+--------------------+--------------+
198+
| DPR | ✅ | ✅ | ✅ | ✅ | ❌ |
199+
+-----------------------------+----------------+----------------+-----------------+--------------------+--------------+
200+
| DeBERTa | ✅ | ❌ | ✅ | ❌ | ❌ |
201+
+-----------------------------+----------------+----------------+-----------------+--------------------+--------------+
202+
| DistilBERT | ✅ | ✅ | ✅ | ✅ | ❌ |
203+
+-----------------------------+----------------+----------------+-----------------+--------------------+--------------+
204+
| ELECTRA | ✅ | ✅ | ✅ | ✅ | ❌ |
205+
+-----------------------------+----------------+----------------+-----------------+--------------------+--------------+
206+
| Encoder decoder | ❌ | ❌ | ✅ | ❌ | ❌ |
207+
+-----------------------------+----------------+----------------+-----------------+--------------------+--------------+
208+
| FairSeq Machine-Translation | ✅ | ❌ | ✅ | ❌ | ❌ |
209+
+-----------------------------+----------------+----------------+-----------------+--------------------+--------------+
210+
| FlauBERT | ✅ | ❌ | ✅ | ✅ | ❌ |
211+
+-----------------------------+----------------+----------------+-----------------+--------------------+--------------+
212+
| Funnel Transformer | ✅ | ✅ | ✅ | ✅ | ❌ |
213+
+-----------------------------+----------------+----------------+-----------------+--------------------+--------------+
214+
| LXMERT | ✅ | ✅ | ✅ | ✅ | ❌ |
215+
+-----------------------------+----------------+----------------+-----------------+--------------------+--------------+
216+
| LayoutLM | ✅ | ✅ | ✅ | ❌ | ❌ |
217+
+-----------------------------+----------------+----------------+-----------------+--------------------+--------------+
218+
| Longformer | ✅ | ✅ | ✅ | ✅ | ❌ |
219+
+-----------------------------+----------------+----------------+-----------------+--------------------+--------------+
220+
| Marian | ✅ | ❌ | ✅ | ✅ | ❌ |
221+
+-----------------------------+----------------+----------------+-----------------+--------------------+--------------+
222+
| MobileBERT | ✅ | ✅ | ✅ | ✅ | ❌ |
223+
+-----------------------------+----------------+----------------+-----------------+--------------------+--------------+
224+
| OpenAI GPT | ✅ | ✅ | ✅ | ✅ | ❌ |
225+
+-----------------------------+----------------+----------------+-----------------+--------------------+--------------+
226+
| OpenAI GPT-2 | ✅ | ✅ | ✅ | ✅ | ❌ |
227+
+-----------------------------+----------------+----------------+-----------------+--------------------+--------------+
228+
| Pegasus | ✅ | ✅ | ✅ | ✅ | ❌ |
229+
+-----------------------------+----------------+----------------+-----------------+--------------------+--------------+
230+
| ProphetNet | ✅ | ❌ | ✅ | ❌ | ❌ |
231+
+-----------------------------+----------------+----------------+-----------------+--------------------+--------------+
232+
| RAG | ✅ | ❌ | ✅ | ❌ | ❌ |
233+
+-----------------------------+----------------+----------------+-----------------+--------------------+--------------+
234+
| Reformer | ✅ | ✅ | ✅ | ❌ | ❌ |
235+
+-----------------------------+----------------+----------------+-----------------+--------------------+--------------+
236+
| RetriBERT | ✅ | ✅ | ✅ | ❌ | ❌ |
237+
+-----------------------------+----------------+----------------+-----------------+--------------------+--------------+
238+
| RoBERTa | ✅ | ✅ | ✅ | ✅ | ✅ |
239+
+-----------------------------+----------------+----------------+-----------------+--------------------+--------------+
240+
| SqueezeBERT | ✅ | ✅ | ✅ | ❌ | ❌ |
241+
+-----------------------------+----------------+----------------+-----------------+--------------------+--------------+
242+
| T5 | ✅ | ✅ | ✅ | ✅ | ❌ |
243+
+-----------------------------+----------------+----------------+-----------------+--------------------+--------------+
244+
| Transformer-XL | ✅ | ❌ | ✅ | ✅ | ❌ |
245+
+-----------------------------+----------------+----------------+-----------------+--------------------+--------------+
246+
| XLM | ✅ | ❌ | ✅ | ✅ | ❌ |
247+
+-----------------------------+----------------+----------------+-----------------+--------------------+--------------+
248+
| XLM-RoBERTa | ✅ | ✅ | ✅ | ✅ | ❌ |
249+
+-----------------------------+----------------+----------------+-----------------+--------------------+--------------+
250+
| XLMProphetNet | ✅ | ❌ | ✅ | ❌ | ❌ |
251+
+-----------------------------+----------------+----------------+-----------------+--------------------+--------------+
252+
| XLNet | ✅ | ✅ | ✅ | ✅ | ❌ |
253+
+-----------------------------+----------------+----------------+-----------------+--------------------+--------------+
254+
| mBART | ✅ | ✅ | ✅ | ✅ | ❌ |
255+
+-----------------------------+----------------+----------------+-----------------+--------------------+--------------+
256+
| mT5 | ✅ | ✅ | ✅ | ✅ | ❌ |
257+
+-----------------------------+----------------+----------------+-----------------+--------------------+--------------+
258+
259+
169260
.. toctree::
170261
:maxdepth: 2
171262
:caption: Get started

‎src/transformers/__init__.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -98,6 +98,7 @@
9898
from .models.auto import (
9999
ALL_PRETRAINED_CONFIG_ARCHIVE_MAP,
100100
CONFIG_MAPPING,
101+
MODEL_NAMES_MAPPING,
101102
TOKENIZER_MAPPING,
102103
AutoConfig,
103104
AutoTokenizer,
@@ -876,6 +877,7 @@
876877

877878

878879
if is_flax_available():
880+
from .models.auto import FLAX_MODEL_MAPPING, FlaxAutoModel
879881
from .models.bert import FlaxBertModel
880882
from .models.roberta import FlaxRobertaModel
881883
else:

‎src/transformers/models/auto/__init__.py

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,8 @@
22
# There's no way to ignore "F401 '...' imported but unused" warnings in this
33
# module, but to preserve other warnings. So, don't check this module at all.
44

5-
from ...file_utils import is_tf_available, is_torch_available
6-
from .configuration_auto import ALL_PRETRAINED_CONFIG_ARCHIVE_MAP, CONFIG_MAPPING, AutoConfig
5+
from ...file_utils import is_flax_available, is_tf_available, is_torch_available
6+
from .configuration_auto import ALL_PRETRAINED_CONFIG_ARCHIVE_MAP, CONFIG_MAPPING, MODEL_NAMES_MAPPING, AutoConfig
77
from .tokenization_auto import TOKENIZER_MAPPING, AutoTokenizer
88

99

@@ -57,3 +57,6 @@
5757
TFAutoModelForTokenClassification,
5858
TFAutoModelWithLMHead,
5959
)
60+
61+
if is_flax_available():
62+
from .modeling_flax_auto import FLAX_MODEL_MAPPING, FlaxAutoModel

‎src/transformers/models/auto/modeling_flax_auto.py

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@
3636
for key, value, in pretrained_map.items()
3737
)
3838

39-
MODEL_MAPPING = OrderedDict(
39+
FLAX_MODEL_MAPPING = OrderedDict(
4040
[
4141
(RobertaConfig, FlaxRobertaModel),
4242
(BertConfig, FlaxBertModel),
@@ -79,13 +79,13 @@ def from_config(cls, config):
7979
model = FlaxAutoModel.from_config(config)
8080
# E.g. model was saved using `save_pretrained('./test/saved_model/')`
8181
"""
82-
for config_class, model_class in MODEL_MAPPING.items():
82+
for config_class, model_class in FLAX_MODEL_MAPPING.items():
8383
if isinstance(config, config_class):
8484
return model_class(config)
8585
raise ValueError(
8686
f"Unrecognized configuration class {config.__class__} "
8787
f"for this kind of FlaxAutoModel: {cls.__name__}.\n"
88-
f"Model type should be one of {', '.join(c.__name__ for c in MODEL_MAPPING.keys())}."
88+
f"Model type should be one of {', '.join(c.__name__ for c in FLAX_MODEL_MAPPING.keys())}."
8989
)
9090

9191
@classmethod
@@ -173,11 +173,11 @@ def from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs):
173173
if not isinstance(config, PretrainedConfig):
174174
config = AutoConfig.from_pretrained(pretrained_model_name_or_path, **kwargs)
175175

176-
for config_class, model_class in MODEL_MAPPING.items():
176+
for config_class, model_class in FLAX_MODEL_MAPPING.items():
177177
if isinstance(config, config_class):
178178
return model_class.from_pretrained(pretrained_model_name_or_path, *model_args, config=config, **kwargs)
179179
raise ValueError(
180180
f"Unrecognized configuration class {config.__class__} "
181181
f"for this kind of FlaxAutoModel: {cls.__name__}.\n"
182-
f"Model type should be one of {', '.join(c.__name__ for c in MODEL_MAPPING.keys())}"
182+
f"Model type should be one of {', '.join(c.__name__ for c in FLAX_MODEL_MAPPING.keys())}"
183183
)

‎src/transformers/utils/dummy_flax_objects.py

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,18 @@
22
from ..file_utils import requires_flax
33

44

5+
FLAX_MODEL_MAPPING = None
6+
7+
8+
class FlaxAutoModel:
9+
def __init__(self, *args, **kwargs):
10+
requires_flax(self)
11+
12+
@classmethod
13+
def from_pretrained(self, *args, **kwargs):
14+
requires_flax(self)
15+
16+
517
class FlaxBertModel:
618
def __init__(self, *args, **kwargs):
719
requires_flax(self)

0 commit comments

Comments
 (0)