Models proposed to test

[djstrong]

It would be helpful to see those models from HF on the leaderboard:

• xlm-roberta-base (base of HerBERT)
• xlm-roberta-large (base of HerBERT)
• facebook/xlm-roberta-xl - needs more VRAM
• facebook/xlm-roberta-xxl - needs more VRAM
• sdadas/polish-distilroberta - something smaller
• sdadas/polish-roberta-base-v2 - maybe better than HerBERT
• sdadas/polish-roberta-large-v2 - maybe better than HerBERT

Other models:

• Flair
• word2vec
• GloVe
• FastText
• some non-vector based models from scikit-learn, e.g NB, SVM, XGB, logistic regression

[laugustyniak]

Hi @djstrong, thanks for your recommendation.

Right now, we are gathering baseline models for testing, we already added to our todo list static embeddings with SVM/XGB.

Regarding LMs, we also plan to add them but rather in 3rd batch of our experiment running.