SupportBERT

In this work, we plan to combine SMART method with BERT model.

Major Resources used (so far):

SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization: https://www.aclweb.org/anthology/2020.acl-main.197.pdf
STEIDL, G.:Lecture Notes on Convex Analysis. 2020

The information geometry of mirror descent: https://arxiv.org/pdf/1310.7780.pdf
Computing proximal points of nonconvex functions: http://www.optimization-online.org/DB_FILE/2005/06/1163.pdf
Regularization using a parameterized trust region subproblem: https://link.springer.com/article/10.1007/s10107-007-0126-4
iPiano: Inertial Proximal Algorithm for Nonconvex Optimization: https://arxiv.org/pdf/1404.4805.pdf

Name		Name	Last commit message	Last commit date
bert_race		bert_race
data_utils		data_utils
race_utils		race_utils
utils		utils
LICENSE		LICENSE
README.md		README.md
batcher.py		batcher.py
inference.py		inference.py
loss.py		loss.py
matcher.py		matcher.py
model.py		model.py
perturbation.py		perturbation.py
pretrained_models.py		pretrained_models.py
supportBert.ipynb		supportBert.ipynb
tomaster-BERT INFERENCE.ipynb		tomaster-BERT INFERENCE.ipynb
tomaster-BERT-RACE.ipynb		tomaster-BERT-RACE.ipynb
tomaster-BERT.ipynb		tomaster-BERT.ipynb
tosmaster_mtdnn.py		tosmaster_mtdnn.py