In this work, we plan to combine SMART method with BERT model.
- SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization: https://www.aclweb.org/anthology/2020.acl-main.197.pdf
- STEIDL, G.:Lecture Notes on Convex Analysis. 2020
- The information geometry of mirror descent: https://arxiv.org/pdf/1310.7780.pdf
- Computing proximal points of nonconvex functions: http://www.optimization-online.org/DB_FILE/2005/06/1163.pdf
- Regularization using a parameterized trust region subproblem: https://link.springer.com/article/10.1007/s10107-007-0126-4
- iPiano: Inertial Proximal Algorithm for Nonconvex Optimization: https://arxiv.org/pdf/1404.4805.pdf