Hi there! 👋 I am a first-year Ph.D. student of UCLA following Prof. Quanquan Gu, mainly researching in optimization and architecture of Large Language Models. Previously I was a Yao Class Student in Tsinghua University advised by Prof. Zhilin Yang.
See my:
- Main Page: https://lauyikfung.github.io
- Blog: https://lauyikfung.github.io/blog
- MARS-M: When Variance Reduction Meets Matrices: Paper, github
- Robust Layerwise Scaling Rules by Proper Weight Decay Tuning: Paper
- RPG: On the Design of KL-Regularized Policy Gradient Algorithms for LLM Reasoning: Paper, github
- Kimi K1.5: Scaling Reinforcement Learning with LLMs: Paper
- TPA: Tensor Product Attention Is All You Need: Paper, github
- MARS: Unleashing the Power of Variance Reduction for Training Large Models: Paper, github
- T-Rex: Text-assisted Retrosynthesis Prediction: Paper, github
- Capricorn: Enhancing Hi-C contact matrices for loop detection with Capricorn, a multi-view diffusion model: Paper, github
Besides research, I love traveling around and I am also fond of topics including transportation (subways/undergrounds/light rails, etc.), geography (especially Chinese geography), linguistics, Vocaloid (Hatsune Miku & IA in Japanese, Luo Tianyi in Chinese) as well as financing (VC investment etc., welcome to talk with me about AI startups and investment in AI).