This repo contains references to various resources covering topics in machine learning. By no means this is an exhaustive list. In order to keep it up-to-date I'll try to review it once in while.
https://arxiv.org/pdf/1502.05767.pdf
- Intro Machine Learning
- Deep Learning
- Generative Model
- Reinforcement Learning
- Probabilistic Graphical Model
- Optimization
- Statistics
- Topics in Machine Learning
- Math Background
- Blogs and Podcasts
There are many ML resources, which position themselves as introductory
Below is the list of textbooks and courses I came across.
- Machine Learning: A Probabilistic Perspective (Murphy)
- Pattern Recognition and Machine Learning (Bishop)
- The Elements of Statistical Learning (Hastie et al.)
- Pattern Classification (Duda et al.)
- Bayesian Reasoning and Machine Learning (Barber)
- Learning from Data (Abu-Mostafa) ✨
- Foundations of Machine Learning (Mohri et al.)
- An Introduction to Computational Learning Theory (Kearns & Vazirani)
- Understanding Machine Learning: An introductory course in Machine Learning by Shai Shalev-Shwartz. The course is in Hebrew and is based on the book with the same title.
- Caltech Learning from Data ✨
- Stanford CS229 Machine Learning ✨
- CMU 10701 Mahine Learning
- CMU 10702 Statistical Machine Learning
- Oxford Machine Learning
- CMU 15859 Machine Learning Theory
- Berkeley CS281b/Stat241b Statistical Learning Theory
- MIT 9.520 Statistical Learning Theory and Applications
- Statistical Learning Theory
Goodfellow et al. is the new classic.
For vision and NLP, Stanford lectures would be helpful.
- Deep Learning (Goodfellow et al.) ✨
- Stanford CS231n Convolutional Neural Networks for Visual Recognition ✨
- Stanfrod CS224d Deep Learning for Natural Language Processing
- TensorFlow for Deep Learning Research
- Deep Learning book ✨
- Stanfrod CS224s Spoken Language Processing
- Oxford Deep Natural Language Processing
- CMU 11747 Neural Networks for NLP
- Stanford Stat385 Theories of Deep Learning
I seperated generative model as an independent topic,
since I think it is big and important area.
- Toronto CSC2541 Differentiable Inference and Generative Models
- Toronto CSC2547 Learning Discrete Latent Structure
- Toronto CSC2541 Scalable and Flexible Models of Uncertainty
For classic (non-deep) RL, Sutton & Barto is the classic.
For deep RL, lectures from Berkeley/CMU looks good.
- Reinforcement Learning: An Introduction (Sutton & Barto) ✨
- UCL Reinforcement Learning
- Berkeley CS294 Deep Reinforcement Leanring
- CMU 10703 Deep Reinforcement Learing and Control
- Probabilistic Graphical Models: Principles and Techniques (Koller & Friedman)
- Graphical Models, Exponential Families, and Variational Inference (Wainwright & Jordan) ✨
Boyd & Vandenberghe is the classic, but I think it's too boring.
Reading chapter 2-5 would be enough.
Bertsekas more concentrates on convex analysis.
Nocedal & Wright more concentrates on optimization.
- Convex Optimization (Boyd & Vandenberghe) ✨
- Convex Optimization Theory (Bertsekas)
- Numerical Optimization (Nocedal & Wright)
- Stanford EE364a Convex Optimization I ✨
- Stanford EE364b Convex Optimization II
- MIT 6.253 Convex Analysis and Optimization
For online learning, Cesa-Bianchi & Lugosi is the classic.
For multi-armed bandit, Bubeck & Cesa-Bianchi provides a good survey.
- Prediction, Learning, and Games (Cesa-Bianchi & Lugosi)
- Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems (Bubeck & Cesa-Bianchi)
Miscellaneous topics directrly related to machine learning.
- Probability: Theory and Examples (Durrett)
- Theoretical Statistics (Keener)
- Stochastic Processes (Bass)
- Probability and Statistics Cookbook (Vallentin)
- All of Statistics (Wasserman)
- Computer Age Statistical Inference (Efron & Hastie) ✨
- Time Series Analysis and Its Applications: With R Examples (Shumway & Stoffer)
- All of Nonparametric Statistics (Wasserman)
- Introduction to Nonparametric Estimation (Tsybakov)
- Gaussian Process and Machine Learning (Rasmussen & Williams) ✨
- Bayesian Nonparametrics (Ghosh & Ramamoorthi) ✨
- High-Dimensional Statistics: A Non-Asymptotic Viewpoint (Wainwright) ✨
- Statistics for High-Dimensional Data (Bühlmann & van de Geer)
- Asymptotic Statistics (van der Vaart)
- Empirical Processes in M-Estimation (van der Vaart)
- Berkeley Stat281b Statistical Learning Theory
- Berkeley Stat210a Theoretical Statistics I
- Berkeley Stat210b Theoretical Statistics II
- Berkeley Stat210b Theoretical Statistics II
- Stanford Stat300a Theory of Statistics
- Stanford CS369m Algorithms for Massive Data Set Analysis
- CMU 36755 Advanced Statistical Theory I
- MIT 18.S997 High-Dimensional Statistics
- UCLA Theoretical Statistics
- Linear Algebra (Hoffman & Kunze)
- Matrix Analysis (Horn & Johnson)
- Matrix Computations (Golub & Van Loan)
- The Matrix Cookbook (Petersen & Pedersen)
- Concentration Inequalities and Martingale Inequalities (Chung & Lu)
- An Introduction to Matrix Concentration Inequalities (Tropp)
- Elements of Information Theory (Cover & Thomas)
- Information Theory, Inference, and Learning Algorithms (MacKay)
- Networks, Crowds, and Markets (Easley & Kleinberg)
- Social and Economic Networks (Jackson)
- Markov Chains (Norris)
- Markov Chains and Mixing Times (Levin et al.)
- Algorithmic Game Theory (Nisan et al.)
- Multiagent Systems (Shoham & Leyton-Brown)
- The Probabilistic Method (Alon & Spencer)
- A First Course in Combinatorial Optimization (Lee)
- Introduction to Algorithms (Cormen et al.)
- Randomized Algorithms (Motwani & Raghavan)
- Approximation Algorithms (Vazirani)
- Topological Data Analysis (Wasserman)
- Methods of Information Geometry (Amari & Nagaoka)
- Algebraic Geometry and Statistical Learning Theory (Watanabe)
- DeepMind Blog
- OpenAI Blog
- FAIR Blog
- BAIR Blog
- Columbia ML Seminar
- Mathematical Coffees
- Lunit Tech Blog (Korean)