Backpropagation_Notes.md

Backpropagation

[TOC]

Overview

This page is a collection of various lecture notes and web-pages which explain Back-Propagation.

What is Backpropagation

Back-propagation is a commonly used method for training Neural Networks.

Online classes which explain Backpropagation.

1. CS-231N: Convolutional Neural Networks for Visual Recognition
   1. Stanford CS-231N (Spring 2017) Syllabus
   2. I posted the link to the Spring 2017 version since it has YouTube videos.
   3. Backpropagation is covered in Lecture 4.
2. CS-224N: Natural Language Processing with Deep Learning
   1. Stanford CS-224N (Winter 2019 Link)
   2. I posted the link to the Winter 2019 version since it has YouTube videos.
   3. Backpropagation is covered in Lecture 4.
3. CS-131: Computer Vision: Foundations and Applications:
   1. Stanford CS-131 Fall 2019
   2. More general course; Backpropagation is covered in Lectures 19 and 20

Blog Posts which explain Backpropagation

1. CS-231N Post on Backpropagation
2. Chapter 2 in Michael Nielsen book
3. Chris Colah Backpropagation Blog

Detailed Matrix Calculus sessions at TWIMAL:

1. Review of "The Matrix Calculus you need for Deep Learning" lead by Joseph Catanzarite
   1. Set of three review lectures; Great mathematical detail
   2. Youtube Playlist

Short Youtube video series on Neural Network and Backpropagation

1. 3Blue1Brown Neural Network playlist
   1. This is amazing!
2. Essence of linear Algebra playlist
3. Multivariable functions and Multivariable Calculus

Cheat Sheets

1. Deep Learning cheat sheet by Shervine for Stanford CS-229
   1. This is amazing!