MachineLearning-05.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  Coursera: [Machine Learning](https://www.coursera.org/learn/machine-learning) - Week 5\n",
    "\n",
    "Created by Andrew Ng / Stanford University"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Table of Content\n",
    "\n",
    "* [Neural Networks: Learning](#Neural-Networks:-Learning)\n",
    "   * Cost Function and Backpropagation\n",
    "      * Cost Function\n",
    "      * Backpropagation Algorithm\n",
    "      * Backpropagation Intuition\n",
    "   * Backpropagation in Practice\n",
    "      * Implementation Note: Unrolling Parameters\n",
    "      * Gradient Checking\n",
    "      * Random Initialization\n",
    "      * Putting it Together\n",
    "   * Application of Neural Networks\n",
    "      * Autonomous Driving\n",
    "   * Review"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Neural Networks: Learning\n",
    "\n",
    "In Week 5, you will be learning how to train Neural Networks. The Neural Network is one of the most powerful learning algorithms (when a linear classifier doesn't work, this is what I usually turn to), and this week's videos explain the 'backpropagation' algorithm for training these models. In this week's programming assignment, you'll also get to implement this algorithm and see it work for yourself.\n",
    "\n",
    "The Neural Network programming exercise will be one of the more challenging ones of this class. So please start early and do leave extra time to get it done, and I hope you'll stick with it until you get it to work!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Cost Function and Backpropagation\n",
    "\n",
    "#### Cost Function\n",
    "\n",
    "Let's first define a few variables that we will need to use:\n",
    "\n",
    "* $L$ = total number of layers in the network\n",
    "* $s_l$ = number of units (not counting bias unit) in layer l\n",
    "* $K$ = number of output units/classes\n",
    "\n",
    "Recall that in neural networks, we may have many output nodes. We denote $h_\\Theta(x)_k$ as being a hypothesis that results in the $k^{th}$ output. Our cost function for neural networks is going to be a generalization of the one we used for logistic regression. Recall that the cost function for regularized logistic regression was:\n",
    "\n",
    "$J(\\theta) = - \\frac{1}{m} \\sum_{i=1}^m [ y^{(i)}\\ \\log (h_\\theta (x^{(i)})) + (1 - y^{(i)})\\ \\log (1 - h_\\theta(x^{(i)}))] + \\frac{\\lambda}{2m}\\sum_{j=1}^n \\theta_j^2$\n",
    "\n",
    "For neural networks, it is going to be slightly more complicated:\n",
    "\n",
    "$\\begin{gather*} J(\\Theta) = - \\frac{1}{m} \\sum_{i=1}^m \\sum_{k=1}^K \\left[y^{(i)}_k \\log ((h_\\Theta (x^{(i)}))_k) + (1 - y^{(i)}_k)\\log (1 - (h_\\Theta(x^{(i)}))_k)\\right] + \\frac{\\lambda}{2m}\\sum_{l=1}^{L-1} \\sum_{i=1}^{s_l} \\sum_{j=1}^{s_{l+1}} ( \\Theta_{j,i}^{(l)})^2\\end{gather*}$\n",
    "\n",
    "We have added a few nested summations to account for our multiple output nodes. In the first part of the equation, before the square brackets, we have an additional nested summation that loops through the number of output nodes.\n",
    "\n",
    "In the regularization part, after the square brackets, we must account for multiple theta matrices. The number of columns in our current theta matrix is equal to the number of nodes in our current layer (including the bias unit). The number of rows in our current theta matrix is equal to the number of nodes in the next layer (excluding the bias unit). As before with logistic regression, we square every term.\n",
    "\n",
    "Note:\n",
    "\n",
    "* the double sum simply adds up the logistic regression costs calculated for each cell in the output layer\n",
    "* the triple sum simply adds up the squares of all the individual $Θ$s in the entire network.\n",
    "* the $i$ in the triple sum does **not** refer to training example $i$\n",
    "\n",
    "#### Backpropagation Algorithm\n",
    "\n",
    "\"Backpropagation\" is neural-network terminology for minimizing our cost function, just like what we were doing with gradient descent in logistic and linear regression. Our goal is to compute:\n",
    "\n",
    "$\\min_\\Theta J(\\Theta)$\n",
    "\n",
    "That is, we want to minimize our cost function $J$ using an optimal set of parameters in theta. In this section we'll look at the equations we use to compute the partial derivative of $J(Θ)$:\n",
    "\n",
    "$\\dfrac{\\partial}{\\partial \\Theta_{i,j}^{(l)}}J(\\Theta)$\n",
    "\n",
    "To do so, we use the following algorithm:\n",
    "\n",
    "![title](images/Ul6i5teoEea1UArqXEX_3g_a36fb24a11c744d7552f0fecf2fdd752_Screenshot-2017-01-10-17.13.27.png)\n",
    "\n",
    "##### Back propagation Algorithm\n",
    "\n",
    "Given training set $\\lbrace (x^{(1)}, y^{(1)}) \\cdots (x^{(m)}, y^{(m)})\\rbrace$\n",
    "\n",
    "* Set $\\Delta^{(l)}_{i,j} := 0$ for all $(l,i,j)$, (hence you end up having a matrix full of zeros)\n",
    "\n",
    "For training example $t = 1$ to $m$:\n",
    "\n",
    "* Set $a^{(1)} := x^{(t)}$\n",
    "* Perform forward propagation to compute $a^{(l)}$ for $l=2,3,…,L$\n",
    "\n",
    "![title](images/bYLgwteoEeaX9Qr89uJd1A_73f280ff78695f84ae512f19acfa29a3_Screenshot-2017-01-10-18.16.50.png)\n",
    "\n",
    "* Using $y^{(t)}$, compute $\\delta^{(L)} = a^{(L)} - y^{(t)}$\n",
    "\n",
    "Where $L$ is our total number of layers and $a^{(L)}$ is the vector of outputs of the activation units for the last layer. So our \"error values\" for the last layer are simply the differences of our actual results in the last layer and the correct outputs in y. To get the delta values of the layers before the last layer, we can use an equation that steps us back from right to left:\n",
    "\n",
    "* Compute $\\delta^{(L-1)}, \\delta^{(L-2)},\\dots,\\delta^{(2)}$ using $\\delta^{(l)} = ((\\Theta^{(l)})^T \\delta^{(l+1)})\\ .*\\ a^{(l)}\\ .*\\ (1 - a^{(l)})$\n",
    "\n",
    "The delta values of layer $l$ are calculated by multiplying the delta values in the next layer with the theta matrix of layer $l$. We then element-wise multiply that with a function called $g'$, or g-prime, which is the derivative of the activation function $g$ evaluated with the input values given by $z^{(l)}$.\n",
    "\n",
    "The g-prime derivative terms can also be written out as:\n",
    "\n",
    "$g'(z^{(l)}) = a^{(l)}\\ .*\\ (1 - a^{(l)})$\n",
    "\n",
    "* $\\Delta^{(l)}_{i,j} := \\Delta^{(l)}_{i,j} + a_j^{(l)} \\delta_i^{(l+1)}$ or with vectorization, $\\Delta^{(l)} := \\Delta^{(l)} + \\delta^{(l+1)}(a^{(l)})^T$\n",
    "\n",
    "Hence we update our new $\\Delta$ matrix.\n",
    "\n",
    "* $D^{(l)}_{i,j} := \\dfrac{1}{m}\\left(\\Delta^{(l)}_{i,j} + \\lambda\\Theta^{(l)}_{i,j}\\right)$, if $j≠0$.\n",
    "* $D^{(l)}_{i,j} := \\dfrac{1}{m}\\Delta^{(l)}_{i,j}$ if $j=0$\n",
    "\n",
    "The capital-delta matrix $D$ is used as an \"accumulator\" to add up our values as we go along and eventually compute our partial derivative. Thus we get $\\frac \\partial {\\partial \\Theta_{ij}^{(l)}} J(\\Theta)= D_{ij}^{(l)}$\n",
    "\n",
    "#### Backpropagation Intuition\n",
    "\n",
    "**Note:** [4:39, the last term for the calculation for $z^3_1$ (three-color handwritten formula) should be $a^2_2$ instead of $a^2_1$. 6:08 - the equation for $cost(i)$ is incorrect. The first term is missing parentheses for the $log()$ function, and the second term should be $(1-y^{(i)})\\log(1-h{_\\theta}{(x^{(i)}}))$. 8:50 - $\\delta^{(4)} = y - a^{(4)}$ is incorrect and should be $\\delta^{(4)} = a^{(4)} - y$.]\n",
    "\n",
    "Recall that the cost function for a neural network is:\n",
    "\n",
    "$\\begin{gather*}J(\\Theta) = - \\frac{1}{m} \\sum_{t=1}^m\\sum_{k=1}^K \\left[ y^{(t)}_k \\ \\log (h_\\Theta (x^{(t)}))_k + (1 - y^{(t)}_k)\\ \\log (1 - h_\\Theta(x^{(t)})_k)\\right] + \\frac{\\lambda}{2m}\\sum_{l=1}^{L-1} \\sum_{i=1}^{s_l} \\sum_{j=1}^{s_l+1} ( \\Theta_{j,i}^{(l)})^2\\end{gather*}$\n",
    "\n",
    "If we consider simple non-multiclass classification ($k = 1$) and disregard regularization, the cost is computed with:\n",
    "\n",
    "$cost(t) =y^{(t)} \\ \\log (h_\\Theta (x^{(t)})) + (1 - y^{(t)})\\ \\log (1 - h_\\Theta(x^{(t)}))$\n",
    "\n",
    "Intuitively, $\\delta_j^{(l)}$ is the \"error\" for $a^{(l)}_j$ (unit $j$ in layer $l$). More formally, the delta values are actually the derivative of the cost function:\n",
    "\n",
    "$\\delta_j^{(l)} = \\dfrac{\\partial}{\\partial z_j^{(l)}} cost(t)$\n",
    "\n",
    "Recall that our derivative is the slope of a line tangent to the cost function, so the steeper the slope the more incorrect we are. Let us consider the following neural network below and see how we could calculate some $\\delta_j^{(l)}$:\n",
    "\n",
    "![title](images/qc309rdcEea4MxKdJPaTxA_324034f1a3c3a3be8e7c6cfca90d3445_fixx.png)\n",
    "\n",
    "In the image above, to calculate $\\delta_2^{(2)}$, we multiply the weights $\\Theta_{12}^{(2)}$ and $\\Theta_{22}^{(2)}$ by their respective $\\delta$ values found to the right of each edge. So we get $\\delta_2^{(2)} = \\Theta_{12}^{(2)} * \\delta_1^{(3)} + \\Theta_{22}^{(2)} * \\delta_2^{(3)}$. To calculate every single possible $\\delta_j^{(l)}$, we could start from the right of our diagram. We can think of our edges as our $\\Theta_{ij}$. Going from right to left, to calculate the value of $\\delta_j^{(l)}$, you can just take the over all sum of each weight times the $\\delta$ it is coming from. Hence, another example would be $\\delta_2^{(3)} = \\Theta_{12}^{(3)} * \\delta_1^{(4)}$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###  Backpropagation in Practice\n",
    "\n",
    "#### Implementation Note: Unrolling Parameters\n",
    "\n",
    "With neural networks, we are working with sets of matrices:\n",
    "\n",
    "$\\begin{align*} \\Theta^{(1)}, \\Theta^{(2)}, \\Theta^{(3)}, \\dots \\newline D^{(1)}, D^{(2)}, D^{(3)}, \\dots \\end{align*}$\n",
    "\n",
    "In order to use optimizing functions such as \"fminunc()\", we will want to \"unroll\" all the elements and put them into one long vector:\n",
    "\n",
    "```\n",
    "thetaVector = [ Theta1(:); Theta2(:); Theta3(:); ]\n",
    "deltaVector = [ D1(:); D2(:); D3(:) ]\n",
    "```\n",
    "\n",
    "If the dimensions of Theta1 is 10x11, Theta2 is 10x11 and Theta3 is 1x11, then we can get back our original matrices from the \"unrolled\" versions as follows:\n",
    "\n",
    "```\n",
    "Theta1 = reshape(thetaVector(1:110),10,11)\n",
    "Theta2 = reshape(thetaVector(111:220),10,11)\n",
    "Theta3 = reshape(thetaVector(221:231),1,11)\n",
    "```\n",
    "To summarize:\n",
    "\n",
    "![title](images/kdK7ubT2EeajLxLfjQiSjg_d35545b8d6b6940e8577b5a8d75c8657_Screenshot-2016-11-27-15.09.24.png)\n",
    "\n",
    "#### Gradient Checking\n",
    "\n",
    "Gradient checking will assure that our backpropagation works as intended. We can approximate the derivative of our cost function with:\n",
    "\n",
    "$\\dfrac{\\partial}{\\partial\\Theta}J(\\Theta) \\approx \\dfrac{J(\\Theta + \\epsilon) - J(\\Theta - \\epsilon)}{2\\epsilon}$\n",
    "\n",
    "With multiple theta matrices, we can approximate the derivative with respect to $Θ_j$ as follows:\n",
    "\n",
    "$\\dfrac{\\partial}{\\partial\\Theta_j}J(\\Theta) \\approx \\dfrac{J(\\Theta_1, \\dots, \\Theta_j + \\epsilon, \\dots, \\Theta_n) - J(\\Theta_1, \\dots, \\Theta_j - \\epsilon, \\dots, \\Theta_n)}{2\\epsilon}$\n",
    "\n",
    "A small value for ${\\epsilon}$ (epsilon) such as ${\\epsilon = 10^{-4}}$, guarantees that the math works out properly. If the value for $\\epsilon$ is too small, we can end up with numerical problems.\n",
    "\n",
    "Hence, we are only adding or subtracting epsilon to the $\\Theta_j$ matrix. In octave we can do it as follows:\n",
    "\n",
    "```\n",
    "epsilon = 1e-4;\n",
    "for i = 1:n,\n",
    "  thetaPlus = theta;\n",
    "  thetaPlus(i) += epsilon;\n",
    "  thetaMinus = theta;\n",
    "  thetaMinus(i) -= epsilon;\n",
    "  gradApprox(i) = (J(thetaPlus) - J(thetaMinus))/(2*epsilon)\n",
    "end;\n",
    "```\n",
    "\n",
    "We previously saw how to calculate the deltaVector. So once we compute our gradApprox vector, we can check that gradApprox $≈$ deltaVector.\n",
    "\n",
    "Once you have verified **once** that your backpropagation algorithm is correct, you don't need to compute gradApprox again. The code to compute gradApprox can be very slow.\n",
    "\n",
    "#### Random Initialization\n",
    "\n",
    "Initializing all theta weights to zero does not work with neural networks. When we backpropagate, all nodes will update to the same value repeatedly. Instead we can randomly initialize our weights for our $\\Theta$ matrices using the following method:\n",
    "\n",
    "Hence, we initialize each $\\Theta^{(l)}_{ij}$ to a random value between [$-\\epsilon,\\epsilon$]. Using the above formula guarantees that we get the desired bound. The same procedure applies to all the $\\Theta$'s. Below is some working code you could use to experiment.\n",
    "\n",
    "![title](images/y7gaS7pXEeaCrQqTpeD5ng_8868ccda2c387f5d481d0c54ab78a86e_Screen-Shot-2016-12-04-at-11.27.28-AM.png)\n",
    "\n",
    "rand(x,y) is just a function in octave that will initialize a matrix of random real numbers between 0 and 1.\n",
    "\n",
    "(Note: the epsilon used above is unrelated to the epsilon from Gradient Checking)\n",
    "\n",
    "#### Putting it Together\n",
    "\n",
    "First, pick a network architecture; choose the layout of your neural network, including how many hidden units in each layer and how many layers in total you want to have.\n",
    "\n",
    "* Number of input units = dimension of features $x^{(i)}$\n",
    "* Number of output units = number of classes\n",
    "* Number of hidden units per layer = usually more the better (must balance with cost of computation as it increases with more hidden units)\n",
    "* Defaults: 1 hidden layer. If you have more than 1 hidden layer, then it is recommended that you have the same number of units in every hidden layer.\n",
    "\n",
    "##### Training a Neural Network\n",
    "\n",
    "* Randomly initialize the weights\n",
    "* Implement forward propagation to get $h_\\Theta(x^{(i)})$ for any $x^{(i)}$\n",
    "* Implement the cost function\n",
    "* Implement backpropagation to compute partial derivatives\n",
    "* Use gradient checking to confirm that your backpropagation works. Then disable gradient checking.\n",
    "* Use gradient descent or a built-in optimization function to minimize the cost function with the weights in theta.\n",
    "\n",
    "When we perform forward and back propagation, we loop on every training example:\n",
    "\n",
    "```\n",
    "for i = 1:m,\n",
    "   Perform forward propagation and backpropagation using example (x(i),y(i))\n",
    "   (Get activations a(l) and delta terms d(l) for l = 2,...,L\n",
    "```\n",
    "\n",
    "The following image gives us an intuition of what is happening as we are implementing our neural network:\n",
    "\n",
    "![title](images/hGk18LsaEea7TQ6MHcgMPA_8de173808f362583eb39cdd0c89ef43e_Screen-Shot-2016-12-05-at-10.40.35-AM.png)\n",
    "\n",
    "Ideally, you want $h_\\Theta(x^{(i)}) \\approx y^{(i)}$. This will minimize our cost function. However, keep in mind that $J(\\Theta)$ is not convex and thus we can end up in a local minimum instead."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Application of Neural Networks\n",
    "\n",
    "#### Autonomous Driving\n",
    "\n",
    "-"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Review\n",
    "\n",
    "[Lecture9.pdf](lectures/Lecture9.pdf)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}