YUSAG_linear_regression_model.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# YUSAG Linear Regression Model\n",
    "    - Matt Robinson, Yale Undergraduate Sports Analytics Group"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook is one of several notebooks exploring ratings systems often used in sports. All of the notebooks can be found in [this repo](https://github.com/mc-robinson/Ratings_Methods). \n",
    "\n",
    "Specifically, this notebook attempts to both explain and implement our own YUSAG ratings model. Much of this model is derived from the work of Professor [Jay Emerson](http://www.stat.yale.edu/~jay/), and we thank him very much for his guidance. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The Basic Overview ##\n",
    "\n",
    "The material in this notebook follows nicely from the [notebook on Massey's method](https://github.com/mc-robinson/Ratings_Methods/blob/master/massey_ratings.ipynb), in which we explain the basics of least squares and linear regression. But in case you are starting with this notebook, I'll repeat the very, very brief review of linear regression. Please see the references if you want more detail.\n",
    "\n",
    "(Note: for a discussion of where the term \"regression\" comes from, see my notebook [here](https://github.com/mc-robinson/random_tutorials/blob/master/why_we_call_it_regression.ipynb))\n",
    "\n",
    "### The Form of Multiple Linear Regression ###\n",
    "\n",
    "In linear regression, we simply attempt to predict a scalar variable $\\hat y$ as the weighted sum of a bunch of input features (explanatory variables) and a bias term (the interecept). The general form is:\n",
    "\n",
    "$$\n",
    "\\hat y = w_0 + w_1 x_1 + w_2 x_2 + \\cdots + w_n x_n\n",
    "$$\n",
    "where:\n",
    "* $\\hat y$ is the predicted response variable\n",
    "* $x_i$ is the i'th feature value\n",
    "* $w_j$ is the j'th model parameter\n",
    "    * $w_0$ is the bias term\n",
    "    * $w_1, w_2,...,w_n$ are the feature weights\n",
    "* $n$ is the number of features\n",
    "\n",
    "So how do we figure out the values of the model coefficients $w_0,...,w_n$? The answer is that we learn these parameters when we train the linear model on the data. In fact, for linear regression, the fit is determined using the least squares criterion. That is, we seek the parameters that minimize the sum of squared errors. Once the model is trained, and the best parameters are learned, we can use the model for prediction."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Working Through an Example ##\n",
    "\n",
    "In order to work through the implementation of this method and understand how it works, I'm going to use an extremely simplified example. Let's imagine the Ivy league is instead the IV league and consists of only four teams who each play each other once: Harvard, Princeton, Yale, and Columbia.\n",
    "\n",
    "Here are the results from the 2016 IV season that I scraped from the NCAA stats website:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>year</th>\n",
       "      <th>month</th>\n",
       "      <th>day</th>\n",
       "      <th>team</th>\n",
       "      <th>opponent</th>\n",
       "      <th>location</th>\n",
       "      <th>team_score</th>\n",
       "      <th>opponent_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2016</td>\n",
       "      <td>10</td>\n",
       "      <td>1</td>\n",
       "      <td>Columbia</td>\n",
       "      <td>Princeton</td>\n",
       "      <td>1</td>\n",
       "      <td>13</td>\n",
       "      <td>48</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2016</td>\n",
       "      <td>10</td>\n",
       "      <td>1</td>\n",
       "      <td>Princeton</td>\n",
       "      <td>Columbia</td>\n",
       "      <td>-1</td>\n",
       "      <td>48</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2016</td>\n",
       "      <td>10</td>\n",
       "      <td>22</td>\n",
       "      <td>Harvard</td>\n",
       "      <td>Princeton</td>\n",
       "      <td>-1</td>\n",
       "      <td>23</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2016</td>\n",
       "      <td>10</td>\n",
       "      <td>22</td>\n",
       "      <td>Princeton</td>\n",
       "      <td>Harvard</td>\n",
       "      <td>1</td>\n",
       "      <td>20</td>\n",
       "      <td>23</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2016</td>\n",
       "      <td>10</td>\n",
       "      <td>28</td>\n",
       "      <td>Columbia</td>\n",
       "      <td>Yale</td>\n",
       "      <td>1</td>\n",
       "      <td>23</td>\n",
       "      <td>31</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2016</td>\n",
       "      <td>10</td>\n",
       "      <td>28</td>\n",
       "      <td>Yale</td>\n",
       "      <td>Columbia</td>\n",
       "      <td>-1</td>\n",
       "      <td>31</td>\n",
       "      <td>23</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>2016</td>\n",
       "      <td>11</td>\n",
       "      <td>5</td>\n",
       "      <td>Columbia</td>\n",
       "      <td>Harvard</td>\n",
       "      <td>-1</td>\n",
       "      <td>21</td>\n",
       "      <td>28</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2016</td>\n",
       "      <td>11</td>\n",
       "      <td>5</td>\n",
       "      <td>Harvard</td>\n",
       "      <td>Columbia</td>\n",
       "      <td>1</td>\n",
       "      <td>28</td>\n",
       "      <td>21</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>2016</td>\n",
       "      <td>11</td>\n",
       "      <td>12</td>\n",
       "      <td>Princeton</td>\n",
       "      <td>Yale</td>\n",
       "      <td>-1</td>\n",
       "      <td>31</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>2016</td>\n",
       "      <td>11</td>\n",
       "      <td>12</td>\n",
       "      <td>Yale</td>\n",
       "      <td>Princeton</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>31</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>2016</td>\n",
       "      <td>11</td>\n",
       "      <td>19</td>\n",
       "      <td>Harvard</td>\n",
       "      <td>Yale</td>\n",
       "      <td>1</td>\n",
       "      <td>14</td>\n",
       "      <td>21</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>2016</td>\n",
       "      <td>11</td>\n",
       "      <td>19</td>\n",
       "      <td>Yale</td>\n",
       "      <td>Harvard</td>\n",
       "      <td>-1</td>\n",
       "      <td>21</td>\n",
       "      <td>14</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    year  month  day       team   opponent  location  team_score  \\\n",
       "0   2016     10    1   Columbia  Princeton         1          13   \n",
       "1   2016     10    1  Princeton   Columbia        -1          48   \n",
       "2   2016     10   22    Harvard  Princeton        -1          23   \n",
       "3   2016     10   22  Princeton    Harvard         1          20   \n",
       "4   2016     10   28   Columbia       Yale         1          23   \n",
       "5   2016     10   28       Yale   Columbia        -1          31   \n",
       "6   2016     11    5   Columbia    Harvard        -1          21   \n",
       "7   2016     11    5    Harvard   Columbia         1          28   \n",
       "8   2016     11   12  Princeton       Yale        -1          31   \n",
       "9   2016     11   12       Yale  Princeton         1           3   \n",
       "10  2016     11   19    Harvard       Yale         1          14   \n",
       "11  2016     11   19       Yale    Harvard        -1          21   \n",
       "\n",
       "    opponent_score  \n",
       "0               48  \n",
       "1               13  \n",
       "2               20  \n",
       "3               23  \n",
       "4               31  \n",
       "5               23  \n",
       "6               28  \n",
       "7               21  \n",
       "8                3  \n",
       "9               31  \n",
       "10              21  \n",
       "11              14  "
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "IV_df = pd.read_csv('IV_league_2016_YUSAG_lin_reg_data.csv')\n",
    "IV_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now you probably noticed that every game is repeated in the above csv file. This is the way we get the data when we scrape the NCAA stats websites. The logical thing to do would be to only keep one copy of each game. But we don't! We've actually found it's quite fine to keep two copies of each game with the `team` and `opponent` variables switched (more on this later)\n",
    "\n",
    "Note that the `location` variable refers to the location of the `team` variable:\n",
    "* Home = 1\n",
    "* Neutral = 0\n",
    "* Away = -1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The Model ###\n",
    "\n",
    "The goal of our linear regression model is quite simple; we are trying to explain the score differential of each game based on the strength of the `team`, the strength of the `opponent`, and the `location`. \n",
    "\n",
    "Thus we need to create a `score_diff` response variable for each game:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "IV_df['score_diff'] = IV_df['team_score']-IV_df['opponent_score']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>year</th>\n",
       "      <th>month</th>\n",
       "      <th>day</th>\n",
       "      <th>team</th>\n",
       "      <th>opponent</th>\n",
       "      <th>location</th>\n",
       "      <th>team_score</th>\n",
       "      <th>opponent_score</th>\n",
       "      <th>score_diff</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2016</td>\n",
       "      <td>10</td>\n",
       "      <td>1</td>\n",
       "      <td>Columbia</td>\n",
       "      <td>Princeton</td>\n",
       "      <td>1</td>\n",
       "      <td>13</td>\n",
       "      <td>48</td>\n",
       "      <td>-35</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2016</td>\n",
       "      <td>10</td>\n",
       "      <td>1</td>\n",
       "      <td>Princeton</td>\n",
       "      <td>Columbia</td>\n",
       "      <td>-1</td>\n",
       "      <td>48</td>\n",
       "      <td>13</td>\n",
       "      <td>35</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2016</td>\n",
       "      <td>10</td>\n",
       "      <td>22</td>\n",
       "      <td>Harvard</td>\n",
       "      <td>Princeton</td>\n",
       "      <td>-1</td>\n",
       "      <td>23</td>\n",
       "      <td>20</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2016</td>\n",
       "      <td>10</td>\n",
       "      <td>22</td>\n",
       "      <td>Princeton</td>\n",
       "      <td>Harvard</td>\n",
       "      <td>1</td>\n",
       "      <td>20</td>\n",
       "      <td>23</td>\n",
       "      <td>-3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2016</td>\n",
       "      <td>10</td>\n",
       "      <td>28</td>\n",
       "      <td>Columbia</td>\n",
       "      <td>Yale</td>\n",
       "      <td>1</td>\n",
       "      <td>23</td>\n",
       "      <td>31</td>\n",
       "      <td>-8</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   year  month  day       team   opponent  location  team_score  \\\n",
       "0  2016     10    1   Columbia  Princeton         1          13   \n",
       "1  2016     10    1  Princeton   Columbia        -1          48   \n",
       "2  2016     10   22    Harvard  Princeton        -1          23   \n",
       "3  2016     10   22  Princeton    Harvard         1          20   \n",
       "4  2016     10   28   Columbia       Yale         1          23   \n",
       "\n",
       "   opponent_score  score_diff  \n",
       "0              48         -35  \n",
       "1              13          35  \n",
       "2              20           3  \n",
       "3              23          -3  \n",
       "4              31          -8  "
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "IV_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You may notice that the `team` and `opponent` features are categorical, and thus are not currently ripe for use with linear regression. However, we can use what is called 'one hot encoding' in order to transform these features into a usable form. One hot encoding works by taking the `team` feature, for example, and transforming it into many features such as `team_Yale` and `team_Harvard`. This  `team_Yale` feature will usally equal zero, except when the `team` is actually Yale, then `team_Yale` will equal 1. In this way, it's a binary encoding (which is actually very useful for us as we'll see later).\n",
    "One can use sklearn.preprocessing.OneHotEncoder for this task, but I am going to use Pandas instead:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# create dummy variables, need to do this in python b/c does not handle automatically like R\n",
    "team_dummies = pd.get_dummies(IV_df.team, prefix='team')\n",
    "opponent_dummies = pd.get_dummies(IV_df.opponent, prefix='opponent')\n",
    "\n",
    "IV_df = pd.concat([IV_df, team_dummies, opponent_dummies], axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>year</th>\n",
       "      <th>month</th>\n",
       "      <th>day</th>\n",
       "      <th>team</th>\n",
       "      <th>opponent</th>\n",
       "      <th>location</th>\n",
       "      <th>team_score</th>\n",
       "      <th>opponent_score</th>\n",
       "      <th>score_diff</th>\n",
       "      <th>team_Columbia</th>\n",
       "      <th>team_Harvard</th>\n",
       "      <th>team_Princeton</th>\n",
       "      <th>team_Yale</th>\n",
       "      <th>opponent_Columbia</th>\n",
       "      <th>opponent_Harvard</th>\n",
       "      <th>opponent_Princeton</th>\n",
       "      <th>opponent_Yale</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2016</td>\n",
       "      <td>10</td>\n",
       "      <td>1</td>\n",
       "      <td>Columbia</td>\n",
       "      <td>Princeton</td>\n",
       "      <td>1</td>\n",
       "      <td>13</td>\n",
       "      <td>48</td>\n",
       "      <td>-35</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2016</td>\n",
       "      <td>10</td>\n",
       "      <td>1</td>\n",
       "      <td>Princeton</td>\n",
       "      <td>Columbia</td>\n",
       "      <td>-1</td>\n",
       "      <td>48</td>\n",
       "      <td>13</td>\n",
       "      <td>35</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2016</td>\n",
       "      <td>10</td>\n",
       "      <td>22</td>\n",
       "      <td>Harvard</td>\n",
       "      <td>Princeton</td>\n",
       "      <td>-1</td>\n",
       "      <td>23</td>\n",
       "      <td>20</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2016</td>\n",
       "      <td>10</td>\n",
       "      <td>22</td>\n",
       "      <td>Princeton</td>\n",
       "      <td>Harvard</td>\n",
       "      <td>1</td>\n",
       "      <td>20</td>\n",
       "      <td>23</td>\n",
       "      <td>-3</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2016</td>\n",
       "      <td>10</td>\n",
       "      <td>28</td>\n",
       "      <td>Columbia</td>\n",
       "      <td>Yale</td>\n",
       "      <td>1</td>\n",
       "      <td>23</td>\n",
       "      <td>31</td>\n",
       "      <td>-8</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   year  month  day       team   opponent  location  team_score  \\\n",
       "0  2016     10    1   Columbia  Princeton         1          13   \n",
       "1  2016     10    1  Princeton   Columbia        -1          48   \n",
       "2  2016     10   22    Harvard  Princeton        -1          23   \n",
       "3  2016     10   22  Princeton    Harvard         1          20   \n",
       "4  2016     10   28   Columbia       Yale         1          23   \n",
       "\n",
       "   opponent_score  score_diff  team_Columbia  team_Harvard  team_Princeton  \\\n",
       "0              48         -35              1             0               0   \n",
       "1              13          35              0             0               1   \n",
       "2              20           3              0             1               0   \n",
       "3              23          -3              0             0               1   \n",
       "4              31          -8              1             0               0   \n",
       "\n",
       "   team_Yale  opponent_Columbia  opponent_Harvard  opponent_Princeton  \\\n",
       "0          0                  0                 0                   1   \n",
       "1          0                  1                 0                   0   \n",
       "2          0                  0                 0                   1   \n",
       "3          0                  0                 1                   0   \n",
       "4          0                  0                 0                   0   \n",
       "\n",
       "   opponent_Yale  \n",
       "0              0  \n",
       "1              0  \n",
       "2              0  \n",
       "3              0  \n",
       "4              1  "
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "IV_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's make our training data, so that we can construct the model. For now, I am going to ignore the `location` feature."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# make the training data\n",
    "X = IV_df.drop(['year','month','day','team','opponent','team_score','opponent_score','score_diff','location'], axis=1)\n",
    "y = IV_df['score_diff']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>team_Columbia</th>\n",
       "      <th>team_Harvard</th>\n",
       "      <th>team_Princeton</th>\n",
       "      <th>team_Yale</th>\n",
       "      <th>opponent_Columbia</th>\n",
       "      <th>opponent_Harvard</th>\n",
       "      <th>opponent_Princeton</th>\n",
       "      <th>opponent_Yale</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   team_Columbia  team_Harvard  team_Princeton  team_Yale  opponent_Columbia  \\\n",
       "0              1             0               0          0                  0   \n",
       "1              0             0               1          0                  1   \n",
       "2              0             1               0          0                  0   \n",
       "3              0             0               1          0                  0   \n",
       "4              1             0               0          0                  0   \n",
       "\n",
       "   opponent_Harvard  opponent_Princeton  opponent_Yale  \n",
       "0                 0                   1              0  \n",
       "1                 0                   0              0  \n",
       "2                 0                   1              0  \n",
       "3                 1                   0              0  \n",
       "4                 0                   0              1  "
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0   -35\n",
       "1    35\n",
       "2     3\n",
       "3    -3\n",
       "4    -8\n",
       "Name: score_diff, dtype: int64"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's train the linear regression model. I am going to force the bias term (intercept) of the model to be 0, just to make the interpretation of the model slightly easier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LinearRegression(copy_X=True, fit_intercept=False, n_jobs=1, normalize=False)"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.linear_model import LinearRegression\n",
    "lin_reg = LinearRegression(fit_intercept=False)\n",
    "lin_reg.fit(X, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.0\n",
      "[-12.5    0.75  15.    -3.25  12.5   -0.75 -15.     3.25]\n"
     ]
    }
   ],
   "source": [
    "# print the coefficients\n",
    "print(lin_reg.intercept_)\n",
    "print(lin_reg.coef_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "R^2 on the training data:\n",
      "0.71995412844\n"
     ]
    }
   ],
   "source": [
    "# get the R^2 value on the training data\n",
    "r_squared = lin_reg.score(X, y)\n",
    "print('R^2 on the training data:')\n",
    "print(r_squared)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that the model is trained, let's look at the model coefficients for each team. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>feature</th>\n",
       "      <th>feature_coef</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>team_Columbia</td>\n",
       "      <td>-12.50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>team_Harvard</td>\n",
       "      <td>0.75</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>team_Princeton</td>\n",
       "      <td>15.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>team_Yale</td>\n",
       "      <td>-3.25</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>opponent_Columbia</td>\n",
       "      <td>12.50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>opponent_Harvard</td>\n",
       "      <td>-0.75</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>opponent_Princeton</td>\n",
       "      <td>-15.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>opponent_Yale</td>\n",
       "      <td>3.25</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              feature  feature_coef\n",
       "0       team_Columbia        -12.50\n",
       "1        team_Harvard          0.75\n",
       "2      team_Princeton         15.00\n",
       "3           team_Yale         -3.25\n",
       "4   opponent_Columbia         12.50\n",
       "5    opponent_Harvard         -0.75\n",
       "6  opponent_Princeton        -15.00\n",
       "7       opponent_Yale          3.25"
      ]
     },
     "execution_count": 84,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# get the coefficients for each feature\n",
    "coef_data = list(zip(X.columns,lin_reg.coef_))\n",
    "coef_df = pd.DataFrame(coef_data,columns=['feature','feature_coef'])\n",
    "coef_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's get our ratings for each team. Note that in this model, a team's rating is simply defined as its linear regression coefficient for the `team_name` variable, which we call the ***YUSAG coefficient***. Let's eliminate the `opponent_name` variables so we have the true ratings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# first get rid of opponent_ variables\n",
    "team_df = coef_df[~coef_df['feature'].str.contains(\"opponent\")]\n",
    "\n",
    "# rank them by coef, not alphabetical order\n",
    "ranked_team_df = team_df.sort_values(['feature_coef'],ascending=False)\n",
    "\n",
    "# reset the indices at 0\n",
    "ranked_team_df = ranked_team_df.reset_index(drop=True);\n",
    "\n",
    "# rename 'feature_coef' column\n",
    "ranked_team_df = ranked_team_df.rename(columns={'feature_coef': 'YUSAG_coefficient'})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>feature</th>\n",
       "      <th>YUSAG_coefficient</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>team_Princeton</td>\n",
       "      <td>15.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>team_Harvard</td>\n",
       "      <td>0.75</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>team_Yale</td>\n",
       "      <td>-3.25</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>team_Columbia</td>\n",
       "      <td>-12.50</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          feature  YUSAG_coefficient\n",
       "0  team_Princeton              15.00\n",
       "1    team_Harvard               0.75\n",
       "2       team_Yale              -3.25\n",
       "3   team_Columbia             -12.50"
      ]
     },
     "execution_count": 86,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ranked_team_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note: this is exactly the answer we got from Massey's method, as explained in [this notebook](https://github.com/mc-robinson/Ratings_Methods/blob/master/massey_ratings.ipynb). If you read the end of that notebook, the connection should be relatively clear.\n",
    "\n",
    "Now let's re-train the model, while also including the `location` feature:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# make the training data (now with location)\n",
    "X = IV_df.drop(['year','month','day','team','opponent','team_score','opponent_score','score_diff'], axis=1)\n",
    "y = IV_df['score_diff']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LinearRegression(copy_X=True, fit_intercept=False, n_jobs=1, normalize=False)"
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.linear_model import LinearRegression\n",
    "lin_reg = LinearRegression(fit_intercept=False)\n",
    "lin_reg.fit(X, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.0\n",
      "[-10.1    -9.975   3.275  12.475  -5.775   9.975  -3.275 -12.475   5.775]\n"
     ]
    }
   ],
   "source": [
    "# print the coefficients\n",
    "print(lin_reg.intercept_)\n",
    "print(lin_reg.coef_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "R^2 on the training data:\n",
      "0.953922018349\n"
     ]
    }
   ],
   "source": [
    "# get the R^2 value on the training data\n",
    "r_squared = lin_reg.score(X, y)\n",
    "print('R^2 on the training data:')\n",
    "print(r_squared)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>feature</th>\n",
       "      <th>feature_coef</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>location</td>\n",
       "      <td>-10.100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>team_Columbia</td>\n",
       "      <td>-9.975</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>team_Harvard</td>\n",
       "      <td>3.275</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>team_Princeton</td>\n",
       "      <td>12.475</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>team_Yale</td>\n",
       "      <td>-5.775</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>opponent_Columbia</td>\n",
       "      <td>9.975</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>opponent_Harvard</td>\n",
       "      <td>-3.275</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>opponent_Princeton</td>\n",
       "      <td>-12.475</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>opponent_Yale</td>\n",
       "      <td>5.775</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              feature  feature_coef\n",
       "0            location       -10.100\n",
       "1       team_Columbia        -9.975\n",
       "2        team_Harvard         3.275\n",
       "3      team_Princeton        12.475\n",
       "4           team_Yale        -5.775\n",
       "5   opponent_Columbia         9.975\n",
       "6    opponent_Harvard        -3.275\n",
       "7  opponent_Princeton       -12.475\n",
       "8       opponent_Yale         5.775"
      ]
     },
     "execution_count": 91,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# get the coefficients for each feature\n",
    "coef_data = list(zip(X.columns,lin_reg.coef_))\n",
    "coef_df = pd.DataFrame(coef_data,columns=['feature','feature_coef'])\n",
    "coef_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# first get rid of opponent_ variables\n",
    "team_df = coef_df[~coef_df['feature'].str.contains(\"opponent\")]\n",
    "\n",
    "# get rid of the location variable\n",
    "team_df = team_df.iloc[1:]\n",
    "\n",
    "# rank them by coef, not alphabetical order\n",
    "ranked_team_df = team_df.sort_values(['feature_coef'],ascending=False)\n",
    "\n",
    "# reset the indices at 0\n",
    "ranked_team_df = ranked_team_df.reset_index(drop=True);\n",
    "\n",
    "# rename 'feature_coef' column\n",
    "ranked_team_df = ranked_team_df.rename(columns={'feature_coef': 'YUSAG_coefficient'})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>feature</th>\n",
       "      <th>YUSAG_coefficient</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>team_Princeton</td>\n",
       "      <td>12.475</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>team_Harvard</td>\n",
       "      <td>3.275</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>team_Yale</td>\n",
       "      <td>-5.775</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>team_Columbia</td>\n",
       "      <td>-9.975</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          feature  YUSAG_coefficient\n",
       "0  team_Princeton             12.475\n",
       "1    team_Harvard              3.275\n",
       "2       team_Yale             -5.775\n",
       "3   team_Columbia             -9.975"
      ]
     },
     "execution_count": 93,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ranked_team_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's go through the details of the model. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### How The Model Actually Works ###\n",
    "\n",
    "You may notice that the coefficients for `team_Yale` and `opponent_Yale` are just the negatives of each other. This is true for every team and is precisely why we include two copies of every game in our training data. The coefficient for the `team_Yale` feature is what we have called the YUSAG coefficient.\n",
    "\n",
    "When predicting a game's score differential on a **neutral field**, the predicted score differential (`score_diff`) is just the difference in YUSAG coefficients. The reason this works is the binary encoding we did earlier.\n",
    "\n",
    "So let's think about what we are doing when we predict the score differential for the Princeton-Harvard game with `team` = Princeton and `opponent` = Harvard on a neutral field.\n",
    "\n",
    "In our model, the coefficients are as follows:\n",
    "* team_Princeton_coef = 12.475\n",
    "* opponent_Princeton_coef = -12.475\n",
    "* team_Harvard_coef = 3.275\n",
    "* opponent_Harvard_coef = -3.275\n",
    "\n",
    "when we go to use the model for this game, it looks like this:\n",
    "\n",
    "`score_diff` = (location_coef $*$ `location`) + (team_Princeton_coef $*$ `team_Princeton`) + (opponent_Princeton_coef $*$ `opponent_Princeton`) + (team_Harvard_coef $*$ `team_Harvard`) + (opponent_Harvard_coef $*$ `opponent_Harvard`) + (team_Yale_coef $*$ `team_Yale`) + (opponent_Yale_coef $*$ `opponent_Yale`) + (team_Columbia_coef $*$ `team_Columbia`) + (opponent_Columbia_coef $*$ `opponent_Columbia`)\n",
    "\n",
    "\n",
    "To put numbers in for the variables, the model looks like this:\n",
    "\n",
    "\n",
    "`score_diff` = (location_coef $*$ 0) + (team_Princeton_coef $*$ 1) + (opponent_Princeton_coef $*$ 0) + (team_Harvard_coef $*$ 0) + (opponent_Harvard_coef $*$ 1) + $\\cdots \\\\$\n",
    "\n",
    "where are the other terms are simply $0$.\n",
    "\n",
    "Which can also be written as:\n",
    "\n",
    "`score_diff` = (location_coef $*$ 0) + (12.475 $*$ 1) + (-3.275 $*$ 1) = 12.475 - 3.275 = Princeton_YUSAG_coef - Harvard_YUSAG_coef\n",
    "\n",
    "Thus showing how the difference in YUSAG coefficients is the same as the predicted score differential. Furthermore, the higher YUSAG coefficient a team has, the better they are.\n",
    "\n",
    "Lastly, if the Princeton-Harvard game was to be home at Princeton, we would just add the location_coef:\n",
    "\n",
    "`score_diff` = (location_coef $*$ $1$) + (team_Princeton_coef $*$ $1$) + (opponent_Harvard_coef $*$ $1$) = $-10.1 + 12.475 - 3.275$ = location_coef + Princeton_YUSAG_coef - Harvard_YUSAG_coef\n",
    "\n",
    "Note: With this small sample size of games, we have somehow selected 6 games with the visiting team winning 5 of them. Therefore, our location coefficient has surprisingly come out to be strongly negative. Over the course of a whole season, it will surely become positive (usually around 2-3 points). "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When we actually run our YUSAG model, we use ridge regression (adds an l2 penalty with alpha = 1.0) because that prevents the model from overfitting and also limits the values of the coefficients to not be huge (this sometimes happens when running on whole season of data). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Ridge(alpha=1.0, copy_X=True, fit_intercept=False, max_iter=None,\n",
       "   normalize=False, random_state=None, solver='auto', tol=0.001)"
      ]
     },
     "execution_count": 94,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.linear_model import Ridge\n",
    "ridge_reg = Ridge(fit_intercept=False)\n",
    "ridge_reg.fit(X, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.0\n",
      "[ -9.68421053  -8.06315789   2.53684211  10.06315789  -4.53684211\n",
      "   8.06315789  -2.53684211 -10.06315789   4.53684211]\n"
     ]
    }
   ],
   "source": [
    "# print the coefficients\n",
    "print(ridge_reg.intercept_)\n",
    "print(ridge_reg.coef_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "R^2 on the training data:\n",
      "0.931358052301\n"
     ]
    }
   ],
   "source": [
    "# get the R^2 value\n",
    "r_squared = ridge_reg.score(X, y)\n",
    "print('R^2 on the training data:')\n",
    "print(r_squared)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>feature</th>\n",
       "      <th>feature_coef</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>location</td>\n",
       "      <td>-9.684211</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>team_Columbia</td>\n",
       "      <td>-8.063158</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>team_Harvard</td>\n",
       "      <td>2.536842</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>team_Princeton</td>\n",
       "      <td>10.063158</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>team_Yale</td>\n",
       "      <td>-4.536842</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>opponent_Columbia</td>\n",
       "      <td>8.063158</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>opponent_Harvard</td>\n",
       "      <td>-2.536842</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>opponent_Princeton</td>\n",
       "      <td>-10.063158</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>opponent_Yale</td>\n",
       "      <td>4.536842</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              feature  feature_coef\n",
       "0            location     -9.684211\n",
       "1       team_Columbia     -8.063158\n",
       "2        team_Harvard      2.536842\n",
       "3      team_Princeton     10.063158\n",
       "4           team_Yale     -4.536842\n",
       "5   opponent_Columbia      8.063158\n",
       "6    opponent_Harvard     -2.536842\n",
       "7  opponent_Princeton    -10.063158\n",
       "8       opponent_Yale      4.536842"
      ]
     },
     "execution_count": 97,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# get the coefficients for each feature\n",
    "coef_data = list(zip(X.columns,ridge_reg.coef_))\n",
    "coef_df = pd.DataFrame(coef_data,columns=['feature','feature_coef'])\n",
    "coef_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# first get rid of opponent_ variables\n",
    "team_df = coef_df[~coef_df['feature'].str.contains(\"opponent\")]\n",
    "\n",
    "# get rid of the location variable\n",
    "team_df = team_df.iloc[1:]\n",
    "\n",
    "# rank them by coef, not alphabetical order\n",
    "ranked_team_df = team_df.sort_values(['feature_coef'],ascending=False)\n",
    "\n",
    "# reset the indices at 0\n",
    "ranked_team_df = ranked_team_df.reset_index(drop=True);\n",
    "\n",
    "# rename 'feature_coef' column\n",
    "ranked_team_df = ranked_team_df.rename(columns={'feature_coef': 'YUSAG_coefficient'})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>feature</th>\n",
       "      <th>YUSAG_coefficient</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>team_Princeton</td>\n",
       "      <td>10.063158</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>team_Harvard</td>\n",
       "      <td>2.536842</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>team_Yale</td>\n",
       "      <td>-4.536842</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>team_Columbia</td>\n",
       "      <td>-8.063158</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          feature  YUSAG_coefficient\n",
       "0  team_Princeton          10.063158\n",
       "1    team_Harvard           2.536842\n",
       "2       team_Yale          -4.536842\n",
       "3   team_Columbia          -8.063158"
      ]
     },
     "execution_count": 99,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ranked_team_df"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}