churn.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Predicting Customer Churn\n",
    "\n",
    "**Credit:** http://blog.yhathq.com/posts/predicting-customer-churn-with-sklearn.html"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We first import the required packages: \n",
    "\n",
    "1. Numpy for scientific computing;\n",
    "2. Pandas for data processing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We then import the Scikit-Learn package. In this demo, we will try 3 machine learning algorithms that are\n",
    "\n",
    "1. *k*-Nearest Neigbors;\n",
    "2. Naive Bayes;\n",
    "3. Support Vector Machine."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn import neighbors, naive_bayes, svm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Moreover, we import the modules for evaluating the algorithms and proprocessing data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.metrics import accuracy_score, classification_report"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preprocessing Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "We use Pandas to load the data from a CSV file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Column names:\n",
      "['State', 'Account Length', 'Area Code', 'Phone', \"Int'l Plan\", 'VMail Plan', 'VMail Message', 'Day Mins', 'Day Calls', 'Day Charge', 'Eve Mins', 'Eve Calls', 'Eve Charge', 'Night Mins', 'Night Calls', 'Night Charge', 'Intl Mins', 'Intl Calls', 'Intl Charge', 'CustServ Calls', 'Churn?']\n"
     ]
    }
   ],
   "source": [
    "churn_df = pd.read_csv('data/churn.csv')\n",
    "col_names = churn_df.columns.tolist()\n",
    "\n",
    "print(\"Column names:\")\n",
    "print(col_names)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sample data:\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>State</th>\n",
       "      <th>Account Length</th>\n",
       "      <th>Area Code</th>\n",
       "      <th>Phone</th>\n",
       "      <th>Int'l Plan</th>\n",
       "      <th>VMail Plan</th>\n",
       "      <th>VMail Message</th>\n",
       "      <th>Day Mins</th>\n",
       "      <th>Day Calls</th>\n",
       "      <th>Day Charge</th>\n",
       "      <th>...</th>\n",
       "      <th>Eve Calls</th>\n",
       "      <th>Eve Charge</th>\n",
       "      <th>Night Mins</th>\n",
       "      <th>Night Calls</th>\n",
       "      <th>Night Charge</th>\n",
       "      <th>Intl Mins</th>\n",
       "      <th>Intl Calls</th>\n",
       "      <th>Intl Charge</th>\n",
       "      <th>CustServ Calls</th>\n",
       "      <th>Churn?</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>KS</td>\n",
       "      <td>128</td>\n",
       "      <td>415</td>\n",
       "      <td>382-4657</td>\n",
       "      <td>no</td>\n",
       "      <td>yes</td>\n",
       "      <td>25</td>\n",
       "      <td>265.1</td>\n",
       "      <td>110</td>\n",
       "      <td>45.07</td>\n",
       "      <td>...</td>\n",
       "      <td>99</td>\n",
       "      <td>16.78</td>\n",
       "      <td>244.7</td>\n",
       "      <td>91</td>\n",
       "      <td>11.01</td>\n",
       "      <td>10.0</td>\n",
       "      <td>3</td>\n",
       "      <td>2.70</td>\n",
       "      <td>1</td>\n",
       "      <td>False.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>OH</td>\n",
       "      <td>107</td>\n",
       "      <td>415</td>\n",
       "      <td>371-7191</td>\n",
       "      <td>no</td>\n",
       "      <td>yes</td>\n",
       "      <td>26</td>\n",
       "      <td>161.6</td>\n",
       "      <td>123</td>\n",
       "      <td>27.47</td>\n",
       "      <td>...</td>\n",
       "      <td>103</td>\n",
       "      <td>16.62</td>\n",
       "      <td>254.4</td>\n",
       "      <td>103</td>\n",
       "      <td>11.45</td>\n",
       "      <td>13.7</td>\n",
       "      <td>3</td>\n",
       "      <td>3.70</td>\n",
       "      <td>1</td>\n",
       "      <td>False.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NJ</td>\n",
       "      <td>137</td>\n",
       "      <td>415</td>\n",
       "      <td>358-1921</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>0</td>\n",
       "      <td>243.4</td>\n",
       "      <td>114</td>\n",
       "      <td>41.38</td>\n",
       "      <td>...</td>\n",
       "      <td>110</td>\n",
       "      <td>10.30</td>\n",
       "      <td>162.6</td>\n",
       "      <td>104</td>\n",
       "      <td>7.32</td>\n",
       "      <td>12.2</td>\n",
       "      <td>5</td>\n",
       "      <td>3.29</td>\n",
       "      <td>0</td>\n",
       "      <td>False.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>OH</td>\n",
       "      <td>84</td>\n",
       "      <td>408</td>\n",
       "      <td>375-9999</td>\n",
       "      <td>yes</td>\n",
       "      <td>no</td>\n",
       "      <td>0</td>\n",
       "      <td>299.4</td>\n",
       "      <td>71</td>\n",
       "      <td>50.90</td>\n",
       "      <td>...</td>\n",
       "      <td>88</td>\n",
       "      <td>5.26</td>\n",
       "      <td>196.9</td>\n",
       "      <td>89</td>\n",
       "      <td>8.86</td>\n",
       "      <td>6.6</td>\n",
       "      <td>7</td>\n",
       "      <td>1.78</td>\n",
       "      <td>2</td>\n",
       "      <td>False.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>OK</td>\n",
       "      <td>75</td>\n",
       "      <td>415</td>\n",
       "      <td>330-6626</td>\n",
       "      <td>yes</td>\n",
       "      <td>no</td>\n",
       "      <td>0</td>\n",
       "      <td>166.7</td>\n",
       "      <td>113</td>\n",
       "      <td>28.34</td>\n",
       "      <td>...</td>\n",
       "      <td>122</td>\n",
       "      <td>12.61</td>\n",
       "      <td>186.9</td>\n",
       "      <td>121</td>\n",
       "      <td>8.41</td>\n",
       "      <td>10.1</td>\n",
       "      <td>3</td>\n",
       "      <td>2.73</td>\n",
       "      <td>3</td>\n",
       "      <td>False.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>AL</td>\n",
       "      <td>118</td>\n",
       "      <td>510</td>\n",
       "      <td>391-8027</td>\n",
       "      <td>yes</td>\n",
       "      <td>no</td>\n",
       "      <td>0</td>\n",
       "      <td>223.4</td>\n",
       "      <td>98</td>\n",
       "      <td>37.98</td>\n",
       "      <td>...</td>\n",
       "      <td>101</td>\n",
       "      <td>18.75</td>\n",
       "      <td>203.9</td>\n",
       "      <td>118</td>\n",
       "      <td>9.18</td>\n",
       "      <td>6.3</td>\n",
       "      <td>6</td>\n",
       "      <td>1.70</td>\n",
       "      <td>0</td>\n",
       "      <td>False.</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>6 rows × 21 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "  State  Account Length  Area Code     Phone Int'l Plan VMail Plan  \\\n",
       "0    KS             128        415  382-4657         no        yes   \n",
       "1    OH             107        415  371-7191         no        yes   \n",
       "2    NJ             137        415  358-1921         no         no   \n",
       "3    OH              84        408  375-9999        yes         no   \n",
       "4    OK              75        415  330-6626        yes         no   \n",
       "5    AL             118        510  391-8027        yes         no   \n",
       "\n",
       "   VMail Message  Day Mins  Day Calls  Day Charge   ...    Eve Calls  \\\n",
       "0             25     265.1        110       45.07   ...           99   \n",
       "1             26     161.6        123       27.47   ...          103   \n",
       "2              0     243.4        114       41.38   ...          110   \n",
       "3              0     299.4         71       50.90   ...           88   \n",
       "4              0     166.7        113       28.34   ...          122   \n",
       "5              0     223.4         98       37.98   ...          101   \n",
       "\n",
       "   Eve Charge  Night Mins  Night Calls  Night Charge  Intl Mins  Intl Calls  \\\n",
       "0       16.78       244.7           91         11.01       10.0           3   \n",
       "1       16.62       254.4          103         11.45       13.7           3   \n",
       "2       10.30       162.6          104          7.32       12.2           5   \n",
       "3        5.26       196.9           89          8.86        6.6           7   \n",
       "4       12.61       186.9          121          8.41       10.1           3   \n",
       "5       18.75       203.9          118          9.18        6.3           6   \n",
       "\n",
       "   Intl Charge  CustServ Calls  Churn?  \n",
       "0         2.70               1  False.  \n",
       "1         3.70               1  False.  \n",
       "2         3.29               0  False.  \n",
       "3         1.78               2  False.  \n",
       "4         2.73               3  False.  \n",
       "5         1.70               0  False.  \n",
       "\n",
       "[6 rows x 21 columns]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print(\"Sample data:\")\n",
    "churn_df.head(6)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Separate the classification target from the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "churn_result = churn_df['Churn?']\n",
    "y = np.where(churn_result == 'True.', 1, 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Remove some unnecessary data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "to_drop = ['State', 'Area Code', 'Phone','Churn?']\n",
    "churn_feat_space = churn_df.drop(to_drop, axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since some columns contain 'yes' or 'no' data, we should convert them into Boolean. This will make it easier for processing later."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "yes_no_cols = [\"Int'l Plan\", \"VMail Plan\"]\n",
    "churn_feat_space[yes_no_cols] = churn_feat_space[yes_no_cols] == 'yes'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Convert the data frame to Numpy because Scikit-Learn uses a Numpy array to process."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "<class 'numpy.ndarray'>\n"
     ]
    }
   ],
   "source": [
    "print(type(churn_feat_space))\n",
    "X = churn_feat_space.as_matrix().astype(np.float)\n",
    "print(type(X))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is important to normalize the data in building a machine learning model. The data will be in the same range and it can improve the results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Feature space holds 3333 observations and 17 features\n",
      "Unique target labels: [0 1]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "scaler = StandardScaler()\n",
    "X = scaler.fit_transform(X)\n",
    "\n",
    "print(\"Feature space holds %d observations and %d features\" % X.shape)\n",
    "print(\"Unique target labels:\", np.unique(y))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preparing Training and Testing Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Prepare the training and testing data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "np.random.seed(0)\n",
    "\n",
    "indices = np.random.permutation(len(X))\n",
    "\n",
    "split_index = -(len(X) - int(len(X) * 0.8))\n",
    "\n",
    "X_train = X[indices[:split_index]]\n",
    "y_train = y[indices[:split_index]]\n",
    "\n",
    "X_test = X[indices[split_index:]]\n",
    "y_test = y[indices[split_index:]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Building Predictive Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Define the target for data visualization later."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "target_names = ['yes', 'no']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### k-Nearest Neighbors"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.866566716642\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        yes       0.87      0.99      0.93       562\n",
      "         no       0.79      0.21      0.33       105\n",
      "\n",
      "avg / total       0.86      0.87      0.83       667\n",
      "\n"
     ]
    }
   ],
   "source": [
    "n_neighbors = 15\n",
    "\n",
    "knn = neighbors.KNeighborsClassifier(n_neighbors=n_neighbors)\n",
    "knn.fit(X_train, y_train)\n",
    "\n",
    "y_pred = knn.predict(X_test)\n",
    "\n",
    "print(accuracy_score(y_test, y_pred))\n",
    "print(classification_report(y_test, y_pred, target_names=target_names))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Naive Bayes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.845577211394\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        yes       0.90      0.92      0.91       562\n",
      "         no       0.51      0.46      0.48       105\n",
      "\n",
      "avg / total       0.84      0.85      0.84       667\n",
      "\n"
     ]
    }
   ],
   "source": [
    "gnb = naive_bayes.GaussianNB()\n",
    "gnb.fit(X_train, y_train)\n",
    "\n",
    "y_pred = gnb.predict(X_test)\n",
    "\n",
    "print(accuracy_score(y_test, y_pred))\n",
    "print(classification_report(y_test, y_pred, target_names=target_names))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Support Vector Machines (SVMs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.845577211394\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "        yes       0.90      0.92      0.91       562\n",
      "         no       0.51      0.46      0.48       105\n",
      "\n",
      "avg / total       0.84      0.85      0.84       667\n",
      "\n"
     ]
    }
   ],
   "source": [
    "clf = svm.SVC()\n",
    "clf.fit(X_train, y_train)\n",
    "\n",
    "y_pred = gnb.predict(X_test)\n",
    "\n",
    "print(accuracy_score(y_test, y_pred))\n",
    "print(classification_report(y_test, y_pred, target_names=target_names))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [Root]",
   "language": "python",
   "name": "Python [Root]"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}