Finance_Fraud_Detection_using_Gradient_Decision_Boosted.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Title:** Fraud Detection on Credit Card records using Random Forest.   \n",
    "**Author:** German Martinez-Ayuso   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**A not very formal, neither comprehensive guide to the application of the gradient boosted decision trees to the detection of fraud in finance.** \n",
    "\n",
    "\n",
    "Inspiration article: _\"Credit Card Fraud Detection using Autoencoders in Keras | TensorFlow for Hackers (Part VII)\"_ By Venelin Valkov. [Link](https://www.curiousily.com/posts/credit-card-fraud-detection-using-autoencoders-in-keras/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction\n",
    "\n",
    "Nowadays having a credit or debit card for shopping online seems like mandatory, specially during quarantine. However everybody (at least me!) feels a bit unsecured when they input their card numbers on a website. There are many risks out there, identity fraud, theft of bank details, etc. \n",
    "Fortunately many banks are catching up with fraud detection applying machine learning techniques. In this article/gist we are going to demonstrate how we can use boosted decision trees to detect fraud in credit cards records.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Decision tree"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A decision tree is decision tool whose structure resemble a tree. Each of the nodes of this structure represents a \"test\" (or evaluation of an activation function). Using the results of these test, the classification rules are obtained which goes from the root (input) to the leaves (outputs). \n",
    "\n",
    "One example of decision tree is the Galton machine:\n",
    "\n",
    "|![Galton machine](galton_machine.gif)| \n",
    "|:------:|\n",
    "|   [Galton machine](https://en.wikipedia.org/wiki/File:Galton_box.webm) |\n",
    "|From [Wikipedia-Beam machine](https://en.wikipedia.org/wiki/Bean_machine)  |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Each of the pegs is a test where the ball will be driven to one of the sides. Since all the pegs and balls are equal, the probability of the ball going to one side or another is the same. Since it is the same, the balls will tend to be closer to the center because the probability of them going to the extremes is lower. \n",
    "\n",
    "The decision trees present several advantages. The main one is that they are easy to understand and interpret hence these models are widely applied in order to obtain a good understanding of the underlying model.\n",
    "\n",
    "\n",
    "## Decision Tree Structure\n",
    "The decision trees are composed of:\n",
    "- Nodes. Where the branches growth from. Here is where the tests are applied.\n",
    "- Branches. They represents the links between nodes.\n",
    "- Leaves. They are the final output at the end of the branches. They might not be applied any tests. \n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Gradient Boosted Decision Trees\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The XGBoost is the solution for the optimization of assemblies of decision trees. It is based on the concept that decision trees can be considered as a function which can be optimized using gradient techniques as any common function. \n",
    "This optimization would be \"measured\" using a loss function. \n",
    "This function is a measure of the error in the predicted values compared with the real values in the training set.\n",
    "\n",
    "The algorithm uses the next inputs:\n",
    "\n",
    "- Training set: $\\{(x_i, y_i)\\}^n_{i=1}$ \n",
    "- Differentiable loss function: $L(y,F(x))$ \n",
    "- Maximum number of iterations: $M$\n",
    "\n",
    "and follows the next steps (From Wikipedia):\n",
    "\n",
    "1. Initialize the model with random values.\n",
    "\n",
    "$$F_0(x) = \\arg \\min \\sum^n_{i=1} L(y_i, \\gamma)$$\n",
    "\n",
    "2. For m=1 to M:  \n",
    "\n",
    "    2.1. Compute so-called pseudo-residuals:\n",
    "    $$ r_{im} = - \\left[ \\dfrac{\\partial L(y_i,F(x_i))}{\\partial F(x_i)} \\right]_{F(x) = F_{m-1}(x)} \\hspace{3ex} \\text{for i = 1,  ..., n}$$   \n",
    "    \n",
    "    2.2. Fit a base learner (or weak learner e.g. tree) $h_m(x)$ to pseudo-residuals, i.e. train it using the training set.     \n",
    "    \n",
    "    2.3. Compute multiplier $\\gamma_m$ by solving the one-dimensional optimization problem:\n",
    "    $$ \\gamma_m = \\arg \\min \\sum^n_{i=1} L(y_i, F_{m-1}(x_i) + \\gamma h_m(x_i))$$   \n",
    "    \n",
    "    2.4. Update the model:\n",
    "    $$ F_m(x) = F_{m-1}(x) + \\gamma_m h_m(x)$$\n",
    "\n",
    "3. Output $F_M(x)$.\n",
    "\n",
    "This is as it is explained in Wikipedia. However in a more simplistic way, this procedure could be explained as:\n",
    "\n",
    "1. Initialize the model with random values ($x_0$).\n",
    "2. Start iterations until maximum number of iterations is reached. \n",
    "\n",
    "    2.1. Calculate the increments in the loss function for the values ($x_m$).  \n",
    "    2.2. Train the decision trees ($h_m(x)$) to fit the increments.  \n",
    "    2.3. Obtain $\\gamma$ from the optimization problem given the trained trees.  \n",
    "    2.4. Update the model ($F_m(x)$).   \n",
    "    \n",
    "It should be noticed that the decision trees are trained on the residual, hence their results $h_m(x)$ are added to the solution in the previous iteration $F_{m-1}(x)$ multiplied by $\\gamma_m$. \n",
    "\n",
    "\n",
    "## Avoiding overfitting.\n",
    "\n",
    "Overfitting is the what happens when the model tends to replicate very accurate the training set but it does not perform well on new data. \n",
    "This is considered as a reduction in the generality of the model and it is addressed by using regularization techniques:\n",
    "\n",
    "### Shrinkage\n",
    "This consists in adding another multiplier to the $\\gamma_m$ coefficient:  \n",
    "\n",
    "$$ F_m(x) = F_{m-1}(x) + \\nu \\gamma_m h_m(x)  \\hspace{3ex} 0 < \\nu \\leq 1$$   \n",
    "\n",
    "where $\\nu$ is called \"learning rate\". \n",
    "\n",
    "This improves substantially the performance the model. Empirically it is demonstrated that values of $\\nu$ lower than 0.1 are the best. \n",
    "\n",
    "\n",
    "### Stochastic gradient boosting\n",
    "\n",
    "Stochastic gradient boosting take the idea from the bootstrap aggregation (\"bagging\") method. \n",
    "The idea consists on train each of the trees ($h_m(x)$) in a subset of the training set randomly generated without replacement.\n",
    "It has been observed that this improve substantially the model accuracy. \n",
    "The subsample size is represented as a fraction $f$ of the total set. If $f=1$ this improvement is not applied. Practically it has been observed that for moderate or small size dataset, the optimal $f$ values are between 0.5 and 0.8.\n",
    "Since the subsets are smaller, there is generally an improvement in the training speed. \n",
    "\n",
    "This approach might leave out some observations when building the learners (trees). These observation can be used as a validation inside the scheme hence it is not needed a validation set. The error between these leave-out observations and the model predictions is called \"out-of-bag error. \n",
    "\n",
    "\n",
    "### Number of observation in leaves.\n",
    "Ensuring that there is, at least, a minimum number of observations in each of the leaves of the trees is another way to help to regularize the model. \n",
    "\n",
    "\n",
    "### Penalize tree complexity\n",
    "The complexity of the tree is measured as the number of leaves. Using a suitable pruning approach it might help to reduce the number of leaves by removing leaves or branches that do not contribute to reduce the loss by a threshold. \n",
    "\n",
    "\n",
    "--  \n",
    "**REFERENCES:**   \n",
    "\n",
    "1. [Wikipedia-Decision Tree](https://en.wikipedia.org/wiki/Decision_tree)   \n",
    "2. [Wikipedia-Gradient Boosting](https://en.wikipedia.org/wiki/Gradient_boosting)   \n",
    "3. [Wikipedia-Receiver operating characteristics](https://en.wikipedia.org/wiki/Receiver_operating_characteristic)   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Application of XGBoost to Finance Fraud detection\n",
    "\n",
    "## Dataset: Credit card transfers."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Importing libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pprint import pprint\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Reading the data "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Index(['Time', 'V1', 'V2', 'V3', 'V4', 'V5', 'V6', 'V7', 'V8', 'V9', 'V10',\n",
      "       'V11', 'V12', 'V13', 'V14', 'V15', 'V16', 'V17', 'V18', 'V19', 'V20',\n",
      "       'V21', 'V22', 'V23', 'V24', 'V25', 'V26', 'V27', 'V28', 'Amount',\n",
      "       'Class'],\n",
      "      dtype='object')\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Time</th>\n",
       "      <th>V1</th>\n",
       "      <th>V2</th>\n",
       "      <th>V3</th>\n",
       "      <th>V4</th>\n",
       "      <th>V5</th>\n",
       "      <th>V6</th>\n",
       "      <th>V7</th>\n",
       "      <th>V8</th>\n",
       "      <th>V9</th>\n",
       "      <th>...</th>\n",
       "      <th>V21</th>\n",
       "      <th>V22</th>\n",
       "      <th>V23</th>\n",
       "      <th>V24</th>\n",
       "      <th>V25</th>\n",
       "      <th>V26</th>\n",
       "      <th>V27</th>\n",
       "      <th>V28</th>\n",
       "      <th>Amount</th>\n",
       "      <th>Class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.0</td>\n",
       "      <td>-1.359807</td>\n",
       "      <td>-0.072781</td>\n",
       "      <td>2.536347</td>\n",
       "      <td>1.378155</td>\n",
       "      <td>-0.338321</td>\n",
       "      <td>0.462388</td>\n",
       "      <td>0.239599</td>\n",
       "      <td>0.098698</td>\n",
       "      <td>0.363787</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.018307</td>\n",
       "      <td>0.277838</td>\n",
       "      <td>-0.110474</td>\n",
       "      <td>0.066928</td>\n",
       "      <td>0.128539</td>\n",
       "      <td>-0.189115</td>\n",
       "      <td>0.133558</td>\n",
       "      <td>-0.021053</td>\n",
       "      <td>149.62</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.0</td>\n",
       "      <td>1.191857</td>\n",
       "      <td>0.266151</td>\n",
       "      <td>0.166480</td>\n",
       "      <td>0.448154</td>\n",
       "      <td>0.060018</td>\n",
       "      <td>-0.082361</td>\n",
       "      <td>-0.078803</td>\n",
       "      <td>0.085102</td>\n",
       "      <td>-0.255425</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.225775</td>\n",
       "      <td>-0.638672</td>\n",
       "      <td>0.101288</td>\n",
       "      <td>-0.339846</td>\n",
       "      <td>0.167170</td>\n",
       "      <td>0.125895</td>\n",
       "      <td>-0.008983</td>\n",
       "      <td>0.014724</td>\n",
       "      <td>2.69</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.0</td>\n",
       "      <td>-1.358354</td>\n",
       "      <td>-1.340163</td>\n",
       "      <td>1.773209</td>\n",
       "      <td>0.379780</td>\n",
       "      <td>-0.503198</td>\n",
       "      <td>1.800499</td>\n",
       "      <td>0.791461</td>\n",
       "      <td>0.247676</td>\n",
       "      <td>-1.514654</td>\n",
       "      <td>...</td>\n",
       "      <td>0.247998</td>\n",
       "      <td>0.771679</td>\n",
       "      <td>0.909412</td>\n",
       "      <td>-0.689281</td>\n",
       "      <td>-0.327642</td>\n",
       "      <td>-0.139097</td>\n",
       "      <td>-0.055353</td>\n",
       "      <td>-0.059752</td>\n",
       "      <td>378.66</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1.0</td>\n",
       "      <td>-0.966272</td>\n",
       "      <td>-0.185226</td>\n",
       "      <td>1.792993</td>\n",
       "      <td>-0.863291</td>\n",
       "      <td>-0.010309</td>\n",
       "      <td>1.247203</td>\n",
       "      <td>0.237609</td>\n",
       "      <td>0.377436</td>\n",
       "      <td>-1.387024</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.108300</td>\n",
       "      <td>0.005274</td>\n",
       "      <td>-0.190321</td>\n",
       "      <td>-1.175575</td>\n",
       "      <td>0.647376</td>\n",
       "      <td>-0.221929</td>\n",
       "      <td>0.062723</td>\n",
       "      <td>0.061458</td>\n",
       "      <td>123.50</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2.0</td>\n",
       "      <td>-1.158233</td>\n",
       "      <td>0.877737</td>\n",
       "      <td>1.548718</td>\n",
       "      <td>0.403034</td>\n",
       "      <td>-0.407193</td>\n",
       "      <td>0.095921</td>\n",
       "      <td>0.592941</td>\n",
       "      <td>-0.270533</td>\n",
       "      <td>0.817739</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.009431</td>\n",
       "      <td>0.798278</td>\n",
       "      <td>-0.137458</td>\n",
       "      <td>0.141267</td>\n",
       "      <td>-0.206010</td>\n",
       "      <td>0.502292</td>\n",
       "      <td>0.219422</td>\n",
       "      <td>0.215153</td>\n",
       "      <td>69.99</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 31 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   Time        V1        V2        V3        V4        V5        V6        V7  \\\n",
       "0   0.0 -1.359807 -0.072781  2.536347  1.378155 -0.338321  0.462388  0.239599   \n",
       "1   0.0  1.191857  0.266151  0.166480  0.448154  0.060018 -0.082361 -0.078803   \n",
       "2   1.0 -1.358354 -1.340163  1.773209  0.379780 -0.503198  1.800499  0.791461   \n",
       "3   1.0 -0.966272 -0.185226  1.792993 -0.863291 -0.010309  1.247203  0.237609   \n",
       "4   2.0 -1.158233  0.877737  1.548718  0.403034 -0.407193  0.095921  0.592941   \n",
       "\n",
       "         V8        V9  ...       V21       V22       V23       V24       V25  \\\n",
       "0  0.098698  0.363787  ... -0.018307  0.277838 -0.110474  0.066928  0.128539   \n",
       "1  0.085102 -0.255425  ... -0.225775 -0.638672  0.101288 -0.339846  0.167170   \n",
       "2  0.247676 -1.514654  ...  0.247998  0.771679  0.909412 -0.689281 -0.327642   \n",
       "3  0.377436 -1.387024  ... -0.108300  0.005274 -0.190321 -1.175575  0.647376   \n",
       "4 -0.270533  0.817739  ... -0.009431  0.798278 -0.137458  0.141267 -0.206010   \n",
       "\n",
       "        V26       V27       V28  Amount  Class  \n",
       "0 -0.189115  0.133558 -0.021053  149.62      0  \n",
       "1  0.125895 -0.008983  0.014724    2.69      0  \n",
       "2 -0.139097 -0.055353 -0.059752  378.66      0  \n",
       "3 -0.221929  0.062723  0.061458  123.50      0  \n",
       "4  0.502292  0.219422  0.215153   69.99      0  \n",
       "\n",
       "[5 rows x 31 columns]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv(\"creditcard.csv\")\n",
    "\n",
    "print(df.columns)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The variables are not defined, then we are going to use a black-input approach. We won't do an exploratory analysis. \n",
    "\n",
    "Since decision trees do not required normalization of the data [1](https://datascience.stackexchange.com/questions/60950/is-it-necessary-to-normalise-data-for-xgboost), the XGBoost does not required either. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model application"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Splitting the dataset\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(\n",
    "                                    df.drop('Class',axis=1),\n",
    "                                    df['Class'],\n",
    "                                    test_size=0.3,\n",
    "                                    train_size=0.7,\n",
    "                                    random_state=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import roc_auc_score as measure_roc_auc\n",
    "from xgboost.sklearn import XGBClassifier\n",
    "from sklearn.model_selection import RandomizedSearchCV"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will use a random search algorithm which will find the optimal values for the model parameters. \n",
    "Other option is to use a grid search, however this grid search is frequently more time-consuming than the random search. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 10 folds for each of 20 candidates, totalling 200 fits\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=2)]: Using backend LokyBackend with 2 concurrent workers.\n",
      "[Parallel(n_jobs=2)]: Done  37 tasks      | elapsed: 31.7min\n",
      "[Parallel(n_jobs=2)]: Done 158 tasks      | elapsed: 155.5min\n",
      "[Parallel(n_jobs=2)]: Done 200 out of 200 | elapsed: 195.1min finished\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "RandomizedSearchCV(cv=10, error_score=nan,\n",
       "                   estimator=XGBClassifier(base_score=None, booster=None,\n",
       "                                           colsample_bylevel=None,\n",
       "                                           colsample_bynode=None,\n",
       "                                           colsample_bytree=None, gamma=None,\n",
       "                                           gpu_id=None, importance_type='gain',\n",
       "                                           interaction_constraints=None,\n",
       "                                           learning_rate=None, max_bin=16,\n",
       "                                           max_delta_step=None, max_depth=None,\n",
       "                                           min_child_weight=None, missing=nan,\n",
       "                                           monotone_constr...\n",
       "                                           validate_parameters=False,\n",
       "                                           verbosity=None),\n",
       "                   iid='deprecated', n_iter=20, n_jobs=2,\n",
       "                   param_distributions={'colsample_bytree': [0.1, 0.3, 0.5],\n",
       "                                        'learning_rate': [0.01, 0.033, 0.063,\n",
       "                                                          0.1],\n",
       "                                        'max_depth': [3, 5],\n",
       "                                        'n_estimators': [100, 200, 500],\n",
       "                                        'subsample': [0.1, 0.3, 0.5]},\n",
       "                   pre_dispatch='2*n_jobs', random_state=None, refit=True,\n",
       "                   return_train_score=False, scoring='roc_auc', verbose=2)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xgb = XGBClassifier(max_bin=16)\n",
    "\n",
    "# Space search for the parameters.\n",
    "prams={\n",
    "    'learning_rate':[0.01,0.033,0.063,0.1], #Step size shrinkage used to prevent overfitting. Range is [0,1]\n",
    "    'n_estimators':[100,200,500],        #Number of trees you want to build.\n",
    "    'max_depth':[3,5],                          #The maximum depth of a tree\n",
    "    'colsample_bytree':[0.1,0.3,0.5],            #Denotes the fraction of columns to be randomly samples for each tree.\n",
    "    'subsample':[0.1,0.3,0.5]                    #Denotes the fraction of observations to be randomly samples for each tree.\n",
    "}\n",
    "\n",
    "# prams={\n",
    "#     'learning_rate':[0.05], #Step size shrinkage used to prevent overfitting. Range is [0,1]\n",
    "#     'n_estimators':[100],        #Number of trees you want to build.\n",
    "#     'max_depth':[3],                          #The maximum depth of a tree\n",
    "#     'colsample_bytree':[0.1],            #Denotes the fraction of columns to be randomly samples for each tree.\n",
    "#     'subsample':[0.1]                    #Denotes the fraction of observations to be randomly samples for each tree.\n",
    "# }\n",
    "\n",
    "\n",
    "#RandomizedSearchCV search for the optimal set of hyperparameters\n",
    "opt_xgb=RandomizedSearchCV(xgb,\n",
    "                              param_distributions=prams,\n",
    "                              verbose=2,\n",
    "                              n_iter=20,\n",
    "                              cv=10,\n",
    "                              scoring='roc_auc',\n",
    "                              n_jobs=2 # To speed up calculations\n",
    "                             );\n",
    "\n",
    "opt_xgb.fit(X_train, y_train) #Training"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's save the model for later sessions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stored 'opt_xgb' (RandomizedSearchCV)\n",
      "Stored 'df' (DataFrame)\n"
     ]
    }
   ],
   "source": [
    "# Saving\n",
    "%store opt_xgb df \n",
    "pickle.dump(opt_xgb, open(\"model.dat\", \"wb\"))\n",
    "\n",
    "#load \n",
    "# %store -r opt_xgb df \n",
    "# loaded_model = pickle.load(open(\"model.dat\", \"rb\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Printing model best parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'colsample_bytree': 0.1,\n",
      " 'learning_rate': 0.033,\n",
      " 'max_depth': 3,\n",
      " 'n_estimators': 500,\n",
      " 'subsample': 0.3}\n",
      "\n",
      "The AUC score is 0.9814628856726895\n"
     ]
    }
   ],
   "source": [
    "pprint(opt_xgb.best_params_)\n",
    "print(\"\\nThe AUC score is {}\".format(opt_xgb.best_score_))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Predicting on the test values. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The test AUC for training data is: 0.982795\n"
     ]
    }
   ],
   "source": [
    "predict_y = opt_xgb.predict_proba(X_test)\n",
    "accur_training = measure_roc_auc(y_test, predict_y[:, 1])\n",
    "\n",
    "print(\"The test AUC for training data is: {:0.6f}\".format(accur_training))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The AUC score is 0.962289 on the test samples. The AUC score is the area under the curve ROC which is the relation between the true positive rate (TPR) against the false positive rate (FPR) at various threshold settings. \n",
    "The higher the values of the true positive, the better performs the algorithm. \n",
    "\n",
    "The area under the curve ROC (AUC-ROC) is used as a metric for many algorithms. Other metrics can be the F-1 score, the mean-squared-error (MSE), etc. \n",
    "\n",
    "\n",
    "| ![Example of ROC curve-From wikipedia](https://upload.wikimedia.org/wikipedia/commons/6/6b/Roccurves.png)| \n",
    "|:------:|\n",
    "|   ROC Curve  |\n",
    "|[ROC Curve-From Wikipedia](https://upload.wikimedia.org/wikipedia/commons/6/6b/Roccurves.png) |\n",
    "\n",
    "\n",
    "Let's plot our own ROC curve:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import roc_curve, auc "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEWCAYAAAB42tAoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOzdd3gV1dbA4d9KAkmAQAgRVHoHQboIooCFImDXCxawcK8i2EARvKAixYIKiiBgQS76KSqKIlIERFEEKdKkVyGIdEIoCSnr+2OGcAgpJ5CTk5Os93ny5MzMnpk1c8qa2Xtmj6gqxhhjTEaC/B2AMcaYvM0ShTHGmExZojDGGJMpSxTGGGMyZYnCGGNMpixRGGOMyZQlinxARO4VkR/8HYe/iUgFETkmIsG5uM5KIqIiEpJb6/QlEVkrIq3PY758+xkUkdYiEuPvOPzJEkUOE5EdInLS/cH6R0QmikgxX65TVf9PVdv6ch15kbuvbzg9rKo7VbWYqib7My5/cRNWtQtZhqrWUdWfsljPOcmxoH4GCwpLFL5xk6oWAxoADYHn/BzPefHnUXJ+OULPDtvfJq+yROFDqvoPMBsnYQAgIqEi8oaI7BSRvSIyTkTCPabfIiIrReSoiGwVkfbu+BIi8qGI7BGR3SIy9HQVi4g8ICK/uq/HicgbnnGIyLci0sd9famIfCUi+0Vku4g84VFukIhMEZFPROQo8EDabXLjmOTO/5eIDBSRII84ForIOyISKyIbROT6NPNmtg0LRWSkiBwCBolIVRH5UUQOisgBEfk/EYl0y38MVAC+c8/enk17pCsiP4nIEHe5cSLyg4hEe8TTzd2GgyLyfNozlDTbHS4ib7rlY0XkV8/3DbjXfU8PiMgAj/maisgiETnibvdoESnsMV1FpJeIbAY2u+PeFpFd7mdguYhc41E+WET+63424tzp5UVkgVtklbs/OrvlO7mfpyMi8puI1PNY1g4R6Sciq4HjIhLiuQ/c2Je5cewVkRHurKfXdcRdV3PPz6A7bx0RmSMih9x5/5vBfs3w++DGttjj/XxUnKqxMHf4S3HO2mNFZIGI1PFY7kQReVdEZroxLhSRi0XkLRE57H42G6bZF8+JyDp3+ken15NOzBl+h/ItVbW/HPwDdgA3uK/LAWuAtz2mvwVMA6KACOA74BV3WlMgFmiDk8TLArXcad8A44GiQGlgCfCIO+0B4Ff3dUtgFyDucEngJHCpu8zlwAtAYaAKsA1o55YdBCQCt7plw9PZvknAt27slYBNQHePOJKA3kAhoLO7PVFebkMS8DgQAoQD1dx9EQpchPMD9VZ6+9odrgQoEOIO/wRsBWq4y/sJeNWddhlwDLja3RdvuNt+Qwbv6xh3/rJAMHCVG9fpdb7vrqM+kADUdudrDDRzt6kSsB54ymO5CszB+TyEu+PuA0q58zwN/AOEudP64nymagLirq+Ux7KqeSy7EbAPuNKN+X53n4V67L+VQHmPdafuU2AR0NV9XQxolt5+TuczGAHscWMPc4evzGC/ZvZ9CHLf80FAdeAw0NBj3ofceULd5az0mDYROODu/zDgR2A70M3dF0OB+Wk+S3+6+yIKWAgMdae1BmI8YsrwO5Rf//weQH77cz9wx4A498s0D4h0pwlwHKjqUb45sN19PR4Ymc4yy+D8+IR7jLv79Ac9zZdUgJ1AS3f4P8CP7usrgZ1plv0c8JH7ehCwIJNtC3bjuMxj3CPATx5x/I2bpNxxS4CuXm7DzozW7Za5FViRZl9nlSgGekzvCcxyX78AfOYxrQhwinQShfvjcBKon8600+ssl2abu2SwDU8BUz2GFbgui+0+fHrdwEbglgzKpU0UY4EhacpsBFp57L+H0vn8nk4UC4CXgOgMtjmjRHG35/uUyXZl+n3wWNchnAT7XCbLinRjKuEOTwTe95j+OLDeY/hy4Eia7e7hMdwB2Oq+bs2ZRJHpdyi//lm9pG/cqqpzRaQV8CkQDRzBOSouAiwXkdNlBecHGJyjmRnpLK8izhH6Ho/5gnDOHM6iqioik3G+rAuAe4BPPJZzqYgc8ZglGPjFY/icZXqIxjmK+stj3F84R9mn7Vb32+Mx/VIvt+GsdYtIaWAUcA3OkWMQzo9mdvzj8foEzpExbkyp61PVEyJyMINlROMclW7N7npEpAYwAmiC896H4ByRekq73U8D/3ZjVKC4GwM4n5HM4vBUEbhfRB73GFfYXW66606jOzAY2CAi24GXVHW6F+v1Nsasvg+o6g4RmY/zwz0mtZBTZTkMuMtdToo7KRrnLBZgr8e6TqYznPYiE899cfpzm5Y336F8x9oofEhVf8Y5sjndZnAA5wNaR1Uj3b8S6jR8g/NBrZrOonbhHI1He8xXXFXrpFMW4DPgThGpiHME9JXHcrZ7LCNSVSNUtYNn2Jls0gGc6pmKHuMqALs9hsuKx7fenf63l9uQdt2vuOPqqWpxnCoZyaR8duzBqRoEnDYInOqe9BwA4kn/vcnKWGADUN3dhv9y9jaAx3a47RH9gH8BJVU1EueH7/Q8GX1G0rMLGJbm/S6iqp+lt+60VHWzqt6NU034GjBFRIpmNk82Y8zq+4CIdMA5y5gHvO4x7z3ALcANQAmcMw84d99mR3mP16c/t2l58x3KdyxR+N5bQBsRaaCqKTh12SPdo2VEpKyItHPLfgg8KCLXi0iQO62Wqu4BfgDeFJHi7rSq7hnLOVR1BbAf+ACYraqnj36WAEfdRsJwt2G0rohc4c2GqHPZ6RfAMBGJcBNRH86csYDzo/KEiBQSkbuA2sCM7G6DKwKnGu+IiJTFqZ/3tBenjvh8TAFuEpGrxGlcfokMfmTc920CMMJtyAx2G3BDvVhPBHAUOCYitYBHvSifhPP+hYjICzhnFKd9AAwRkeriqCcipxNc2v3xPtBDRK50yxYVkY4iEuFF3IjIfSJykbv9pz9DyW5sKWS876cDF4vIU25jdYSIXJm2UFbfB3EuPPgQ5+zqfpz36/QPcgTOgcdBnLOSl73Zpiz0EpFyIhKFk9A/T6fMBX2HApUlCh9T1f04DcDPu6P6AVuAxeJcWTQXp2ESVV0CPAiMxDmK/JkzR+/dcKoN1uFUv0wBLslk1Z/hHG196hFLMnATzlVY23GO6D7AOSLz1uM49crbgF/d5U/wmP47TsPjAZyqgTtV9XSVTna34SWcBtlY4Hvg6zTTXwEGinNFzzPZ2AZUda27LZNxzi7icBp+EzKY5RmcRuSlOHXmr+Hd9+cZnKPfOJwfxfR+fDzNBmbiXCTwF86ZjGeVyAicZP0DTgL6EKcRHZw2pv+5++NfqroMp41qNM7+3kI6V7Jloj2wVkSOAW/jtLvEq+oJnPd2obuuZp4zqWoczkUIN+FUyW0Grs1gHRl+H4D3gG9VdYb7GeoOfOAmxknu/tmN83lanI3tysinOPt1m/s3NG2BHPoOBZzTV8YYc8FE5AHg36p6tb9jyS5xboo8glNFtN3f8ZjcJSI7cD67c/0dS15kZxSmwBKRm0SkiFvv/gbOGcMO/0ZlTN5jicIUZLfgNFj+jVNd1kXtFNuYc1jVkzHGmEzZGYUxxphMBdwNd9HR0VqpUiV/h2GMMQFl+fLlB1T1ovOZN+ASRaVKlVi2bJm/wzDGmIAiIn9lXSp9VvVkjDEmU5YojDHGZMoShTHGmExZojDGGJMpSxTGGGMyZYnCGGNMpnyWKERkgojsE5E/M5guIjJKRLaIyGoRaeSrWIwxxpw/X95HMRGne+NJGUy/Ead/neo4D9cZ6/43xhj/OnkQ/lni7yhyzKlTKVkXyoTPEoWqLhCRSpkUuQWY5HbCtlhEIkXkEvcBN8YY4z9ftYO9aZ9YG5j6fteGFX9n9tiXrPnzzuyynP1Alhh33DmJQkQeBh4GqFChQq4Ely+sGAObvvR3FMYEnv2rnf/lW0NwmF9DuVB1L49m1MJKF7QMfyaK9B47mW5Xtqr6Hs7TrmjSpEnB7u72VBwcWOtd2d+eh/jDvo3HmPyqcATc9j0UKuLvSLJl3br9/PHHHu67rx4A3W5XWj0dS+XKg897mf5MFDGc/TDzcqT/MHPj6bOr4EC61wdk7NbvoHCxrMsZY84oWSOgksSJE4kMHbqA11//jeBgoVmzclSrFoWIUKlS5AUt25+JYhrwmIhMxmnEji1w7RN7V8CSVyHllPfzHFzn/C/TBIKCsy5/SXOo2un84jPGBISZMzfTq9cMtm8/AkD37o0pVSo8i7m857NEISKfAa2BaBGJAV4ECgGo6jhgBtAB58HqJ4AHfRVLjjv6FyQlXPhylrwMm6Zkf77QSLj7NwgudOExGGMC1u7dR3nqqdlMmeIcQNarV4Zx4zrSvHn5LObMHl9e9XR3FtMV6OWr9fvMktfgl/45u8yGTziNZt66qJ4lCWMMvXrN4NtvN1KkSCEGD27Nk082IyQk52+PC7jnUfjd4iHO/yJlILT4hS8v/CK4oi9ElLvwZRlj8r2kpJTUZPDaazdQqFAwb77ZlgoVSvhsnZYoskNTIPG487rdh1Clo3/jMcYUGLGx8Qwc+CObNh1i1qx7ERFq1ozmyy/v8vm6LVFkx+etzryu2NZ/cRhjCgxV5csv1/HUU7PYs+cYwcHCypX/0LDhhd1Elx2WKLJybA+sGguJJ2D3Qmdczc7WRmCM8bmtWw/x2GMzmTVrCwDNm5dj3LhO1KtXJlfjsESRkZRkQGHlaPj95TPjCxWDDv/nt7CMMQXDG2/8xvPPzyc+PonIyDBee+0G/v3vRgQFpXevsm9ZokjPH6Pgp95Om8RpVTpBuVZwSTPv7l8wxpgLcOJEIvHxSXTtWo833mhL6dJF/RaLJYr0LBx4JklIsHPfQrPn4ZKm/o3LGJNv7d9/nI0bD3L11U5/dv36taB160q0bFnRz5FZojhXUrzTnxLAnXOh4vX+jccYk6+lpCgTJqzg2WfnEBISxIYNjxEVFU5oaEieSBJgicLxxyj47UXQZFCPPgfLt8p4HmOMuUB//rmPHj2ms3Ch05F2mzZVOHEikaionOt+IydYolCF+U+eO75yBwiy3WOMyXnHj59i8OCfGTFiMUlJKZQpU5S33mpP5851EMn9xuqs2C/hnt/PvL5zLlx8hfO6cIR/4jHG5Ht33vkls2ZtQQR69mzCsGHXExmZd597UXATxaGNML0zHPPo2bx8a7uiyRjjc/36tWDv3mOMHduRK6/M+933FKxEkZIMO+c6D/PZ8QPsX3VmWtPnLEkYY3JcUlIK77zzOzt2HOHtt28EoHXrSixb9rBf7ok4HwUrUWz+yjmL8FS3O1z1EkSU9U9Mxph8a8mS3TzyyHRWrvwHgIcfbkydOqUBAiZJQH5OFCcPwYx74fg/HuP2O/8jq0GZxhAS7vbcaknCGJNzjhyJ57//nce4cctQhYoVSzB6dIfUJBFo8k+i2L8Gjnu0N/y9CHbMSr9sw8egUTpXOhljzAWaPPlPnnpqFnv3HickJIinn27O88+3pGjRwv4O7bzlj0SxdwV80ij9aRVugJbDzwyHhENUzdyJyxhT4Pzww1b27j1OixblGTu2I5dfnrsd+PlCYCeKpASY+wjs/ePMOM/uv4MLwRX9oEzD3I/NGFMgJCQksXt3HFWqlARg+PA2XHNNBe6/v0FAtUNkJrATxT9LYe3/zgzX7wE3jPVfPMaYAuXHH7fz6KPfExQkrFrVg8KFg4mOLsKDD+avg9Ocf7hqbjqw+szrzj/DtW/7LxZjTIGxd+8xunadyvXXT2LTpoMAxMQc9XNUvhPYZxTL3nT+R1aFci39G4sxJt9LSVHef385/fvP48iReMLCQhg48Br69m1B4cL59z6swE0UR3dB7DbndY1/+TcWY0yBcNttnzNt2kYA2rWrypgxHahaNcrPUfle4FY9TW5x5nWz5/0XhzGmwLj99lpcfHExPv/8TmbOvLdAJAkI1DOK+MMQ53TLS7OBUChvdclrjMkfpk3bSEzMUXr2dDoL7datPrffXpuIiFA/R5a7AjNRbPjszOvmL/ovDmNMvrRzZyxPPDGTb7/dSGhoMO3bV6NKlZKISIFLEhCIiSLlFMzr5by+pLk9M8IYk2MSE5MZNep3XnzxJ44fTyQiojBDh15HxYol/B2aXwXer+zxfWdeN7e2CWNMzli8OIZHHpnO6tV7AbjrrssYObIdZcsW93Nk/hd4ieKE8yZS7VaofKN/YzHG5BvPPz+f1av3UrlyJKNHd6BDh+r+DinPCLxEcZpn/03GGJNNqkpc3CmKF3faHEaPvpFJk1YxYEBLihQp5Ofo8pbAvTw2spq/IzDGBKiNGw9www0fc/vtn6OqANSsGc2wYddbkkhHYJ5RFCsHefAB5MaYvC0+PolXXvmFV19dyKlTyZQqFc6OHUeoXLmkv0PL0wIzUdR/xN8RGGMCzJw5W+nZcwZbthwC4KGHGjB8eBtKlSri58jyPp9WPYlIexHZKCJbRKR/OtMriMh8EVkhIqtFpINXC76keY7HaozJn1SVhx76lrZtP2HLlkNcdtlFLFjwAB9+eIslCS/57IxCRIKBMUAbIAZYKiLTVHWdR7GBwBeqOlZELgNmAJWyXHiw1SEaY7wjIlSqFEl4eAgvvNCKPn2a5+sO/HzBl1VPTYEtqroNQEQmA7cAnolCgdMXKZcA/sYbEpg1ZsaY3LFy5T/s2RPHjTc6l7j269eCrl3rWVvEefJl1VNZYJfHcIw7ztMg4D4RicE5m3g8vQWJyMMiskxElgF2RmGMSVdcXAJ9+symceP3uP/+bzh06CQAoaEhliQugC8TRXqXJWma4buBiapaDugAfCwi58Skqu+pahNVbeKOyOlYjTEBTFWZOnU9l132LiNHLgbgnnsup1ChwL0DIC/xZR1ODFDeY7gc51YtdQfaA6jqIhEJA6KBfWQmtGD3u2KMOeOvv47w2GMzmT59EwBNmlzK+PGdaNToEj9Hln/4Mt0uBaqLSGURKQx0AaalKbMTuB5ARGoDYcD+rBdt91AYY5wziTvu+ILp0zdRvHgoo0ffyOLF3S1J5DCfnVGoapKIPAbMBoKBCaq6VkQGA8tUdRrwNPC+iPTGqZZ6QNXqlYwxmUtJUYKCBBHhjTfaMm7cMkaObMcll0T4O7R8SQLtd7lJedFlqzZCVA1/h2KMyWUHD56gf/+5ALz//s1+jiawiMjy1HbebLKWHmNMnqeq/O9/K6lVawwffLCCSZNWExNz1N9hFRh2Q4IxJk9bv34/jz76PT///BcArVtXYuzYjpQrZ8+JyC2WKIwxeZKq8sIL83nttYUkJqYQHV2EN99sS9eu9RDrFDRXWaIwxuRJIsLu3XEkJqbwn/804tVXbyAqKtzfYRVIgdmYvXoTlLSnTxmT3/z9dxwHDpygXr0yABw4cIKNGw/QokUFP0cW+Kwx2xgT0JKTUxg9egm1a4+hS5cpnDqVDEB0dBFLEnmAVT0ZY/zqjz/28Mgj01m2zOm4oWXLihw9mkB0tHUBnld4lSjcO6srqOoWH8djjCkgjh5N4Pnnf2T06KWkpCjlyhVn1Kj23HprLWuszmOyTBQi0hEYARQGKotIA+BFVb3N18EZY/InVaVly49YtWovwcFCnz7NGDSoNRERof4OzaTDmzaKwcCVwBEAVV0JVPNlUMaY/E1E6N27GU2blmXZsod58812liTyMG+qnhJV9UiaU8HAulTKGONXp04lM2LEIoKDhb59WwDQrVt97ruvHsHBdk1NXudNolgvIv8CgkSkMvAksNi3YRlj8otffvmLHj2+Z926/YSGBtOtW33KlCmGiBAcbG0RgcCbVP4Y0BhIAb4G4nGShTHGZOjAgRM89NC3tGw5kXXr9lO9ehTTp99DmTLF/B2aySZvzijaqWo/oN/pESJyO07S8BM7CjEmr1JVJk5cSd++czh48CSFCwfz3HNX07//1YSF2RX5gcibM4qB6YwbkNOBGGPyj08+WcPBgye57rrKrF7dg0GDWluSCGAZvnMi0g7nMaVlRWSEx6TiONVQxhgDwIkTicTGxnPJJRGICO++24GlS//m3nsvt3si8oHMUvw+4E+cNom1HuPjgP6+DMoYEzhmztxMr14zqFKlJHPmdEVEqFkzmpo1o/0dmskhGSYKVV0BrBCR/1PV+FyMyRgTAHbvPspTT81mypR1AEREhHLw4EnreiMf8qbSsKyIDAMuA8JOj1RVexapMQVQcnIKY8YsZeDAH4mLO0XRooUYPPhannjiSkJC7J6I/MibRDERGAq8AdwIPIi1URhTIKWkKK1aTWThwl0A3HprLd5+uz0VKpTwc2TGl7xJ/0VUdTaAqm5V1YHAtb4NyxiTFwUFCW3bVqV8+eJ8+20Xpk7tbEmiAPDmjCJBnMsWtopID2A3UNq3YWXBrqIwJleoKl98sZaQkCDuuOMyAPr1a0GfPs0pVqywn6MzucWbRNEbKAY8AQwDSgAP+TIoY4z/bd16iJ49Z/DDD1u56KIiXHddZUqWDCc0NIRQ67+vQMkyUajq7+7LOKArgIiU82VQxhj/SUhI4vXXf2PYsF+Ij0+iZMkwhg27jhIlwrKe2eRLmSYKEbkCKAv8qqoHRKQOTlce1wGWLIzJZ376aQePPvo9GzYcAKBr13q88UZbSpcu6ufIjD9l2JgtIq8A/wfcC8wSkQHAfGAVYJfGGpPPJCen0LOnkyRq1izFjz92Y9Kk2yxJmEzPKG4B6qvqSRGJAv52hzfmTmjGGF9LSVHi45MoUqQQwcFBjB3bkQUL/uLZZ1sQGmp9MxlHZp+EeFU9CaCqh0RkgyUJY/KPNWv20qPH99SqVYoPP7wFgFatKtGqVSX/BmbynMwSRRUROd2VuACVPIZR1dt9GpkxxieOHz/F4ME/M2LEYpKSUti+/TCHD5+kZMlwf4dm8qjMEsUdaYZH+zKQ7LH7KIw5H999t5HHHpvJzp2xiEDPnk0YNux6IiPtiiaTscw6BZyXm4EYY3wnKSmFzp2n8PXX6wFo0OBixo/vRNOmZf0cmQkE1lplTAEQEhJEiRKhFCtWmCFDruWxx5paB37Ga6Kqvlu4SHvgbSAY+EBVX02nzL+AQYACq1T1nsyW2aS86LI1WyGyig8iNib/+P33GACuvNK55engwROcPJlEuXLF/RmW8RMRWa6qTc5nXq/PKEQkVFUTslE+GBgDtAFigKUiMk1V13mUqQ48B7RQ1cMi4t8+pIzJB44ciee55+YyfvxyatWKZuXKHhQuHEypUvacCHN+sjz3FJGmIrIG2OwO1xeRd7xYdlNgi6puU9VTwGScezM8/QcYo6qHAVR1X7aiN8akUlU+/XQNtWqNZty45QQHB3HzzTVJTranApgL480ZxSigE/ANgKquEhFvuhkvC+zyGI4BrkxTpgaAiCzEqZ4apKqzvFi2McbD5s0H6dlzBnPnbgOgRYvyjBvXibp17STdXDhvEkWQqv6V5gHpyV7Ml941rGkbREKA6kBrnL6jfhGRuqp65KwFiTwMPAzQ2HqYMuYsiYnJXHfdJGJijhIVFc7w4Tfw4IMNCQqyy8hNzvAmUewSkaaAuu0OjwObvJgvBijvMVwOpxuQtGUWq2oisF1ENuIkjqWehVT1PeA9cBqz7XkUxjhVTSJCoULBDBt2HfPn72D48Bu46CLrm8nkLG+uj3sU6ANUAPYCzdxxWVkKVBeRyiJSGOgCTEtT5hvcp+WJSDROVdQ270I3pmDau/cYXbtOZejQBanjunWrz0cf3WJJwviEN2cUSaraJbsLVtUkEXkMmI3T/jBBVdeKyGBgmapOc6e1FZF1ONVZfVX1YHbXZUxBkJKivP/+cvr3n8eRI/FERobx1FPNiIiwpwgZ38ryPgoR2QpsBD4HvlbVuNwILCNNyosu+3MblKjszzCMyVWrVv1Djx7fs3ixc29E+/bVGDOmA1WqlPRzZCZQ+PQ+ClWtKiJX4VQdvSQiK4HJqjr5fFZojPFeYmIyzz03j7feWkxysnLJJcV4++323HnnZYi11Zlc4tU9/Kr6m6o+ATQCjuI80MgY42MhIUGsWPEPKSnK4483Zf36Xtx1Vx1LEiZXZXlGISLFcG6U6wLUBr4FrvJxXMYUWDt3xpKcnELlyiUREcaN60hsbAJNmlzq79BMAeVNY/afwHfAcFX9xcfxGFNgJSYm8/bbv/Piiz/RvHk55szpiohQvXopf4dmCjhvEkUVVc1jfQDYabfJXxYt2kWPHt+zevVeAKKiwjlxIpGiRQv7OTJjMkkUIvKmqj4NfCUi51waZU+4M+bCHT58kv795/Lee38AULlyJGPGdODGG6v7OTJjzsjsjOJz938eerKdMflHQkISDRqMZ+fOWAoVCqJv36sYMKAlRYoU8ndoxpwlsyfcLXFf1lbVs5KFeyOdPQHPmAsQGhpC9+4NmTdvO2PHduSyyy7yd0jGpMubG+7+UNVGacatUNWGPo0sA84Nd9uhRCV/rN6Y8xYfn8Qrr/xCzZrR3HPP5YDziNLgYLHLXY3P+eSGOxHpjHNJbGUR+dpjUgRwJP25jDHpmTNnKz17zmDLlkOULl2U226rRXh4IXscqQkImbVRLAEO4vT6OsZjfBywwpdBGZNf/PPPMfr0mc1nn/0JQJ06FzFuXCfCw60dwgSOzNootgPbgbm5F44x+UNycgrjxy/nv/+dR2xsAuHhIbz4Yit6925O4cLB/g7PmGzJrOrpZ1VtJSKHOfuBQwKoqkb5PDpjAlRysvLOO0uIjU2gQ4fqjB59I5UrWwd+JjBlVvV0+nGn0bkRSLZYw5/Jg+LiEkhOViIjwyhcOJj337+JvXuPcfvtta2x2gS0DFvSPO7GLg8Eq2oy0Bx4BLCnoxjjUlW+/no9tWuP4emnZ6eOv/rqCtxxh/XyagKfN5dcfIPzGNSqwCScjgE/9WlUxgSIHTuOcPPNk7njji/YvTuOP//cT3x8kr/DMiZHeZMoUtxnWt8OvKWqjwNlfRuWMXlbYmIyr732K5ddNobp0zdRvHgoo0ffyG+/PURYmDddqBkTOLx6FKqI3AV0BW51x9m1fabAOnEikWbNPmDNmgxuKYcAACAASURBVH0AdOlSlxEj2nLJJRF+jswY3/AmUTwE9MTpZnybiFQGPvNtWMbkXUWKFKJJk0s5cSKRd9/tSNu2Vf0dkjE+lWUXHgAiEgJUcwe3qKrfKmGblBddtnYHFK/orxBMAaOqTJq0iqpVo7j66goAxMbGU7hwsN04ZwKGT5+ZLSLXAB8Du3HuobhYRLqq6sLzWaExgWT9+v08+uj3/PzzX9SuHc3KlT0oXDiYEiXC/B2aMbnGm6qnkUAHVV0HICK1cRLHeWWmnGGXGxrfOnkykWHDfmH48IUkJqZw0UVFeO65qylUyPpmMgWPN4mi8OkkAaCq60XEHrtl8q1Zs7bQq9cMtm07DMB//tOIV1+9gaiocD9HZox/eJMo/hCR8ThnEQD3Yp0Cmnzq2LFTdO06lQMHTlC3bmnGjetIixYV/B2WMX7lTaLoATwBPItT57MAeMeXQRmTm5KTU0hJUQoVCqZYscK8/XZ7YmKO0rt3MwoVsg78jMk0UYjI5UBVYKqqDs+dkIzJPcuX/80jj0znlltq8vzzrQBSHypkjHFk2DInIv/F6b7jXmCOiDyUa1EZ42NHjybw5JMzadr0A5Yv38PHH68mMTHZ32EZkydldkZxL1BPVY+LyEXADGBC7oRljG+oKlOmrOPJJ2exZ88xgoOFPn2a8dJL11o1kzEZyCxRJKjqcQBV3S8idl2gCWhxcQl07jyFmTO3AHDllWUZN64TDRpc7OfIjMnbMksUVTyelS1AVc9nZ6vq7T6NLFN2H4XJvmLFCpOQkEyJEqG8+uoNPPxwY4KC7LNkTFYySxR3pBke7ctAjPGFBQv+4pJLilG9eilEhAkTbiYsLIQyZYr5OzRjAkZmz8yel5uBGJOTDhw4wbPPzuGjj1Zy/fWVmTOnKyJCxYqR/g7NmIBjHeebfCUlRZk4cSV9+87h0KGTFC4czDXXVCA5WQkJsWomY86HTxuoRaS9iGwUkS0i0j+TcneKiIqIH/uPMoFu7dp9tG49ke7dp3Ho0Emuv74ya9Y8yosvtiYkxK7FMOZ8eX1GISKhqpqQjfLBwBigDRADLBWRaZ79RrnlInDu/P7d22Ubk1ZsbDzNmn3IsWOnKF26KCNGtOWeey6351UbkwOyPMwSkaYisgbY7A7XFxFvuvBoivPsim2qegqYDNySTrkhwHAg3vuwjXGcfp5KiRJh9OvXgh49GrNhQy/uvbeeJQljcog35+OjgE7AQQBVXQVc68V8ZYFdHsMxpHnWtog0BMqr6vTMFiQiD4vIMhFZ5sV6TQGwe/dR7rzzCz75ZHXquAEDrmHs2E6ULGm9vBqTk7xJFEGq+leacd70dZDe4Vzq4/TcG/hGAk9ntSBVfU9Vm6Q+ncmOFAuspKQU3n57MbVqjeGrr9bz4os/kZycAmBnEMb4iDdtFLtEpCmgbrvD48AmL+aLAcp7DJcD/vYYjgDqAj+5X/CLgWkicrOq2pmDOcfSpbvp0eN7/vhjDwC33lqLUaPaExxsDdXG+JI3ieJRnOqnCsBeYK47LitLgeoiUhnnMapdgHtOT1TVWCD69LCI/AQ8Y0nCpHX8+Cn69ZvLu+8uRRUqVCjBO+/cyM031/R3aMYUCFkmClXdh/Mjny2qmiQijwGzgWBggqquFZHBwDJVnZbtaE2BFBISxNy52wgKEvr0ac6LL7aiaFF7yKIxuUVOXzWSYQGR9/FoWzhNVR/2VVCZaVJedNm6XRBRzh+rN7lk69ZDREaGUapUEcCpdgoLC+Hyy8v4OTJjApOILE9t580mbyp35wLz3L+FQGnA6/spjMmOhIQkhg5dQN26Y+nXb27q+CuuKGtJwhg/8abq6XPPYRH5GJjjs4hMgfXTTzt49NHv2bDhAOBc4ZScnGKN1cb42fn09VQZqJjTgZiCa9++4/TtO4dJk1YBULNmKcaO7ci111b2c2TGGPAiUYjIYc60UQQBh4AM+20yJjsOHDhB7dpjOHToJKGhwQwYcA3PPtuC0FDrr9KYvCLTb6M4NzjUx7m8FSBFs2r9zhV2Y1V+ER1dhFtuqUlMzFHefbcj1apF+TskY0wamSYKVVURmaqqjXMrIJO/HT9+isGDf6Zjxxq0bOnUYL77bkdCQ4Ptzmpj8ihvWgmXiEgjn0di8r3vvtvIZZe9y/Dhv9Gz5/ekpDgnp2FhIZYkjMnDMjyjEJEQVU0Crgb+IyJbgeM49T6qqpY8jFd27YrlySdnMXXqBgAaNryY8eM72fOqjQkQmVU9LQEaAbfmUiwmn0lKSmHUqN954YX5HD+eSLFihRk69Fp69WpqDxIyJoBkligEQFW35lIsJp85ejSBV175lePHE7njjtq89VZ7ypUr7u+wjDHZlFmiuEhE+mQ0UVVH+CAeE+COHIknPDyE0NAQoqLCGT++E6GhwXTsWMPfoRljzlNm5//BQDGc7sDT+zMmlary6adrqFlzNMOHL0wdf/vttS1JGBPgMjuj2KOqg3MtkuywK2TylE2bDtKz5/fMm7cdgAULdqKqdiWTMflElm0UxmQkPj6J1177lZdf/pVTp5KJigrn9dfb8MADDSxJGJOPZJYors+1KEzA+eefY7Rs+RGbNx8C4IEHGvD6622Iji7i58iMMTktw0ShqodyMxATWMqUKUr58iUICQli7NiOtGpVyd8hGWN8xHpeM15JSVHef385115bmRo1SiEifPrp7ZQsGU7hwsH+Ds8Y40N215PJ0qpV/9CixQR69Pienj2/53S/kGXKFLMkYUwBYGcUJkPHjp1i0KCfeOutxSQnK5deGkGPHuf1JEVjTACzRGHS9c03G3j88ZnExBwlKEh4/PGmDB16HcWLh/o7NGNMLgvQRGGXXvrS7t1H6dJlCgkJyTRufAnjxnWiSZNL/R2WMcZPAjRRmJyWmJhMSEgQIkLZssUZNuw6ChcOpmfPK+yZ1cYUcPYLYPjtt100bvwen3yyOnXc009fxeOPX2lJwhhjiaIgO3ToJI888h0tWkxgzZp9vPvuMvLEk26NMXmKVT0VQKrKJ5+s5umnf2D//hMUKhTEs8+2YMCAa6zrDWPMOSxRFDB79x7j7ru/Yv78HQC0alWRsWM7Urv2Rf4NzBiTZ1miKGAiI8PYs+cY0dFFeOONNnTrVt/OIowxmbJEUQDMmbOVRo0uoVSpIoSGhvDll3dxySXFKFXKOvAzxmQtMBuz7QjYK3v2xHH33V/Rtu0n9Os3N3V83bqlLUkYY7xmZxT5UHJyCuPHL+e55+Zx9GgC4eEh1KxZyh4mZIw5L5Yo8pk//thDjx7TWbr0bwA6dqzO6NEdqFQp0s+RGWMClSWKfGTHjiM0bfo+yclK2bIRjBp1I7fdVsvOIowxF8SniUJE2gNvA8HAB6r6aprpfYB/A0nAfuAhVf3LlzHlZ5UqRfLggw2IiAjlpZdaExFhHfgZYy6czxqzRSQYGAPcCFwG3C0il6UptgJooqr1gCnAcF/Fkx/t2HGEm276jJ9/3pE67r33bmLEiHaWJIwxOcaXZxRNgS2qug1ARCYDtwDrThdQ1fke5RcD9/kwnnwjMTGZESMW8dJLP3PyZBIHDpxg0aLuAFbNZIzJcb68PLYssMtjOMYdl5HuwMz0JojIwyKyTESW5WB8AenXX3fSsOF4+vefx8mTSXTpUpevv/6Xv8MyxuRjvjyjSO/QNt0e50TkPqAJ0Cq96ar6HvAeQJPyogXxeRSHD5+kb985fPjhCgCqVi3Ju+92pG3bqn6OzBiT3/kyUcQA5T2GywF/py0kIjcAA4BWqprgw3gCWkqK8u23GylUKIj+/a/mueeuJjy8kL/DMsYUAL5MFEuB6iJSGdgNdAHu8SwgIg2B8UB7Vd3nw1gC0oYNB6hcOZLQ0BBKlSrC//3f7VSoUIJataL9HZoxpgDxWRuFqiYBjwGzgfXAF6q6VkQGi8jNbrHXgWLAlyKyUkSm+SqeQHLiRCIDBsyjXr2xDB++MHV827ZVLUkYY3KdT++jUNUZwIw0417weH2DL9cfiGbN2kLPnt+zffsRAA4cOOHniIwxBZ3dmZ1H/P13HE89NYsvv3SuHr788tKMG9eJq64qn8WcxhjjW5Yo8oBNmw7SpMl7xMWdokiRQgwa1IqnnmpGoULB/g7NGGMsUeQF1atHccUVZSlatBDvvHMjFStaB37GmLzDEoUfHD2awAsvzKdnzyuoUaMUIsK0aV0oWrSwv0MzxphzBGaiCNBuKlSVKVPW8eSTs9iz5xgbNhxg1iyn1xJLEsaYvCowE0UA2rbtMI89NoOZM7cA0KxZOV57zS76MsbkfZYofOzUqWTeeOM3hgxZQHx8EpGRYbz66vX85z+NCQoKzDMjY0zBYonCx3btimXw4J9JSEjm3nsv580321KmTDF/h2WMMV6zROEDhw+fJDIyDBGhatUo3n67PdWqRXH99VX8HZoxxmSbL7sZL3BSUpQJE1ZQrdo7fPLJ6tTxjzzSxJKEMSZgWaLIIWvX7qN164l07z6NQ4dOpjZaG2NMoLOqpwt04kQiQ4b8zBtvLCIpKYXSpYsycmQ77r67rr9DM8aYHBGgiSJvXC20adNB2rX7hB07jiACPXo05uWXr6dkyXB/h2aMMTkmQBNF3lCxYgnCwkKoX78M48Z1olmzcv4OyfhQYmIiMTExxMfH+zsUYzIUFhZGuXLlKFQo5x5sZokiG5KSUhg3bhl3312XUqWKEBoawqxZ91K2bHFCQqy5J7+LiYkhIiKCSpUqIQHaO4DJ31SVgwcPEhMTQ+XKlXNsufbr5qUlS3bTtOn7PP74TPr1m5s6vmLFSEsSBUR8fDylSpWyJGHyLBGhVKlSOX7Wa2cUWYiNjWfAgB95992lqEKFCiW45Zaa/g7L+IklCZPX+eIzaokiA6rK55+vpXfv2fzzzzFCQoLo06cZL7zQyjrwM8YUKFZnkoFVq/Zy991f8c8/x7jqqvL88cfDvPZaG0sSxm+Cg4Np0KABdevW5aabbuLIkSOp09auXct1111HjRo1qF69OkOGDEFVU6fPnDmTJk2aULt2bWrVqsUzzzzjj03I1IoVK/j3v//t7zAy9corr1CtWjVq1qzJ7Nmz0y3z448/0qhRI+rWrcv9999PUlISALGxsdx0003Ur1+fOnXq8NFHHwGwcuVKmjdvTp06dahXrx6ff/556rLmzZtHo0aNaNCgAVdffTVbtjj3Z40ePTp1/lyhqgH117gcqsf3qS8kJSWfNdy79yx9//3lmpyc4pP1mcCybt06v66/aNGiqa+7deumQ4cOVVXVEydOaJUqVXT27Nmqqnr8+HFt3769jh49WlVV16xZo1WqVNH169erqmpiYqKOGTMmR2NLTEy84GXceeedunLlylxdZ3asXbtW69Wrp/Hx8bpt2zatUqWKJiUlnVUmOTlZy5Urpxs3blRV1eeff14/+OADVVUdNmyYPvvss6qqum/fPi1ZsqQmJCToxo0bddOmTaqqunv3br344ov18OHDqqpavXr11M/dmDFj9P7771dV5z1u0KBBhrGm91kFlul5/u4GaNVTztfBzZ+/nZ49ZzB+fCdatqwIwIgR7XJ8PSafeNNHbRVPa9ZlgObNm7N6tdNNzKeffkqLFi1o27YtAEWKFGH06NG0bt2aXr16MXz4cAYMGECtWrUACAkJoWfPnucs89ixYzz++OMsW7YMEeHFF1/kjjvuoFixYhw7dgyAKVOmMH36dCZOnMgDDzxAVFQUK1asoEGDBkydOpWVK1cSGek8obFatWosXLiQoKAgevTowc6dOwF46623aNGixVnrjouLY/Xq1dSvXx+AJUuW8NRTT3Hy5EnCw8P56KOPqFmzJhMnTuT7778nPj6e48eP8+OPP/L666/zxRdfkJCQwG233cZLL70EwK233squXbuIj4/nySef5OGHH87WW5HWt99+S5cuXQgNDaVy5cpUq1aNJUuW0Lx589QyBw8eJDQ0lBo1agDQpk0bXnnlFbp3746IEBcXh6py7NgxoqKiCAkJSS0LcOmll1K6dGn2799PZGQkIsLRo0cB54zk0ksvTX2PK1WqxJIlS2jatOkFbZc3AjRR5Jx9+47Tt+8cJk1aBcCIEYtSE4UxeVFycjLz5s2je/fugFPt1Lhx47PKVK1alWPHjnH06FH+/PNPnn766SyXO2TIEEqUKMGaNWsAOHz4cJbzbNq0iblz5xIcHExKSgpTp07lwQcf5Pfff6dSpUqUKVOGe+65h969e3P11Vezc+dO2rVrx/r1689azrJly6hb90xvBrVq1WLBggWEhIQwd+5c/vvf//LVV18BsGjRIlavXk1UVBQ//PADmzdvZsmSJagqN998MwsWLKBly5ZMmDCBqKgoTp48yRVXXMEdd9xBqVKlzlpv7969mT9//jnb1aVLF/r373/WuN27d9OsWbPU4XLlyrF79+6zykRHR5OYmMiyZcto0qQJU6ZMYdeuXQA89thj3HzzzVx66aXExcXx+eefExR0du3/kiVLOHXqFFWrVgXggw8+oEOHDoSHh1O8eHEWL16cWrZJkyb88ssvlih8KSVF+fDDP+jXby6HD8cTGhrMwIEt6dv3Kn+HZgKBl0f+OenkyZM0aNCAHTt20LhxY9q0aQM41ccZXemSnStg5s6dy+TJk1OHS5YsmeU8d911F8HBwQB07tyZwYMH8+CDDzJ58mQ6d+6cutx169alznP06FHi4uKIiIhIHbdnzx4uuuii1OHY2Fjuv/9+Nm/ejIiQmJiYOq1NmzZERUUB8MMPP/DDDz/QsGFDwDkr2rx5My1btmTUqFFMnToVgF27drF58+ZzEsXIkSO92zlwVpvPaWn3r4gwefJkevfuTUJCAm3btiUkxPmZnT17Ng0aNODHH39k69attGnThmuuuYbixYun7oOuXbvyv//9LzWBjBw5khkzZnDllVfy+uuv06dPHz744AMASpcuzYYNG7yO/0IUyESxffth7rtvKr/95mT6tm2rMmZMB6pVi/JzZMZkLDw8nJUrVxIbG0unTp0YM2YMTzzxBHXq1GHBggVnld22bRvFihUjIiKCOnXqsHz58tRqnYxklHA8x6W9Pr9o0aKpr5s3b86WLVvYv38/33zzDQMHDgQgJSWFRYsWER6ecdc24eHhZy37+eef59prr2Xq1Kns2LGD1q1bp7tOVeW5557jkUceOWt5P/30E3PnzmXRokUUKVKE1q1bp3tvQXbOKMqVK5d6dgDODZinq4I8NW/enF9++QVwEtmmTZsA+Oijj+jfvz8iQrVq1ahcuTIbNmygadOmHD16lI4dOzJ06NDUs5b9+/ezatUqrrzySsBJxO3bt09dT3x8fKb7NCcVyKueihcPZdOmg1x8cTEmT76DWbPutSRhAkaJEiUYNWoUb7zxBomJidx77738+uuvzJ3r3Ah68uRJnnjiCZ599lkA+vbty8svv5z6g5WSksKIESPOWW7btm0ZPXp06vDpqqcyZcqwfv361KqljIgIt912G3369KF27dqpR+9pl7ty5cpz5q1du3bqFT3gnFGULVsWgIkTJ2a4znbt2jFhwoTUNpTdu3ezb98+YmNjKVmyJEWKFGHDhg1nVdl4GjlyJCtXrjznL22SALj55puZPHkyCQkJbN++nc2bN6db7bNv3z4AEhISeO211+jRowcAFSpUYN68eQDs3buXjRs3UqVKFU6dOsVtt91Gt27duOuuu1KXU7JkSWJjY1Pftzlz5lC7du3U6Zs2bTqrus6nzrcV3F9/zlVP+zNs7c/IrFmbNT7+zFUSv/22U48cOZnt5ZiCKy9d9aSq2qlTJ500aZKqqq5evVpbtWqlNWrU0KpVq+qgQYM0JeXM1XrfffedNmrUSGvVqqW1a9fWZ5555pzlx8XFabdu3bROnTpar149/eqrr1RV9csvv9QqVapoq1attFevXqlX3tx///365ZdfnrWMpUuXKqATJ05MHbd//37917/+pZdffrnWrl1bH3nkkXS3r27dunr06FFVVf3tt9+0evXqetVVV+nAgQO1YsWKqqr60Ucfaa9evc6a76233tK6detq3bp1tVmzZrplyxaNj4/X9u3b6+WXX6533nmntmrVSufPn5/FHs7a0KFDtUqVKlqjRg2dMWNG6vgbb7xRd+/eraqqzzzzjNaqVUtr1KihI0eOTC2ze/dubdOmjdatW1fr1KmjH3/8saqqfvzxxxoSEqL169dP/VuxYoWqqn799ddat25drVevnrZq1Uq3bt2auryGDRvq/v3p/xbm9FVPounUu+VlTcqLLtu4H4pEe1V+165YnnhiFt98s4EhQ65l4MCWPo7Q5Ffr168/64jO5KyRI0cSERGR5++lyAtWrFjBiBEj+Pjjj9Odnt5nVUSWq2qT81lfvq16SkpKYcSIRdSuPYZvvtlAsWKFiYqy7r+NyaseffRRQkND/R1GQDhw4ABDhgzJtfUFZmN2FldyLF4cQ48e01m1ai8Ad9xRm7ffbk/ZssVzIzpjzHkICwuja9eu/g4jIJy+4i23BGaiyMTvv8dw1VUfogqVKkUyevSNdOxYI+sZjfGCZnIpqjF5gS+aE/JdomjatCzt2lWjYcOLGTiwJUWK5NzDO0zBFhYWxsGDB62rcZNnqTrPowgLC8vR5QZ8oti8+SC9e89mxIh21KjhfIG///4egoLsi2xyVrly5YiJiWH//v3+DsWYDJ1+wl1OCthEkZCQxKuv/sorr/xKQkIyYWEhTJnyLwBLEsYnChUqlKNPDTMmUPj0qicRaS8iG0Vki4iccweLiISKyOfu9N9FpJI3y503fxf16o1j0KCfSUhI5sEHGzBuXKecDt8YYww+PKMQkWBgDNAGiAGWisg0VV3nUaw7cFhVq4lIF+A1oHNmy91+KJIbOn4LQO3a0Ywb18k68TPGGB/y5RlFU2CLqm5T1VPAZOCWNGVuAf7nvp4CXC9ZtBIePhFOWFgwL798HStX9rAkYYwxPuazO7NF5E6gvar+2x3uClypqo95lPnTLRPjDm91yxxIs6yHgdOdydcF/vRJ0IEnGjiQZamCwfbFGbYvzrB9cUZNVY3Iuti5fNmYnd6ZQdqs5E0ZVPU94D0AEVl2vreh5ze2L86wfXGG7YszbF+cISLLzndeX1Y9xQDlPYbLAX9nVEZEQoASwCEfxmSMMSabfJkolgLVRaSyiBQGugDT0pSZBtzvvr4T+FEDrZdCY4zJ53xW9aSqSSLyGDAbCAYmqOpaERmM093tNOBD4GMR2YJzJtHFi0W/56uYA5DtizNsX5xh++IM2xdnnPe+CLhuxo0xxuSufNvNuDHGmJxhicIYY0ym8myi8FX3H4HIi33RR0TWichqEZknIvn2LsSs9oVHuTtFREUk314a6c2+EJF/uZ+NtSLyaW7HmFu8+I5UEJH5IrLC/Z508EecviYiE0Rkn3uPWnrTRURGuftptYg08mrB5/sMVV/+4TR+bwWqAIWBVcBlacr0BMa5r7sAn/s7bj/ui2uBIu7rRwvyvnDLRQALgMVAE3/H7cfPRXVgBVDSHS7t77j9uC/eAx51X18G7PB33D7aFy2BRsCfGUzvAMzEuYetGfC7N8vNq2cUPun+I0BluS9Udb6qnnAHF+Pcs5IfefO5ABgCDAficzO4XObNvvgPMEZVDwOo6r5cjjG3eLMvFDj9iMsSnHtPV76gqgvI/F60W4BJ6lgMRIrIJVktN68mirLALo/hGHdcumVUNQmIBUrlSnS5y5t94ak7zhFDfpTlvhCRhkB5VZ2em4H5gTefixpADRFZKCKLRaR9rkWXu7zZF4OA+0QkBpgBPJ47oeU52f09AfLu8yhyrPuPfMDr7RSR+4AmQCufRuQ/me4LEQkCRgIP5FZAfuTN5yIEp/qpNc5Z5i8iUldVj/g4ttzmzb64G5ioqm+KSHOc+7fqqmqK78PLU87rdzOvnlFY9x9neLMvEJEbgAHAzaqakEux5bas9kUETqeRP4nIDpw62Gn5tEHb2+/It6qaqKrbgY04iSO/8WZfdAe+AFDVRUAYToeBBY1Xvydp5dVEYd1/nJHlvnCrW8bjJIn8Wg8NWewLVY1V1WhVraSqlXDaa25W1fPuDC0P8+Y78g3OhQ6ISDROVdS2XI0yd3izL3YC1wOISG2cRFEQn2k7DejmXv3UDIhV1T1ZzZQnq57Ud91/BBwv98XrQDHgS7c9f6eq3uy3oH3Ey31RIHi5L2YDbUVkHZAM9FXVg/6L2je83BdPA++LSG+cqpYH8uOBpYh8hlPVGO22x7wIFAJQ1XE47TMdgC3ACeBBr5abD/eVMcaYHJRXq56MMcbkEZYojDHGZMoShTHGmExZojDGGJMpSxTGGGMyZYnC5DkikiwiKz3+KmVStlJGPWVmc50/ub2PrnK7vKh5HsvoISLd3NcPiMilHtM+EJHLcjjOpSLSwIt5nhKRIhe6blNwWaIwedFJVW3g8bcjl9Z7r6rWx+ls8vXszqyq41R1kjv4AHCpx7R/q+q6HInyTJzv4l2cTwGWKMx5s0RhAoJ75vCLiPzh/l2VTpk6IrLEPQtZLSLV3fH3eYwfLyLBWaxuAVDNnfd69xkGa9y+/kPd8a/KmWeAvOGOGyQiz4jInTh9bv2fu85w90ygiYg8KiLDPWJ+QETeOc84F+HRoZuIjBWRZeI8e+Ild9wTOAlrvojMd8e1FZFF7n78UkSKZbEeU8BZojB5UbhHtdNUd9w+oI2qNgI6A6PSma8H8LaqNsD5oY5xu2voDLRwxycD92ax/puANSISBkwEOqvq5Tg9GTwqIlHAbUAdVa0HDPWcWVWnAMtwjvwbqOpJj8lTgNs9hjsDn59nnO1xuuk4bYCqNgHqAa1EpJ6qjsLpy+daVb3W7cpjIHCDuy+XAX2yWI8p4PJkFx6mR0m4agAAAlRJREFUwDvp/lh6KgSMduvkk3H6LUprETBARMoBX6vqZhG5HmgMLHW7NwnHSTrp+f/27t41iigK4/DvbSwUDKRQrPxA0M5KCdjZiY2IREVCmmCjjZDG/8DGTiWISFKoBEFB/ECDSIrAxkqjhqAQ7URSBJGgIHgszl2J6+y4Uy6+T7ezu3PvDOycvWeGc25K+gZ8JMtQ7wE+RMS78v4UcBa4TPa6uC7pIdBzSfOIWJG0XOrsvC9jzJX9NpnnJrJcxfoOZcOSzpC/621kg56Fju8Ole1zZZwN5Hkz68qBwvrFeeAzsI9cCf/VlCgibkmaB44ATySNkWWVpyLiQg9jnF5fQFBSZX+TUlvoAFlk7iRwDjjU4FimgWFgCbgXEaG8avc8T7KL20XgCnBM0k5gHNgfEauSJsnCd50EzETEqQbztf+cU0/WLwaAT6V/wAj5b/oPknYByyXdcp9MwTwDjkvaUj4zqN57ii8BOyTtLq9HgNmS0x+IiEfkjeKqJ4++kmXPq9wFjpI9EqbLtkbzjIgfZAppqKStNgNrwBdJW4HDXebSAg62j0nSRklVqzOz3xworF9cBUYltci001rFZ04AbyS9BPaSLR8XyQvqU0kLwAyZlvmniPhOVte8I+k18BOYIC+6D8r+ZsnVTqdJYKJ9M7tjv6vAIrA9Il6UbY3nWe59XALGI+IV2R/7LXCDTGe1XQMeS3oeESvkE1m3yzgt8lyZdeXqsWZmVssrCjMzq+VAYWZmtRwozMyslgOFmZnVcqAwM7NaDhRmZlbLgcLMzGr9As4LJXGnzf0rAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Compute ROC curve and ROC area for each class\n",
    "fpr, tpr, _ = roc_curve(y_test, predict_y[:, 1],\n",
    "                        drop_intermediate = True)\n",
    "\n",
    "roc_auc = auc(fpr, tpr)\n",
    "\n",
    "# Plot of a ROC curve for a specific class\n",
    "plt.figure()\n",
    "lw = 2\n",
    "\n",
    "plt.plot(fpr, tpr, color='darkorange',\n",
    "         lw=lw, label='ROC curve (area = %0.4f)' % roc_auc)\n",
    "\n",
    "plt.plot([0, 1], [0, 1], color='navy', lw=lw, linestyle='--')\n",
    "\n",
    "plt.xlim([0.0, 1.0])\n",
    "plt.ylim([0.0, 1.05])\n",
    "plt.xlabel('False Positive Rate')\n",
    "plt.ylabel('True Positive Rate')\n",
    "plt.title('Receiver operating characteristic example')\n",
    "plt.legend(loc=\"lower right\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally let's plot the importance of the features extracted from the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAd4AAAEWCAYAAADIJfYaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOzde5xN9f748debCdEFuc0MYcyY+54RojoyaIqoTuWUknunnG6OQk4dnZxTPzqoTBdC0bcipUQRFY3kKOVayggzpSHMDDGDMWPevz/Wmt2e+y7NEO/n47Ef9v6s9fmsz/pss9/7s9Zn5i2qijHGGGOqRrWT3QFjjDHmTGKB1xhjjKlCFniNMcaYKmSB1xhjjKlCFniNMcaYKmSB1xhjjKlCFniN+Y1EZKqIjDnZ/TDG/LGI/R6vqWoikgY0Bo77FLdW1V0n0GYC8KqqNj2x3v0xicgs4EdV/efJ7osxpnw24zUnyzWqeo7P4zcH3d+DiASczOOfCBGpfrL7YIzxnwVec0oRkY4i8j8ROSAiG92ZbOG2QSLyrYgcEpEdInKnW14HeB8IEpFs9xEkIrNE5DGf+gki8qPP6zQReVBENgE5IhLg1ntLRPaJSKqI3FdOX73tF7YtIqNEZK+I7BaRP4vI1SKyVUSyROQhn7qPisg8EZnrns86EYnz2R4pIsnuOGwWkWuLHXeKiCwWkRxgCNAXGOWe+7vufqNFZLvb/jcicr1PGwNF5FMRmSgi+91z7eGzvb6IzBSRXe72d3y29RKRDW7f/iciHr/fYGOMBV5z6hCRYGAR8BhQHxgBvCUiDd1d9gK9gPOAQcBTInKRquYAPYBdv2EGfQvQE6gLFADvAhuBYKAb8HcRucrPtpoAtdy6jwDTgduAtkAn4BERCfHZ/zrgTfdcZwPviMhZInKW248PgEbAvcBrIhLuU/dW4HHgXOD/gNeA/7rnfo27z3b3uOcDY4FXRSTQp40OQArQAPgv8KKIiLvtFaA2EO324SkAEbkIeAm4E7gAeAFYKCI1/RwjY854FnjNyfKOO2M64DObug1YrKqLVbVAVT8EvgSuBlDVRaq6XR0rcAJTpxPsR5Kq7lTVI0B7oKGq/ltVj6nqDpzg2cfPtvKAx1U1D3gdJ6BNVtVDqroZ2Az4zg7Xquo8d/8ncYJ2R/dxDjDe7cdy4D2cLwmFFqjqKnecjpbWGVV9U1V3ufvMBb4DLvbZ5XtVna6qx4GXgUCgsRucewBDVXW/qua54w3wV+AFVf1cVY+r6stArttnY4wf/rD3tcwf3p9V9aNiZc2Bv4jINT5lZwEfA7iXQv8FtMb50lgb+OoE+7Gz2PGDROSAT1l1YKWfbWW6QQzgiPvvHp/tR3ACaoljq2qBexk8qHCbqhb47Ps9zky6tH6XSkT6A/cDLdyic3C+DBT6yef4h93J7jk4M/AsVd1fSrPNgQEicq9PWQ2ffhtjKmCB15xKdgKvqOpfi29wL2W+BfTHme3luTPlwkujpS3Pz8EJzoWalLKPb72dQKqqhv2Wzv8GzQqfiEg1oClQeIm8mYhU8wm+FwJbfeoWP98ir0WkOc5svRuwWlWPi8gGfhmv8uwE6otIXVU9UMq2x1X1cT/aMcaUwi41m1PJq8A1InKViFQXkVruoqWmOLOqmsA+IN+d/V7pU3cPcIGInO9TtgG42l0o1AT4ewXHXwMcdBdcne32IUZE2v9uZ1hUWxG5wV1R/XecS7afAZ/jfGkY5d7zTQCuwbl8XZY9gO/94zo4wXgfOAvTgBh/OqWqu3EWqz0vIvXcPlzubp4ODBWRDuKoIyI9ReRcP8/ZmDOeBV5zylDVnTgLjh7CCRg7gZFANVU9BNwHvAHsx1lctNCn7hZgDrDDvW8chLNAaCOQhnM/eG4Fxz+OE+DigVQgA5iBszipMiwAbsY5n37ADe791GPAtTj3WTOA54H+7jmW5UUgqvCeuap+A0wCVuME5Vhg1a/oWz+ce9ZbcBa1/R1AVb/Euc/7rNvvbcDAX9GuMWc8+wMaxpwEIvIoEKqqt53svhhjqpbNeI0xxpgqZIHXGGOMqUJ2qdkYY4ypQjbjNcYYY6rQH/73eOvWrauhoaEnuxunvJycHOrUqXOyu3HKs3Hyj42Tf07lcVq7dm2GqjaseE/ze/vDB97GjRvz5ZdfnuxunPKSk5NJSEg42d045dk4+cfGyT+n8jiJyPcnuw9nKrvUbIwxxlQhC7zGGGNMFbLAa4wxxlQhC7zGGGNMFbLAa4wxxlQhC7zGGGNMFbLAa4wxxlQhC7zGGGNMFbLAa4wxxlQhC7zGGGNMFbLAa4wxxlQhC7zGGGNMFbLAa4wxxlQhC7zGGGNMFbLAa4wxf1A7d+6kS5cuREZGEh0dzeTJkwHIysoiMTGRsLAwEhMT2b9/f6n1ReS4iGxwHwt9yu8RkW0ioiLSwKe8nojMF5FNIrJGRGLc8nCfdjaIyEER+bu77VERSffZdrVbnigia0XkK/ffrj7HeVxEdopIdrH+1hSRuW7fPheRFj7b/uGWp4jIVT7l3d2ybSIy2qe8q4isE5GvReRlEQlwy0VEktz9N4nIRT51nnD3/1pEbq5ovMpSKYFXRJJ9T9wt+7uIPC8iS0TkgIi8V2z7Sp83ZpeIvFMZfTPGmNNFQEAAkyZN4ttvv+Wzzz7jueee45tvvmH8+PF069aN7777jm7dujF+/PiymjiiqvHu41qf8lXAFUDxnL0PARtU1QP0ByYDqGpKYTtAW+AwMN+n3lM+x1nslmUA16hqLDAAeMVn/3eBi0vp7xBgv6qGAk8BTwCISBTQB4gGugPPi0h1EakOPAf0AKKAW0QkSkSqAS8DfVQ1xj3PAe4xegBh7uMOYIp7jJ7ARUA80AEYKSLnVTBepQrwZ6ffYA7OICz1KesDjARqALWBO30rqGqnwuci8hawwJ8DHck7TovRi060v6e9B2LzGWjjVCEbJ//YOPmnonFKG9/zhNoPDAwkMDAQgHPPPZfIyEjS09NZsGABycnJAAwYMICEhASeeOIJv9tV1fUAIlJ8UxQwzt1ni4i0EJHGqrrHZ59uwHZVLTcIFR7DtRmoJSI1VTVXVT8r4/jXAY+6z+cBz4qz03XA66qaC6SKyDZ+CdzbVHWH297r7r77gFxV3eru8yHwD+BFd/v/qaoCn4lIXREJdM99harmA/kishEnyL9RzniVqrIuNc8DeolITbczLYAg4FNVXQYcKquiiJwLdAVsxmuMMX5KS0tj/fr1dOjQgT179ngDcmBgIHv37i2rWi0R+VJEPhORP/txmI3ADQAicjHQHGhabJ8+OJMvX/e4l21fEpF6pbR7I7DeDZzlCQZ2ArgB8GfgAt9y149uWVnlGcBZItLOLe8NNCt+jGJ1NgI9RKS2ezm5i0+dX6VSZryqmikia3C+DSzAeSPmut8gKnI9sExVD5a1g4jcgXMJgAYNGvJIbP7v0OvTW+OznW/fpnw2Tv6xcfJPReNUOCs9UUeOHGHYsGHcfvvtrFu3jvz8/CJtF3/t40JV3SUiIcByEflKVbeXc6jxwGQR2QB8BawHvCcoIjWAa3Fmj4WmAP8B1P13EjDYp040ziXjK/041dKmlFpOeWmTS1VVFZE+wFPuBPEDn/MotS1V/UBE2gP/w5kxr/ap86tU1qVm+OVyc2HgHVz+7l63ADPK20FVpwHTAC4MCdVJX1XmaZweHojNx8apYjZO/rFx8k9F45TWN+GEj5GXl0evXr0YOnQo999/PwDBwcGEh4cTGBjI7t27CQoKIiGh5LFUdZf77w4RSQbaAGUGXndCNAicRUhAqvso1ANY53vp2fe5iEwH3vN53RTnXnD/CgJ+oR9xZpk/uouhzgeyfMoLNQV2uc9LLVfV1UAntx9XAq2LHaO0Oo8Dj7t1ZgPf+dHnklS1Uh7AOcBenJvRKcW2JQDvlVLnAiATqOXvcVq3bq2mYh9//PHJ7sIfgo2Tf2yc/FPZ41RQUKD9+vXTYcOGFSkfMWKEjhs3TlVVx40bpyNHjixRF2e2WtN5SgM3iERp0c/kNKCBz+u6QA33+V9x7oX67v86MKhYWaDP8+E492IL29oI3KhlfL4D2cVe3w1MdZ/3wbm/Cs6iqo1ATaAlsAOojjO53OGW1XD3iXbrNHL/rQksA7q6r3sC7+PMfDsCa9zy6sAF7nMP8DUQUN54lfWotF8nUtVsIBl4iZLX+8vyF5yAfLSy+mWMMaeLVatW8corr7B8+XLi4+OJj49n8eLFjB49mg8//JCwsDA+/PBDRo92fovmyy+/5Pbbby+sXgv40l0k9DEwXlW/ARCR+0TkR5zZ3iYRKbwKGQlsFpEtOLPbYYWNiUhtIBF4u1g3/+v+ytAmnPuiw93ye4BQYIzPb7Q0ctv6r3v82iLyo4g86tZ5EbjAXTx1PzAaQFU3A28A3wBLgLtV9bg694HvwVno+y1OoN7stjVSRL4FNgHvqupyt3wxTrDeBkwH7nLLzwJWisg3OFdcb3PbL2+8SiXq123X30ZErsd5EyJVdYtbthKIwJkRZwJDVHWpuy0Z581f4u8xwsPDNSUl5ffu+mknOTm51EtNpigbJ//YOPnnVB4nEVmrqu0q3tP83ir1Jo2qzqfYjWr1+bWhUvZPqMz+GGOMMSeb/eUqY4wxpgpZ4DXGGGOqkAVeY4wxpgpZ4DXGGGOqkAVeY4wxpgpZ4DXGGGOqkAVeY4wxpgpZ4DXGGGOqkAVeY4wxpgpZ4DXGGGOqkAVeY4z5FQYPHkyjRo2IiYnxlm3YsIGOHTsSHx9Pu3btWLNmTal1u3fvTt26denVq1eR8iFDhhAXF4fH46F3795kZ2cD8P3339OtWzc8Hg8JCQn8+OOPv7mtH374gS5dutCmTRs8Hg84KfUAEBGPiKwWkc1uQoNavm2KyEIR+drndZy7/1ci8q6InOeW1xCRmW75RhFJ8KlzS2GyBBFZ4iaTR0QmiMgWt3y+iNSt+F34g6sofdFveeBkJbqqWNnfgedxMkccoJS0gO5+z1AsFVR5D0sL6B9L4+YfGyf/nMnjtGLFCl27dq1GR0d7yxITE3Xx4sWqqrpo0SLt3LmzqpYcp48++kgXLlyoPXv2LFL+888/e58PHz7cm9Kvd+/eOmvWLFVVXbZsmd52222/ua2//vWv+vzzz6uq6ubNmxXIVeczNwAnQ0+c/pKetbr+8pl8AzAb+Nqn7Augs/t8MPAf/SVt30z3eSNgLc4ELwAnTWwDd9t/gUfd51fiptcDngCeUD8///+oj8pKkjAHJ1fiUp+yPsBInJyItYE7i1cSkXY4ORr9diTvOC1GL/rtPT1DPBCbz0AbpwrZOPnnjzxOaeN7nlD9yy+/nLS0tCJlIsLBgwcB+PnnnwkKCiq1brdu3UhOTi5Rft555wHOROjIkSM4Oebhm2++4amnngKgS5cu/PnPf/7NbRXvI5DnVrkS2KSqG916mT7ndQ5O+r07cNLuFQoHPnGff4jzWT8GiMLJbYuq7hWRA0A7nNy/AtQRkUzgPJy0e6jqBz7tfgb0LnFSp5nKutQ8D+glIjUBRKQFEAR8qqrLgEPFK4hIdWACMKqS+mSMMZXi6aefZuTIkTRr1owRI0Ywbty4X93GoEGDaNKkCVu2bOHee+8FIC4ujrfeeguA+fPnc+jQITIzM8trpsy2Hn30UV599VWaNm3K1VdfDfCDu3trQEVkqYisExHfz+D/AJOAw8UO8TVwrfv8L0Az9/lG4DoRCRCRlkBboJmq5gF/A74CduEE6BdL6fpgnCT0p7VKy8crIouAaaq6QERGAxeo6kh3WwIwQlV7+ew/DKimqk+JSLaqnlNO23fgfAOjQYOGbR95enqlnMPppPHZsOfIye7Fqc/GyT9/5HGKDT6/4p0q8NNPP/GPf/yDmTNnApCUlERcXBydO3fm448/5r333mPSpElkZ2dzzjlFP8o2bNjA3LlzSw3Ox48fJykpiYiICHr06EFGRgZJSUns3r0bj8fDJ598wsyZM71t/pq23njDmbDedNNNbN68mXvuuecoUAdnRns30B4nwC4D/omTL/0/qnqNO3l6T1VjAEQkAkjCuSy9ELhPVS8QkQCcCVQX4Huc5PEv4CSXX4Lzub0D55biT6r6WGF/ReRhnNnxDVpZgekUUZmB9zagp6reIiIbgMGqus7dloBP4BWRIJzLGAmqml9R4PV1YUioVrtpcqWcw+nkgdh8Jn1VqemXTws2Tv75I4/TiV5qBkhLS6NXr158/bWz3uj888/nwIEDiAiqyvnnn8/BgwdJTk4mISGhSN3k5GQmTpzIe++9V2rbK1asYMKECSW2Z2dnExERUWSB1a9pKzo6miVLltCsmTM5FZFjODPVrkB3VR3olo8BjgLZOJePj+Hco20E/E+L5U0XkdbAq6p6cfHji8j/gNtxAvx4Ve3mll8OjFbVq93XA4ChQDdVLT67Pu1U5k/OO8CTInIRcHZh0C1DGyAU2Obej6gtIttUNbSig5x9VnVSfocfpNNdcnIyaX0TTnY3Tnk2Tv6xcSoqKCiIFStWkJCQwPLlywkLC/O7rqqyfft2QkNDUVXeffddIiIiAMjIyKB+/fpUq1aNcePGMXjw4N/c1oUXXsiyZcsYOHAg3377LTj3XPfh3J8dJSK1cYJsZ+ApVV0ETAHv7cL3CoOuiDRy7+FWw5kdT3XLa+NM6HJEJBHIV9Vv3MlVlIg0VNV9QCLwrVunO/AgzmKt0z7oQiUGXlXNFpFk4CWcxVbl7bsIaFL42p3xVhh0jTGmqt1yyy0kJyeTkZFB06ZNGTt2LNOnT2fYsGHk5+dTq1Ytpk2bBkBKSgqvvvoqM2bMAKBTp05s2bKF7OxsmjZtyosvvkhiYiIDBgzg4MGDqCpxcXFMmTIFcL7g/OMf/0BEuPzyy3nuuee8/fi1bU2aNIm//vWvPPXUU4ULrtLcS7r7ReRJnJXKCix2P5PLHQYRudt9/jYw033eCFgqIgVAOtAPQFV3ichY4BMRycO5DD3QrfMsUBP40O3XZ6o69Fe+LX8olXapGUBErsd5UyJVdYtbthKIAM7BuYcwRFWXFqvn96Xm8PBwTUlJ+X07fhoq7ZKXKcnGyT82Tv45lcdJRNaqaruT3Y8zUaXepFHV+TiXM3zLOvlRz6+ga4wxxvzR2F+uMsYYY6qQBV5jjDGmClngNcYYY6qQBV5jjDGmClngNcYYY6qQBV5jjDGmClngNcYYY6qQBV5jjDGmClngNcYYY6qQBV5jjDGmClngNcb84QwePJhGjRoRExNTpPyZZ54hPDyc6OhoRo0aVWrdJUuWEB4eTmhoKOPHj/eWDxkyhLi4ODweD7179yY7OxuATz75hIsuuoiAgADmzZtXpK3q1asTHx9PfHw81157rbdcVXn44Yfp168fkZGRJCUlAbBgwQI8Hg/x8fG0a9eOTz/91FvnwQcfJCYmhpiYGObOnest79u3L+Hh4cTExDB48GDy8vIAeO211/B4PHg8Hi699FI2btzorXPgwAF69+5NREQEkZGRrF69GnDy93bs2JH4+HiASBG5GJz8uiKyWkRyRWSE7zmKyHAR2SwiX4vIHBGp5ZbfIyLbRERFpIHP/n1FZJP7+J+IxJX6RpzJVPV3fwDJwFXFyv4OPI+TDPkAToop3+0tgc+B74C5QA1/jtW6dWs1Ffv4449Pdhf+EGyc/HOyx2nFihW6du1ajY6O9pYtX75cu3XrpkePHlVV1T179pSol5+fryEhIbp9+3bNzc1Vj8ejmzdvVlXVn3/+2bvf8OHDddy4caqqmpqaqhs3btR+/frpm2++WaS9OnXqlNq/l156Sfv166fLli0r0pdDhw5pQUGBqqpu3LhRw8PDVVX1vffe0yuuuELz8vI0Oztb27Zt6+3PokWLtKCgQAsKCrRPnz76/PPPq6rqqlWrNCsrS1VVFy9erBdffLH3+P3799fp06erqmpubq7u379fVVUTExN18eLFqqrqftYmO09pBLQHHsfJlV74uRwMpOKkdgUnb/pA93kboAWQBjTwqXMpUM993gP4XP34LD+THpWVJGEO0Acnz2OhPsBIoAZQG7izWJ0ncHJAvi4iU4EhuLkgy3Mk7zgtRleUwco8EJvPQBunCtk4+edEx+lEk9FffvnlpKWlFSmbMmUKo0ePpmbNmgA0atSoRL01a9YQGhpKSEgIAH369GHBggVERUVx3nnnAc5k5MiRI4Wp82jRogUA1ar5f4FwypQpzJ4925u0vrAv55zzS/6XnJwc7zG++eYbOnfuTEBAAAEBAcTFxbFkyRJuuukmrr76am+diy++2NvmpZde6i3v2LGjt/zgwYN88sknzJo1C4AaNWpQo0YNAESEgwcPFlarDuxyz3kvsFdESntjAoCz3XR+tX3qrC9s05eq/s/n5WdA04rG60xTWZea5wG9RKQmeJMoBwGfquoy4JDvzuK8c13degAvA3+upL4ZY05DW7duZeXKlXTo0IHOnTvzxRdflNgnPT2dZs2aeV83bdqU9PR07+tBgwbRpEkTtmzZwr333lvhMY8ePUq7du3o2LEj77zzjrd8+/btzJ07lzvvvJMePXrw3XffebfNnz+fiIgIevbsyUsvvQRAXFwc77//PocPHyYjI4OPP/6YnTt3FjlWXl4er7zyCt27dy/RjxdffJEePXoAsGPHDho2bMigQYNo06YNt99+Ozk5OQA8/fTTjBw5snAMmgL/KO/8VDUdmAj8AOwGflbVDyocmF8MAd7/FfufESplxquqmSKyBugOLMCZ7c51L2+U5gLggKrmu69/xLnEUSoRuQO4A6BBg4Y8Eptf1q7G1fhsZ5Ziymfj5J8THafk5OQT7sNPP/1ETk6Ot62ff/6Zr776ivHjx7NlyxauvfZaZs+eXWRG9vXXX7N7925vnW+//ZZdu3Z5Xw8YMIDbbruNpKQkxo4d6w1mhcfbvHkzDRp4b2fy+uuv06BBA3bt2sXQoUPJyckhODiYw4cPk56ezqRJk1i3bh033nij9z5vvXr1mDp1Khs3buSee+5h0qRJ1KhRg8jISDweD3Xr1iUkJITU1NQi4zRx4kRCQkI4fvx4kfL169fzzDPPkJSURHJyMikpKaxdu5aBAwcycOBAnnnmGf72t78xePBgkpKSGDJkCJ07d6ZLly47gReBK8oaYxGpB1yHcyvwAPCmiNymqq9W9P6ISBecwPunivY901RmPt7Cy82FgXdwOftKKWVlBWlUdRowDeDCkFCd9FWlphU+LTwQm4+NU8VsnPxzouOU1jfhhPuQlpZGnTp1vInmw8PDue+++0hISKBLly5MnDiRmJgYGjZs6K1Ts2ZNVq9e7a2zevVq2rdvXyJZfUBAABMmTOCJJ57wls2aNYvo6OgyE9t/8MEH1KxZk4SEBJo3b86oUaNIS0tjzJgxTJo0qUS9hIQEnn76aWJiYmjQoEGR7bfeeitXX321t2zs2LEEBATwxhtvFLnkvWnTJp599lk+/PBDWrduDUBERATjxo3jrrvuApwFYOPHjychIYHrrruOt956q/DLyH7g4gqG+QogVVX3AYjI2zj3cMsNvCLiAWYAPVQ1s4JjnHEq8xPmHeBJEbkI58b8unL2zQDqikiAO+ttinsfoSJnn1WdlBO8X3QmSE5O/l0+7E53Nk7+ORXH6c9//jPLly8nISGBrVu3cuzYsSKzU4D27dvz3XffkZqaSnBwMK+//jqzZ89GVdm+fTuhoaGoKu+++y4RERHlHm///v3Url2bmjVrkpGRwapVq7wrqQv7EhISwooVK7xBcdu2bbRq1QoRYd26dRw7dowLLriA48ePc+DAAS644AI2bdrEpk2buPLKKwGYMWMGS5cuZdmyZUWC7g8//MANN9zAK6+84m0foEmTJjRr1oyUlBTCw8NZtmwZUVFRAAQFBbFixYrCgH4uzgKr8vwAdBSR2sARoBvwZXkVRORC4G2gn6puraD9M1NlrtzCWQG3AXi0WHkCJVc1vwn0cZ9PBe7y5xi2qtk/J3sV6h+FjZN/TvY49enTR5s0aaIBAQEaHBysM2bM0NzcXO3bt69GR0drmzZtvCuK09PTtUePHt66ixYt0rCwMA0JCdHHHntMVVWPHz+ul156qcbExGh0dLTeeuut3lXFa9as0eDgYK1du7bWr19fo6KiVNVZVRwTE6Mej0djYmJ0xowZ3mPs379fr776am3ZsqV27NhRN2zYoKqq48eP16ioKI2Li9OOHTvqypUrVVX1yJEjGhkZqZGRkdqhQwddv369t63q1atrSEiIxsXFaVxcnI4dO1ZVVYcMGaJ169b1lrdt29ZbZ/369dq2bVuNjY3V6667zrv6eeXKlXrRRRepx+NRIBtoq85nbhOcW3wHcS4p/wic524bC2wBvgZeAWq65fe5++XjTJRmuOUzcGbTG9zHl3oKrCQ+lR6iZd52PXEicj3ON59IVd3ilq0EIoBzgExgiKouFZEQ4HWgPrAeuE1Vcys6Rnh4uKakpFTWKZw2kpOTy7xEZn5h4+QfGyf/nMrjJCJrVbXdye7HmahSb2ap6nyK3b9V1U5l7LuDiu83GGOMMX9o9perjDHGmCpkgdcYY4ypQhZ4jTHGmCpkgdcYY4ypQhZ4jTHGmCpkgdcYY4ypQhZ4jTHGmCpkgdcYY4ypQhZ4jTHGmCpkgdcY86sMHjyY66+/npiYGG/ZmDFj8Hg8xMfHc+WVV7JrV+k5TkaNGkV0dDSRkZHcd999FP+Ttddee22RdkeOHElERAQej4frr7+eAwcOAE5u2gEDBhAbG0tkZCTjxo3z1mnRogWxsbHEx8fTrl3Jv4g4ceJERISMjAzASSd4zTXXEBcXR3R0NDNnzvTu+/LLLxMWFkZYWBgvv/yyt3zu3Ll4PB6io6O9iREKvfHGG0RFRREdHc1//vOfItsOHjxIcHAw99xzT4VtzZo1i4YNGxIfH098fDwzZswAYMOGDVxyySVER0fj8XiYO3eut07fvn0JDw8nJiaGwYMHk5eXV+r7YE6yyvgD0EAycFWxsr8DzwNLcP4Id/EkCfcA23DSAbbOEvEAACAASURBVDbw91iWJME/J/uP2v9R2DhVbMWKFfrCCy9odHS0t6wwoYCq6uTJk/XOO+8sUW/VqlV66aWXan5+vubn52vHjh2LjPdbb72lt9xyS5F2ly5dqnl5eaqqOmrUKB01apSqqr722mt68803q6pqTk6ONm/eXFNTU1VVtXnz5rpv375S+/7DDz/olVdeqRdeeKF3n8cff9zb7t69e7VevXqam5urmZmZ2rJlS83MzNSsrCxt2bKlZmVlaUZGhjZr1kz37t2rqqr9+/fXjz76SFVVt27dqvHx8d6kBG+//XaR49933316yy236N13362qWm5bM2fO9O7nKyUlRbdu3aqqTgKIJk2a6P79+1XVSQBRUFCgBQUF2qdPH33++edLHQdVVSx5wUl7VNbfai7MxbvUp6wPMBKoAdQG7ixWZxXwHk7Q9tuRvOO0GL3oN3f0TPFAbD4DbZwqdCaMU9oJptG8/PLLS8xozzvvPO/znJycIsnnC4kIR48e5dixY6gqeXl5NG7cGIDs7GyefPJJpk2bxk033eStU5gaD6Bjx47MmzfP21ZOTg75+fkcOXKEGjVqFOlDWYYPH85///tfrrvuuiL9OnToEKpKdnY29evXJyAggKVLl5KYmEj9+vUBSExMZMmSJYSGhtK6dWtvnt8rrriCt956i27dujF9+nTuvvtu6tWrB+D9F2Dt2rXs2bOH7t278+WXTma9HTt2lNlWWXxTAAYFBdGoUSP27dtH3bp1ufrqq73bLr74Yn788ccKx8RUvcq61DwP6CUiNQFEpAUQBHyqqsuAQ8UrqOp6VU2rpP4YYyrZww8/TLNmzXjttdf497//XWL7JZdcQpcuXQgMDCQwMJCrrrqKyMhIwLlU/cADD1C7du0y23/ppZfo0aMHAL1796ZOnToEBgZy4YUXMmLECG+AFBGuvPJK2rZty7Rp07z1Fy5cSHBwMHFxcUXaveeee/j2228JCgoiNjaWyZMnU61aNdLT02nWrJl3v6ZNm5Kenk5oaChbtmwhLS2N/Px83nnnHXbu3AnA1q1b2bp1K5dddhkdO3ZkzZo1ABQUFPDAAw8wYcKEIscury2At956C4/HQ+/evYuUF1qzZg3Hjh2jVatWRcrz8vJ45ZVX6N69e5njaU6eSpnxqmqmiKwBugMLcGa7c93LGydMRO4A7gBo0KAhj8Tm/x7NntYan+3M5kz5zoRxSk5OPuE2cnJyyMnJKdJWYmIiiYmJvPbaa4wYMYJBgwYVqZOens6nn37KnDlzABgxYgSNGjWiTp06fP7551x33XV89tlnJdoFePXVVzlw4ADBwcEkJyfz1VdfkZGRwZw5czh06BDDhg3jnHPOISgoiAkTJtCgQQP279/PiBEjOHLkCOHh4Tz44INMmDCB5ORkjh49yqpVqzj//PNZsWIFDRo0YPbs2ezatYvbb7+dGTNmsG3bNvLy8rx9SU1NpVatWmzcuJG77rqLHj16UK1aNaKjozlw4ADJycns2bOHzMxMxo4dy759+7j33nuJioriww8/JDw8nO3bt7NlyxbS09O97ZbVVr169Xj55ZepUaMGCxcu5LrrruPJJ5/0jklmZibDhw9n9OjRfPLJJ0XGa+LEiYSEhHD8+PHf5f02v6/KTAtYeLm5MPAO/r0aVtVpwDSAC0NCddJXlZrd8LTwQGw+Nk4VOxPGKa1vwgm38dNPP1GnTp1Sc822bNmSnj17FlmMBDBhwgR69uzpnbV+8cUX5ObmUqNGDdLS0hg4cCD5+fns3buXRx991BswXn75ZTZv3syyZcu8M+I333yTAQMGcMUVVwDw7rvvEhAQUKI/GzduJC8vj+DgYDIzM72LmjIyMrj33ntZs2YNEyZMYPTo0XTq5GQsffHFF2nYsCGdO3cukk93zpw5dOrUiYSEBBISEnjooYcAmDZtGtu2bSMhIYG4uDg6duzo7deECRNo3LgxGRkZrFy5kqVLl5Kdnc2xY8cIDw9n/PjxZbblq1OnTtSvX99bfvDgQRISEpg0aRJ/+ctfiuw7duxYAgICeOONN6hWzdbPnpIq6+YxTqL7vcBFQEqxbQkUW1zlsy0NW1z1u7NFQ/6xcfLPnDlziiyCKlzso6qalJSkN954Y4k6r7/+unbr1k3z8vL02LFj2rVrV124cGGRfVJTU4u0+/7772tkZKR38VGh8ePH68CBA7WgoECzs7M1MjJSN27cqNnZ2Xrw4EFVVc3OztZLLrlE33///RJ98V2ANXToUP3Xv/6lqqo//fSTBgUF6b59+zQzM1NbtGihWVlZmpWVpS1atNDMzExVVd2zZ4+qqmZlZWlcXJympKR4+9u/f39VVd23b582bNhQMzIyihy7+KKpstratWuXd5+3335bO3TooKqqubm52rVrV33qqadKnNf06dP1kksu0cOHD5fYVhy2uOqkPSrtq72qZotIMvASzuzXGHMauOWWW/jggw84ePAgTZs2ZezYsSxevJiUlBSqVatG8+bNmTp1KgBffvklU6dOZcaMGfTu3Zvly5cTGxuLiNC9e3euueaaco91zz33kJubS2JiIuAssJo6dSp33303gwYNIiYmBlVl0KBBeDweduzYwfXXXw9Afn4+t956a4X3OceMGcPAgQOJjY1FVXniiSdo0KCBd1v79u0BeOSRR7z3kYcNG8bGjRu95YULnq666io++OADoqKiqF69OkOHDuWCCy4o9/hltZWUlMTChQsJCAigfv36zJo1C3B+XemTTz4hMzPTWzZr1izi4+MZOnQozZs355JLLgHghhtu4JFHHin3+KbqifPFp5IaF7keeBuIVNUtbtlKIAJnRpwJDFHVpSJyHzAKaIIzU16sqrdXdIzw8HBNSUmprFM4bfheMjNls3Hyj42Tf07lcRKRtapa8hedTaWr1JtZqjofkGJlncrYNwlIqsz+GGOMMSeb3Xk3xhhjqpAFXmOMMaYKWeA1xhhjqpAFXmOMMaYKWeA1xhhjqtCvDrwiUk9EPJXRGWOMMeZ051fgFZFkETlPROoDG4GZIvJkRfWMMcYYU5S/M97zVfUgcAMwU1XbAldUXreMMcaY05O/gTdARAKBm3By5hpjjDHmN/A38P4bJ6n9dlX9QkRCgO8qr1vGGGPM6cmvwKuqb6qqR1X/5r7eoao3Vm7XjDl9DR48mEaNGhETE+Mty8rKIjExkbCwMBITE9m/f3+pdX/44QeuvPJKIiMjiYqKIi0tDYCBAwfSsmVL4uPjiY+PZ8OGDQC89tpreDwePB4Pl156qfcP8u/cuZMuXboQGRlJdHQ0kydPLnKcZ555hvDwcKKjoxk1ahTgJF6Pj4/n9ttvJy4ujvnz53v3X7JkCeHh4YSGhjJ+/HhveWpqKh06dCAsLIybb76ZY8eOAfDJJ59w0UUXERAQwLx584oc++WXXyYsLIywsDBvesHDhw/Ts2dPIiIiiI6OZvTo0UXqvPHGG0RFRREdHc2tt94KwPfff0/btm2Jj48nOjram7yhvLaGDx/uHcPWrVtTt27dMt9HY34Tf1IYAa2BZcDX7msP8M9y9k8GripW9nfgeWAJcIBiaQFx/qbz48BW4FvgPn/6ZmkB/WPp7vxTVeO0YsUKXbt2bZEUeCNHjtRx48apquq4ceN01KhRpdbt3LmzfvDBB6qqeujQIc3JyVFV1QEDBuibb75ZYv9Vq1ZpVlaWqqouXrxYL774YlV10s6tXbtWVVUPHjyoYWFhunnzZlVVXb58uXbr1k2PHj2qqr+krsvJydG8vDz9+OOPddeuXdqwYUPNy8vT/Px8DQkJ0e3bt2tubq56PB5vW3/5y190zpw5qqp655136vPPP6+qTgrAjRs3ar9+/Yr0OzMzU1u2bKmZmZmalZWlLVu21KysLM3JydHly5erqpMa709/+pMuXrxYVZ20hPHx8d7zLOxvbm6u9xwOHTqkzZs31/T09HLb8pWUlKSDBg0q9X3wx6n8c4elBTzl0wJOB0YCL7jBepOIzAYeK2P/OUAfnMvThfq4bdQAagN3FqszEGgGRKhqgYg08qdjR/KO02L0Ij9P48z1QGw+A22cKuTvOKWN73lCx7n88su9M9VCCxYs8CZ/HzBgAAkJCTzxxBNF9vnmm2/Iz8/3psk755xzKjzWpZde6n3esWNHfvzxRwACAwMJDAwE4NxzzyUyMpL09HSioqKYMmUKo0ePpmbNmgA0auT8OBYmogc4evQoIk4OlDVr1hAaGkpISAgAffr0YcGCBURGRrJ8+XJmz57tPa9HH32Uv/3tb7Ro0QKgRLL2pUuXkpiY6E3Bl5iYyJIlS7jlllvo0qULADVq1OCiiy7ynsv06dO5++67qVevXpH+1qhRw9tubm4uBQUF3vMoqy1fc+bMYezYsRWOsTG/hr/3eGur6ppiZfnl7D8P6CUiNQFEpAUQBHyqqsuAQ6XU+Rvwb1UtAFDVvX72zZjTwp49e7yBMDAwkL17S/4IbN26lbp163LDDTfQpk0bRo4cyfHjx73bH374YTweD8OHDyc3N7dE/RdffJEePXqUKE9LS2P9+vV06NDBe5yVK1fSoUMHOnfuzBdffOHd9/PPP/fmr506dSoBAQGkp6fTrFkz7z5NmzYlPT2dzMxM6tatS0BAQJHy8pTVlq8DBw7w7rvv0q1bN29/t27dymWXXUbHjh1ZsmSJd9+dO3fi8Xho1qwZDz74IEFBQeW2Vej7778nNTWVrl27lttfY34tf2e8GSLSCnCuCYv0BnaXtbOqZorIGqA7sABntjvXvbxRllbAzW4O3304l5pLXcAlIncAdwA0aNCQR2LL+w5gABqf7czmTPn8HafCmemJ+Omnn8jJyfG2lZ+fX6Td4q8BNm7cSHJyMtOmTaNx48aMHTuW0aNH07NnT6655hoGDBhAXl4ekyZNYujQoQwYMMBbd/369TzzzDMkJSUVaffIkSMMGzaM22+/nXXr1gHw888/89VXXzF+/Hi2bNnCtddey+zZs70z3GeffZbMzEweeugh6tSpw9dff83u3bu97X777bfs2rWLTz/9lCNHjnjL9+7dy+HDh4sc/6effmLz5s3e5PPbtm0jLy/Pu09qaiq1atXyvj5+/DgPPfQQV199NT/88AM//PADe/bsITMzk7Fjx7Jv3z769evHzJkzvVcEkpKSyMjIYMyYMQQGBnpn06W1VWjOnDlccsklrFy58le9r76ys7N/l/8r5vTib+C9G5gGRIhIOpAK9K2gTuHl5sLAO7iC/WsCR1W1nYjcALwElJW7d5rbHy4MCdVJX1VqWuHTwgOx+dg4VczfcUrrm3DCx0pLS6NOnTreROnBwcGEh4cTGBjI7t27CQoKKpFEvVatWnz88cfexUO7du3is88+K7FfjRo1mDhxord806ZNPPvss3z44Ye0bt3au19eXh69evVi6NCh3H///d7y8PBw7rvvPhISEujSpQsTJ04kJiaGhg0bAs4Xj169ejFr1izq16/PVVddxerVq73HW716Ne3bt+e6667j9ttv509/+hMBAQGsXr2asLCwIv2dNWsW0dHR3rLCAF74es6cOXTq1Mn7evDgwXTo0IGkpF/Sd8fFxdGxY0euuML58wIzZsygcePGtG/fvsi4LFq0iIKCgnLbKjR8+HCee+65Ipfqfy3f8zDGq6KbwDiXo29yn9cBzvXn5jFwDrAXuAhIKbYtgZKLq7YALdznAvzsz3FscZV/TuVFHqeSqhyn1NTUIourRowYUWRx1ciRI0vUyc/PV4/Ho3v37lVV1YEDB+qzzz6rqs5iKVXVgoICHTZsmD744IOqqvr9999rq1atdNWqVUXaKigo0H79+umwYcNKHGfKlCk6ZswYVVVNSUnRpk2bakFBge7YscO7uCotLU0DAwN13759mpeXpy1bttQdO3Z4F1d9/fXXqqrau3fvIournnvuuSLHKr4oLDMzU1u0aKFZWVmalZWlLVq00MzMTFVVffjhh/WGG27Q48ePF2nj/fff1/79+6uq6r59+7Rp06aakZGhO3fu1MOHD6uqalZWloaFhemmTZvKbUtVdcuWLdq8eXMtKCgose3XOJV/7rDFVSft4d9O8MlvahzeADYAjxYrLy3wjgcG+2z/wp9jWOD1z6n8AXAqqapx6tOnjzZp0kQDAgI0ODhYZ8yYoRkZGdq1a1cNDQ3Vrl27eoPNF198oUOGDPHW/eCDDzQ2NlZjYmJ0wIABmpubq6qqXbp00ZiYGI2Ojta+ffvqoUOHVFV1yJAhWrduXY2Li9O4uDht27atqqquXLlSAY2NjfVuW7Rokao6K3379u2r0dHR2qZNG122bJmqqv7f//2fRkVFaatWrbRNmzY6f/58b78WLVqkYWFhGhISoo899pi3fPv27dq+fXtt1aqV9u7d27vKeM2aNRocHKy1a9fW+vXra1RUlLfOiy++qK1atdJWrVrpSy+9pKqqO3fuVEAjIiK8/Z0+fbqqOl8ihg8frpGRkRoTE+MN9IVj5fF4NDY2Vl944YUK21JV/de//uX94nIiTuWfOwu8J+8hzviXT0TGAEeAuUCOz2w5q4J61wNvA5GqusUtWwlEuDPiTGCIqi4VkbrAa8CFQDYwVFU3VtS38PBwTUlJqfAcznR2ycs/Nk7+sXHyz6k8TiKyVlXbnex+nIn8velXeH/2bp8yBULKq6Sq83EuG/uWlXXf9gBwYr+jYYwxxpzi/Aq8qtqysjtijDHGnAn8Crwi0r+0clX9v9+3O8YYY8zpzd9Lzb5r8msB3YB1gAVeY4wx5lfw91Lzvb6vReR84JVK6ZExxhhzGvP3T0YWdxgI+z07YowxxpwJ/L3H+y7un4vECdZRwJuV1SljjDHmdOXvPd6JPs/zge9VtWQqD2OMMcaUy99LzVer6gr3sUpVfxSRJyquZowxxhhf/gbexFLKSuYWM8YYY0y5yr3ULCJ/A+4CQkRkk8+mc4FVldkxY4wx5nRU0Yx3NnANsND9t/DRVlVvq+S+GXPaGDx4MI0aNSImJsZblpWVRWJiImFhYSQmJrJ///4S9b7//nvatm1LfHw80dHRTJ061btt7ty5eDweoqOjGTVqVIm68+bNQ0T48ssvATh27BiDBg0iNjaWuLi4Inli58yZQ2xsLB6Ph+7du5ORkQE4+X8vueQSYmNjueaaazh48CDg5NuNj48nPj6euLg45s+f722rRYsWxMbGEh8fT7t2v/wp4LLOV1W57777CA0NxePxeHMCf/zxx95jxMfHU6tWLd555x3AyQccGhqKiHj7CjBhwgTv/jExMVSvXp2srKwy3wNjTopfk1EBaISTxOBC4MJy9ksGripW9nfgeWAJcICS2YleA1KAr3Fy8Z7lT58sO5F/TuUsKaeSyhqnFStW6Nq1a4ukARw5cmSRNICjRo0qUS83N9ebzefQoUPavHlzTU9P14yMDG3WrJk3PWD//v31o48+8tY7ePCgdurUSTt06KBffPGFqqo+++yzOnDgQFVV3bNnj1500UV6/PhxzcvL04YNG+q+ffu8/frXv/6lqqrt2rXT5ORkVXUyBv3zn/9UVScNX15enqo66QgbNmzofd28eXNvW77KOt9FixZp9+7dtaCgQFevXq0XX3xxibqZmZlar149zcnJUVXVdevWaWpqapnHUlVduHChdunSpdz3oLKdyj93WHaik/bw99eJrgGeBIJwcuw2B74FosuoMgfoAyz1KesDjARqALWBO4vVeQ0onEXPBm4HplTUtyN5x2kxepE/p3FGeyA2n4E2ThUqa5zSxp9Y/o7LL7+ctLS0ImULFizwzjoHDBhAQkICTzxRdM1ijRo1vM9zc3MpKCgAYMeOHbRu3dqbmP6KK67grbfeolu3bgCMGTOGUaNGMXHiL7+Q8M0333i3N2rUiLp16/Lll1/Spk0bVJWcnBwuuOACDh48SGhoKAApKSlcfvnlACQmJnLVVVfxn//8h1q1ahEQ4Hx8HD16FJEiuVBKVdb5LliwgP79+yMidOzYkQMHDrB7924CAwO9defNm0ePHj2oXbs2AG3atKnweHPmzOGWW27xvi7tPTDmZPB3cdVjQEdgqzoJE7pR/j3eeUAvEakJICItcIL2p6q6DDhUvIKqLvb5JrYGaOrvSRjzR7Rnzx5vcAkMDGTv3r2l7rdz5048Hg/NmjXjwQcfJCgoiNDQULZs2UJaWhr5+fm888477Ny5E4D169ezc+dOevXqVaSduLg4FixYQH5+Pqmpqaxdu5adO3dy1llnMWXKFGJjYwkKCuKbb75hyJAhAMTExLBw4UIA3nzzTe8xAD7//HOio6OJjY1l6tSp3kAsIlx55ZW0bduWadOmVXi+6enpNGvWzLtf06ZNSU9PL9L3119/vUgQrcjhw4dZsmQJN954o991jKkq/v4eb56qZopINRGppqofl/frRO6+a4DuwAKc2e5cN6iWS0TOAvoBw8rZ5w7gDoAGDRrySGy+n6dx5mp8tjObM+Ura5x874f+Vj/99BM5OTnetvLz84u0W/y1r6SkJDIyMhgzZgyBgYHUr1+fu+66ix49elCtWjWio6M5cOAAy5cv5/7772f06NEkJydz4MAB1q5dS3Z2Nq1ateLDDz8kIiKCxo0bExERwbfffstHH33E//t//48pU6YQFBREUlISd9xxB/369WPo0KE89thjjBw5kssuu4xq1aqRnJxMdnY2AM899xzff/89Dz30EHXq1KFGjRpMmDCBBg0asH//fkaMGMGRI0eIi4sr83wzMjJYv349+fnOuO/fv9/bZ4DMzEzWrVtHrVq1SozP0aNHWbVqFeeff36R8uXLlxMREcGmTZuKlBd/DypbdnZ2lR3L/HH4G3gPiMg5wErgNRHZi/OHNMpTeLm5MPAOLn93r+eBT1R1ZVk7qOo0YBrAhSGhOukrf0/jzPVAbD42ThUra5zS+iaccNtpaWnUqVPHmxg9ODiY8PBwAgMD2b17N0FBQRUmTV+0aBEFBQUkJCSQkJDAQw89BMC0adPYtm0bbdu25ccff2T06NGAE2jGjh3LwoULadeunfdSM8Cll17KDTfcQE5ODvXq1aNv374AVK9enfHjx3v70r+/k5xs69atbN68mYSEhBIJ3mfNmkX9+vWLLKYCZ3FWXl4eCQkJZZ5vXFwcDRo08LaXk5PDtdde650dT548mZtuuokrrriixHjUqlWLyy67jAYNGhQpnzx5Mvfcc0+J8Sz+HlS24uNkDODf4iqgDs5l6QBgAHAfcEEFdc7BuR98EZBSbFsCxRZXueX/At4Bqvl7k9oWV/nnVF7kcSqpzHFKTU0tsrBnxIgRRRYbjRw5skSdnTt36uHDh1VVNSsrS8PCwnTTpk2q6iyQKiyPi4vTlJSUEvU7d+7sXVyVk5Oj2dnZqqr6wQcfaKdOnVRVNT09XZs0aeJdqPXPf/5T77///iLHOH78uPbr109ffPFFVVWdPXu2dzFVWlqaBgYG6r59+zQ7O1sPHjyoqqrZ2dl6ySWX6Pvvv1/u+b733ntFFle1b9++yDl06NBBly9fXuqYlra46sCBA1qvXj3vufoq/h5UtlP55w5bXHXSHv7v6CyousJ9Xhs41486bwAbgEeLlZcIvDiLqf4HnP1rTsACr39O5Q+AU0lljVOfPn20SZMmGhAQoMHBwTpjxgzNyMjQrl27amhoqHbt2lUzMzNVVfWLL77QIUOGqKoTIGNjY9Xj8WhsbKy+8MILRdqMjIzUyMhInTNnTqnH9Q28qamp2rp1a42IiNBu3bppWlqad78pU6ZoRESExsbGaq9evTQjI0NVVZ9++mkNCwvTsLAwffDBB7WgoEBVVf/xj39oVFSUxsXFaZs2bXT+/Pmqqrp9+3b1eDzq8Xg0KipKH3vsMe8xyjrfgoICveuuuzQkJERjYmK8/S3sc1BQkB4/frzIeU2ePFmDg4O1evXqGhgY6B0vVdWZM2fqzTff7Nd7UNlO5Z87C7wn7yHO+JdPRP6Kc0+1vqq2EpEwYKqqdqug3vXA20Ckqm5xy1YCEe6MOBMYoqpLRSQf+J5fFl69rar/rqhv4eHhmpKSUuE5nOnskpd/bJz8Y+Pkn1N5nERkraq2q3hP83vz96bf3cDFwOcAqvqdiDSqqJKqzgekWFmnMva1G5DGGGNOe/7+OlGuqh4rfCEiAfySJtAYY4wxfvI38K4QkYeAs0UkEScX77uV1y1jjDHm9ORv4B0N7AO+wvmLU4uBf1ZWp4wxxpjTVUXZiS5U1R9UtQCY7j6MMcYY8xtVNON9p/CJiLxVyX0xxhhjTnsVBV7fFckhldkRY4wx5kxQUeDVMp4bY4wx5jeo6Hdn40TkIM7M92z3Oe5rVdXzKrV3xhhjzGmm3MCr/7+9Ow+PqsoWv//dpACBNIEQwCRAkMHMA4Mgt0Wi3VEQLiKgoChBREGhFUUgastw39eLCigNamhEBqVBAZvBB/tKCylAOjQYNCpDVEwQwhASghkIGdfvj0qOFEkgDViVwPo8Tz1U7XPOPuvsqrBy9jmVJeLhqkCUUkqp60FNv06klFJKqatAE6+6br355puEhoYSFhbGgw8+yLlz55yWL1u2jJYtWxIVFUVUVBSLFy+2lnl4eFjtAwcOtNpHjBhBYGAgYWFhjB49muLiYqc+9+zZg4eHB2vXrrXapk6dSlhYGGFhYXz00UdWe+/eva19+Pn5MWjQIAAOHjxIr169aNiwIXPmzLHWT0lJsdaPioqiadOmzJs3D4Bhw4ZZ7e3btycqKgqA3bt3W+2RkZGsW7fukuMjIixevJibb76Z4OBg5s+fD8CGDRuIiIggKiqK7t2788UXXwDw9ddf06tXL0JDQ4mIiHA6xscee4zIyEgiIiIYOnSoVYNXqWuaq6syAHbg7gvaJgLvlD9vCqQDBCFv1AAAIABJREFUb9WkP61OVDO1uUqKOxw9elTat29vldy7//77ZenSpU7jtHTpUhk/fnyV2zdp0qTK9k2bNklZWZmUlZXJ8OHD5Z133rGWlZSUyB133CH9+vWTNWvWiIijJN4f//hHKS4ulry8POnWrZv88ssvlfodPHiwLF++XEQcpfp2794tL774osyePbvKOEpKSqR169ZOFYgqPPfcczJz5kwRcZQKrCjvd+zYMWnZsqUUFxdXOz4iIkuWLJGYmBirYlBF6cDc3FyrelFycrIEBgaKiEhKSop8//33IvJrCcLs7GwREadjffbZZ62ygdeK2vxzh1YnctvDHYUJVgHDgc/OaxsOTC5//v8B22raWUFxKe3jNl296K5Rk8JLGHWNjVPaq/2vaPuSkhIKCgqoX78+Z8+exc/P74pjuueee6znPXr04OjRo9brBQsWMGTIEPbs2WO17d+/nz59+mCz2bDZbERGRvJ///d/PPDAA9Y6ubm5bN26laVLlwLQqlUrWrVqxaZN1b+fW7ZsoWPHjgQEBDi1iwirV69m69atADRu3Nhadu7cOYz59RuE1Y1PfHw8zzzzDPXq1bPiAfD09LS2zc/Pt/q6+eabrXY/Pz9atWrFqVOnaNasGU2bNrXiKigocNq/Utcqd0w1rwUGGGMaAhhj2gN+wBfGmG5Aa2CzG+JS1xF/f3+ef/552rVrh6+vL15eXtx1112V1vv444+tadAjR45Y7efOnaN79+7ceuutrF+/vtJ2xcXFfPDBB/Tt2xeA9PR01q1bx7hx45zWi4yM5B//+Adnz54lMzOThIQEp/0ArFu3jj/84Q9WkqqJDz/8kAcffLBS+44dO2jdujWdO3e22v79738TGhpKeHg4CxcuxGazXXR8Dh06REJCAt27d6dfv3788MMPTrEGBQXRv39/lixZUmn/u3fvpqioiI4dO1ptjz76KDfeeCMHDx7kT3/6U42PUam6yuVnvCKSZYzZDfQFNuA42/0Ix1eU5gKPAJeq8/sEjvrA+Pi0ZFp4yW8a87WgdSPHWe+1xG63X/a2ubm5LF++nBUrVuDp6cmMGTN46aWX6NWrl9Vv8+bNWb58OQ0aNGDjxo3ce++9vPHGG4Ajsfn4+HDs2DHGjRtHfn4+/v7+Vv9z5syhQ4cOlJaWYrfbmTFjBsOGDWPHjh2cOHGCffv24ePjQ4MGDQgODiYiIoJmzZrRoUMHUlNTnY7t7bff5p577ql0vGlpaTRq1KhSe3FxMR9//DEDBgyotOzNN9+kR48eldrffvttDh8+zIsvvkiTJk0oLCyscnxiYmI4e/YsIsKcOXPYvn07Q4YMsa7zNm/enIULF5KcnMyECROYO3eutY+srCyeffZZ4uLi2L59u9UeGxvLww8/zPz585k5cyb9+vX7D97J2i0vL++KPqfq2uSuGrgV080ViXc08BTwqYgcudR0k4gsAhYBtOvQSeZ+q6V8L2VSeAnX2jiljYi+7G3XrFlDly5drBuWjh07xq5du/D09KyycHnv3r3x9vauctnmzZtp2LChtWzmzJnYbDZWr15tTccePnyY119/HYDMzEz27t1LZGQkgwYNcurzoYce4p577rHasrKy+PHHH5k6dSo33HCD037tdnuV8W7YsIGePXsyePBgp/aSkhKGDRtGUlISbdq0qXJcli1bhre3N6mpqVWOT3R0NAEBAcTExBAdHU2fPn2YO3dupRiio6OZN28eYWFh+Pj4kJOTQ3R0NHPnzuX++++vct82m43Zs2fz2muvVbm8LrLb7VV+ZtT1zV3/E68H3jDGdAUaicheY8wkoLcx5inAE2hgjMkTkbiLddSovgcpV3it73pgt9uvKFFda9q1a8euXbs4e/YsjRo1YsuWLXTv3t1pnePHj+Pr6wvAxo0bCQ4OBiA7O5vGjRvTsGFDMjMz2blzJ1OmTAFg8eLFfPbZZ2zZssVKugCpqanW81GjRjFgwAAGDRpEaWkpZ86coUWLFnzzzTd88803TlPea9asYcCAAZWS7sWsWrWqymnmzz//nKCgIKekm5qaStu2bbHZbBw+fJiUlBTat29PaWlpteMzaNAg9u7dy/Dhw9m2bZt1DffHH3+kY8eOGGPYu3cvRUVFtGjRgqKiIu677z5GjhzplHRFhEOHDtGpUydEhE8++YSgoKAaH6dSdZVbEq+I5Blj7MASHGe/iMiIiuXGmFFA90slXaUuV8+ePRk6dChdu3bFZrPRpUsXnnjiCR5//HFycnIYOHAg8+fPZ+PGjdhsNry9vVm2bBkABw4cYOzYsdSrV4+ysjLi4uIICQkBYNy4cQQEBNCrVy8ABg8ezLRp06qNo7i4mN69ewPQtGlTVqxYgc3264/lhx9+SFyc84/BiRMn6N69Ozk5OdSrV4958+axf/9+mjZtytmzZ/nnP//JX//610r7quq67xdffMGrr75K/fr1qVevHu+88w4+Pj74+PhUOT4AcXFx9O3bl/DwcDw9Pa2vWX388ce8//771K9fn0aNGvHRRx9hjGH16tVs376drKwsawyXLVtGREQEsbGx5OTkICJERkYSHx9f07dQqTrLOO4qd8OOjbkP+DsQLCIHL1g2CkfinXCpfgIDAyUlJeW3CfIaolNeNaPjVDM6TjVTm8fJGJMkIt0vvaa62tx20U9E1uFc/ej8ZcuAZa6MRymllHIF/ctVSimllAtp4lVKKaVcSBOvUkop5UKaeJVSSikX0sSrlFJKuZAmXqWUUsqFNPEqpZRSLqSJVymllHIhTbxKKaWUC2niVbVeSkoKUVFR1qNp06bMmzfPaZ2DBw/Sq1cvGjZsyJw5c5yWjR49mlatWhEWFubUfvr0aWJiYujcuTMxMTFkZ2cDjj/e//TTT9OpUyciIiLYu3evtY2Hh4cVx8CBA632xx57jMjISKt2b15eHgCFhYUMGzaMTp060bNnT9LS0gBH1aE77rgDT09PJkz49S+j5ubmOh2rj48PEydOvGhfxcXFxMbGEh4eTnBwMLNmzQIcNYN79OhBZGQkoaGhTJ8+/ZLxLly4kPDwcKKiorjtttvYv38/8GsJwoq4KsojKqUug4i49AHYgbsvaJsIvAO8BnxX/hhWk/5uvvlmUZeWkJDg7hCuipKSEmndurWkpaU5tZ88eVJ2794tL774osyePdtp2bZt2yQpKUlCQ0Od2idPniyzZs0SEZFZs2bJlClTJCEhQTZt2iR9+/aVsrIySUxMlB49eljbNGnSpMq4fvnlF+v5s88+a/X79ttvy9ixY0VEZNWqVfLAAw+IiEheXp7s2LFD4uPjZfz48dUeb9euXWXbtm0X7etvf/ubDBs2TERE8vPzJSAgQFJTU6WsrExyc3NFRKSoqEh69OghiYmJF433/PYNGzbI3XffLSIiqampTuN3rXyefmu1eZyAL8XF///rw/FwxxlvRS3e8w0HTgJdgSigJzDZGNPUxbGpWm7Lli107NiRgIAAp/ZWrVpxyy23UL9+/Urb3H777Xh7e1dq37BhA7GxsYCjGPv69eut9pEjR2KM4dZbb+XMmTMcP378onE1ber4qIoIBQUFVNSUPn8fQ4cOZcuWLYgITZo04bbbbrtoub8ffviBjIwMq3pRdX0ZY8jPz6ekpISCggIaNGhA06ZNMcbg6ekJOM6Ki4uLrbiqi7eiHSA/P59L1cZWSv3n3FEkYS3w/xtjGopIoTGmPeAHnAW2iUgJUGKMSQb6Aqsv1llBcSnt4zb9xiHXfZPCSxjlpnFKu4r1kqsqbXe5Tp48adXb9fX1JSMjA4D09HTatm1rrdemTRvS09Px9fXl3LlzdO/eHZvNRlxcnFUoHuDRRx/l008/JSQkhLlz51bqy2az4eXlRVZWFj4+PpeMb9WqVQwbNsxKftX1NXToUDZs2ICvry9nz57lzTfftH7RKC0tpVu3bvz444+MHz+enj17XjRegLfffps33niDoqIitm7darWnpqbSpUsXmjZtypAhQ2pt1R2lajuXJ14RyTLG7MaRVDfgONv9CEgGphtj3gAaA3cA+6vqwxjzBPAEgI9PS6aFl7gi9DqtdSNH8nUHu91+VfopLi7m448/ZsCAAdX2WXEt8sLlJ06cID8/36m9pKSk0uu8vDwyMzP56quvKClxjFd2djZJSUnk5eXx4Ycf4uPjw7Fjxxg3bhz5+fn4+/sDjrPmhx9+mPnz5zNz5kz69etHXl4eiYmJtGzZEnBcd925cydeXl6A49p0enp6lcezZMkSXnjhBWtZdX39/PPPZGZmsmrVKnJzc3nmmWfw9PTEz88PgHnz5pGXl8fLL79MUFAQN910U7XxAoSGhvLee+/x+eefM2HCBF544QWKiopYuXIlXl5epKSk8Oc//5mOHTvSpEmTmr1516m8vLyr9vlX1w53lQWsmG6uSLyjRWSvMeYW4F/AKSARqDJTiMgiYBFAuw6dZO63bqtuWGdMCi/BXeOUNiL6qvSzYcMGevbsyeDBg6tdx2634+npWelsLC0tjSZNmji1+/v7ExgYiK+vL8ePH8fPzw9PT08iIyPx8fGx1s3Pz2fgwIHW2XGFzZs307Bhw0r7stlszJ49m9dee43AwEDatGlDr169KCkpobCwkIEDB1pnsWlpaeTl5VXqIzk5mQYNGjB27Firrbq+JkyYQGxsLH/84x8B+OSTT7DZbJX6TEpKIisri0cffbTaeM93++2307x580r9REdHEx8fT+vWreneXcu5Xkxtrser3MddGWs98IYxpivQSET2AojIK8ArAMaYlcAPl+qoUX0PUq7iVOa1ym63X7UE6C6rVq26atPMAAMHDmT58uXExcWxfPly7r33Xqv9rbfeYvjw4fz73//Gy8sLX19fsrOzady4MQ0bNiQzM5OdO3cyZcoURIRDhw7RqVMnRIRPPvmEoKAgp3306tWLtWvXcuedd9boumlVx1pdX+3atWPr1q08/PDDnD17ll27djFx4kROnTpF/fr1adasGQUFBXz++edMnTr1ovH+8MMPdO7cGYBNmzZZz0+dOoW3tzceHh789NNPpKen06FDh6v2Xih1XXHXXV04rt1+Dcwof+0BtCh/HoHjzmbbpfrRu5prpjbfXVkT+fn54u3tLWfOnLHa4uPjJT4+XkREjh8/Lv7+/vK73/1OvLy8xN/f37pDd/jw4XLjjTeKzWYTf39/Wbx4sYiIZGZmyp133imdOnWSO++8U7KysiQhIUHKysrkqaeekg4dOkhYWJjs2bNHRER27twpYWFhEhERIWFhYVY/paWl8l//9V8SFhYmoaGh8tBDD1n7LigokKFDh0rHjh3llltukUOHDlnxBwQESPPmzaVJkybi7+8v+/bts5bddNNNcuDAAacxqK6v3NxcGTp0qISEhEhwcLC8/vrrIiKSnJwsUVFREh4eLqGhoTJz5sxLxvv0009LSEiIREZGSnR0tHz33XciIrJ27VoJCQmRiIgI6dKli7zyyitX/J5eD2rzzx16V7PbHsYx/q5njLkP+DsQLCIHjTE3ABVfmMwBxonI15fqJzAwUFJSUn7DSK8NOuVVMzpONaPjVDO1eZyMMUkiotcK3MBtF0dFZB1gznt9DghxVzxKKaWUK+hfrlJKKaVcSBOvUkop5UKaeJVSSikX0sSrlFJKuZAmXqWUUsqFNPEqpZRSLqSJVymllHIhTbxKKaWUC2niVUoppVxIE69SSinlQpp4Va105swZhg4dSlBQEMHBwSQmJjot/+WXX/jv//5vIiMjCQ0NZenSpdayn3/+mbvuuovg4GBCQkJIS0sD4K233qJTp04YY8jMzLTWt9vteHl5ERUVxZgxY/if//kfwFHvtkePHtY+pk+fbm2TmppKz5496dy5M8OGDaOoqAiA7du307VrV2w2G2vXrrXWP3z4MN26dSMqKorQ0FAWLlxoLXvppZdo27Ytnp6eVY7F2rVrMcbw5ZdfXuZoKqVqFVdXZQDswN0XtE0ElgJJOCoW7cNRJOGS/Wl1opqpzVVSqjJy5Eh59913RUSksLBQsrOznZa/8sorMmXKFBERycjIkObNm0thYaGIiPTp00c2b94sIo7KPfn5+SIisnfvXklNTZWAgAA5deqU1VdCQoL079/fel6hrKxMcnNzRUSkqKhIevToIYmJiSIicv/998uqVatERGTs2LHyzjvviIhIamqqJCcnyyOPPCJr1qyx+iosLJRz585ZMQUEBEh6erqIiCQmJsqxY8ekSZMmlcYhJydHevfuLT179rSqJNUGde3z5C61eZzQ6kRue7ijSMIqYDjw2Xltw4GpwC4RKTTGeALfGWM2isixi3VWUFxK+7hNv12014hJ4SWMcuE4pV1BjeScnBy2b9/OsmXLAGjQoAENGjRwWscYQ25uLiJCXl4e3t7e2Gw29u/fT0lJCTExMQBOZ5FdunT5j+IwxljbFxcXU1xcjDEGEWHr1q2sXLkSgNjYWGbMmMGTTz5J+/btAahXz3ky6fz4CwsLKSsrs17feuut1cbw8ssvM2XKFObMmfMfxa6Uqr3cMdW8FhhgjGkIYIxpD/gB20WksHydhm6KTdUCP/30Ey1btuTRRx+lS5cujBkzhvz8fKd1JkyYwIEDB/Dz8yM8PJy//OUv1KtXj++//55mzZoxePBgunTpwuTJkyktLb3kPhMTE4mMjGTq1Kns27fPai8tLSUqKopWrVoRExNDz549ycrKolmzZthsjt9b27RpQ3p6+iX3ceTIESIiImjbti1Tp07Fz8/vout/9dVXHDlyhAEDBlyyb6VU3eHyM14RyTLG7Ab6AhtwnO1+JCJijGkLbAI6AZOrO9s1xjwBPAHg49OSaeElrgm+DmvdyHHW6yp2u/2yt01JSSEpKYlRo0YxatQoFixYwJNPPsno0aOtdbZt24aPjw8rV67k2LFjjBkzhsWLF5OcnIzdbmfRokW0bt2amTNnEhcXR//+v56Bnzt3jp07d+Ll5QVAfn4+K1asoFGjRtjtdu6++25WrFhhrT9v3jzy8vJ4+eWXCQoKonnz5hQUFFjHmJGRwdmzZ52O+cSJE+zbtw8fHx+nY5s/fz6ZmZm8/PLL+Pr64u3tbS0rLS21+igrK+O5554jLi4Ou93OmTNnSEpKIi8v77LH9WrKy8u7ovf4eqHjpKrirnq8FdPNFYl3NICIHAEijDF+wHpjzFoROXnhxiKyCFgE0K5DJ5n7rdvKCtcZk8JLcOU4pY2Ivuxtg4KCmDVrFk899RQAHh4evPrqq04FxWfPnk1cXBy9e/cG4L333qNly5bcfffdJCQk8NBDDwFw7Ngxdu3a5bTtDTfcwO9///tKSbHCsmXLCAsLq7Q8KSmJrKwsRo0axZgxY7jtttuw2WwkJibSuXNnp30sW7aM0NDQaougb9q0ibKyMqflHh4e1utffvmFo0ePEhcXBzgS+cyZM9m4cSPdu7u/dnltLvBem+g4qaq4K2OtB94wxnQFGonI3vMXisgxY8w+oDeOqelqNarvQcoVXE+8Xtjt9itKhq5044030rZtW1JSUggMDGTLli2EhIQ4rdOuXTu2bNlC7969OXnyJCkpKXTo0IHmzZuTnZ3NqVOnaNmyJVu3br1kojpx4gStW7fGGMOBAwcoKyujRYsWnDp1ivr169OsWTMKCgr4/PPPmTp1KsYY7rjjDtauXcvw4cNZvnw5995770X3cfToUVq0aEGjRo3Izs5m586dPPfcc9Wu7+Xl5XTndXR0NHPmzKkVSVcpdYXcdVcXsBrHHcwzyl+3wZGEAZoD3wPhl+pH72qumdp8d2VVvvrqK+nWrZuEh4fLvffeK6dPn5b4+HiJj48XEZH09HSJiYmRsLAwCQ0NlQ8++MDadvPmzRIeHi5hYWESGxtr3e38l7/8Rfz9/cXDw0N8fX3lscceExGRBQsWSEhIiEREREhwcLDs3LlTRESSk5MlKipKwsPDJTQ0VGbOnGnt49ChQ3LLLbdIx44dZejQodYdy7t37xZ/f39p3LixeHt7S0hIiFNMEREREh4eLn/961+tviZPniz+/v5ijBF/f3+ZPn16pfHo06eP3tVcB9XmcULvanbbwzjG3/WMMfcBfweCReSgMSYGmAsIYIC3xDGlfFGBgYGSkpLy2wZ7DdApr5rRcaoZHaeaqc3jZIxJEhGdQnEDt10cFZF1OBJsxet/AhHuikcppZRyBf3KjlJKKeVCmniVUkopF9LEq5RSSrmQJl6llFLKhTTxKqWUUi6kiVcppZRyIU28SimllAtp4lVKKaVcSBOvUkop5UKaeJVSSikX0sSr3OrMmTMMHTqUoKAggoODSUxMrHK9PXv24OHhwdq1jmJVhw8fplu3bkRFRREaGsrChQsByM3NJSoqynr4+PgwceJEALZv307Xrl2x2WxWPxV+/vln7rrrLmJjYwkJCSEtLQ2AESNGEBgYSFhYGKNHj6a4uPiicVXIycnB39+fCRMmWG1JSUmEh4fTqVMnnn766YpiIUyePJmgoCAiIiK47777OHPmDABpaWk0atTIOpZx48ZZfa1atYrw8HAiIiLo27evVcno9OnTxMTE0LlzZ2JiYsjOzra2sdvt1nj16dPHKd7S0lK6dOnCgAEDqnurlFJXi6urMgB24O4L2iYCB3BUK6p4nAMGXao/rU5UM7W1SsrIkSPl3XffFRGRwsJCyc7OrrROSUmJ3HHHHdKvXz9Zs2aNtW5FRaDc3FwJCAiQ9PT0Stt27dpVtm3bJiIiqampkpycLI888ojVT4U+ffrI5s2bJSEhQXJzcyU/P19ERDZt2iRlZWVSVlYmw4cPl3feeeeicVV4+umn5cEHH5Tx48dbbbfccov861//krKyMunbt698+umnIiLy2WefSXFxsYiITJkyRaZMmWLFGxoaWumYiouLpWXLlnLq1CkRcVQ3qqhoNHnyZJk1a5aIiMyaNcvqKzs7W4KDg+Xw4cMiInLy5EmnPufOnSsPPvig9O/fv9L+qlJbP0+1TW0eJ7Q6kdse7iiSsAoYDnx2Xttw4AkR2QFgjPEGfgQ2X6qzguJS2sdt+i3ivKZMCi9h1FUep7QrrIOck5PD9u3bWbZsGQANGjSgQYMGldZbsGABQ4YMYc+ePVbb+esVFhZSVlZWabsffviBjIwMevfuDUD79u0BqFfPeaJn//79lJSUEBMTg91ux9PT01p2zz33WM979OjB0aNHLxoXOM5sT548Sd++ffnyyy8BOH78ODk5OfTq1QuAkSNHsn79evr168ddd91lbXvrrbdWOnu+UMUPb35+Pi1atCAnJ4dOnToBsGHDBux2OwCxsbFER0fz2muvsXLlSgYPHky7du0AaNWqldXf0aNH2bRpEy+99BJvvPHGRfetlLpy7phqXgsMMMY0BDDGtAf8gC/OW2co8A8ROevy6JTL/PTTT7Rs2ZJHH32ULl26MGbMGPLz853WSU9PZ926dU7TrBWOHDlCREQEbdu2ZerUqfj5+TktX7VqFcOGDcMYU2nb833//fc0a9aMwYMH8/jjjzN58mRKS0ud1ikuLuaDDz6gb9++F42rrKyMSZMmMXv27ErH0aZNG+t1mzZtSE9PrxTLkiVL6Nevn/U6NTWVLl260KdPH3bs2AFA/fr1iY+PJzw8HD8/P/bv389jjz0GwMmTJ/H19QXA19eXjIwM6xizs7OJjo6mW7duvP/++9Y+Jk6cyOuvv17pFxKl1G/D5We8IpJljNkN9AU24Djb/ah86qPCcKDaX72NMU8ATwD4+LRkWnjJbxjxtaF1I8dZ79VUcWZ1uVJSUkhKSmLUqFGMGjWKBQsW8OSTTzJ69GhrnRkzZjBs2DB27NjBiRMn2LdvHz4+Ptby+fPnk5mZycsvv4yvry/e3t7WsiVLlvDCCy9UivPCfpKTk7Hb7SxatIjY2Fjmzp1LXFwc/fv/ekY/Z84cOnToQGlpKXa7vdq41q1bR2BgIIcOHeLgwYOkp6djt9s5ePAg2dnZVizffPMNp0+fdoptxYoVnDlzBn9/f+x2O0VFRaxcuRIvLy9SUlIYMmQIS5cupWHDhvzv//4v8fHx+Pn5MX/+fJ544gkeeeQRSkpKnPqseH348GFSUlKYO3cuRUVFjB8/HmMMR48epbi4mNzcXL7++muysrJq9L7m5eVd8ft/PdBxUlVxVz3eiunmisRr/U9rjPEFwnGeinYiIouARQDtOnSSud+6raxwnTEpvISrPU5pI6KvaPugoCBmzZrFU089BYCHhwevvvqqU+Hww4cP8/rrrwOQmZnJ3r17iYyMZNCgQU59bdq0ibKyMmvb5ORkGjRowNixYyvtd9myZYSGhlrr3nDDDSQkJPDQQw9ht9t5/PHH2bVrl7V85syZ2Gw2Vq9ebZ0VVhdXZmYmO3bs4LPPPiMvL4+ioiICAwN55plnmDdvntXn8ePHCQ8Pt14vX76cffv2sWXLFho3blwp5ujoaFatWkXr1q0REZo3b86IESMqjZu/vz+BgYH4+vpy/Phx/Pz8iI6OZteuXURGRlpn0xs3buSGG24gJyfH+uXn3Llz5OTksHjxYlasWHHR9642F3ivTXScVJXccWEZ8AQygK5AygXLngEW1bQvvbmqZmrrTR633XabHDx4UEREpk+fLs8//3y168bGxlo3MR05ckTOnj0rIiKnT5+Wzp07yzfffGOtO3XqVJk2bdol+xFx3CQVEREhGRkZkpCQIKNGjZK33npLRETeffdd6dWrl7WvmvRXYenSpU43V3Xv3l0SExOtm6s2bdokIiL/+Mc/JDg4WDIyMpy2z8jIkJKSEhEROXTokPj5+UlWVpakp6fLjTfeaK3/5z//WZ577jkREXn++eedbq6aPHmyiIjs379f7rzzTikuLpb8/HwJDQ2Vb7/91ml/CQkJenPVVVZSWli6AAAGxUlEQVSbxwm9ucptD7ecKopInjHGDizBcfZ7vgeBF1welHKLBQsWMGLECIqKiujQoQNLly61vhpU1XXdCgcOHGDSpEkYYxARnn/+ecLDw63lq1ev5tNPP3XaZs+ePdx3331kZ2fzySefMH36dPbt24eHhwdz5szhD3/4A3l5edx+++08/vjjVgwBAQHWTVGDBw9m2rRpl3Ws8fHxjBo1ioKCAvr162edfU6YMIHCwkJiYmIAxw1WCxcuZPv27UybNg2bzYaHhwcLFy60ptKnT5/O7bffTv369QkICLBuUIuLi+OBBx7gvffeo127dqxZswaA4OBg+vbtS0REBPXq1WPMmDGEhYVd1nEopa6Mcfzi44YdG3Mf8HcgWEQOlre1B3YCbUWk8m2qVQgMDJSUlJTfKsxrhk551YyOU83oONVMbR4nY0ySiHR3dxzXI7ddHBWRdYC5oC0N8HdLQEoppZQL6PcHlFJKKRfSxKuUUkq5kCZepZRSyoU08SqllFIupIlXKaWUciFNvEoppZQLaeJVSimlXEgTr1JKKeVCmniVUkopF9LEq5RSSrmQJl6llFLKhTTxKqWUUi6kiVcppZRyIU28SimllAu5rR7v1WKMyQW0IO+l+QCZ7g6iDtBxqhkdp5qpzeMUICIt3R3E9cht9XivohQt5nxpxpgvdZwuTcepZnScakbHSVVFp5qVUkopF9LEq5RSSrnQtZB4F7k7gDpCx6lmdJxqRsepZnScVCV1/uYqpZRSqi65Fs54lVJKqTpDE69SSinlQnU28Rpj+hpjUowxPxpj4twdT21mjEkzxnxrjPnaGPOlu+OpLYwxS4wxGcaY785r8zbG/NMY80P5v83dGWNtUM04zTDGpJd/pr42xtzjzhhrA2NMW2NMgjHmgDFmnzHmmfJ2/UwpJ3Uy8RpjPIC3gX5ACPCgMSbEvVHVeneISJR+p9DJMqDvBW1xwBYR6QxsKX99vVtG5XECeLP8MxUlIp+6OKbaqASYJCLBwK3A+PL/l/QzpZzUycQL9AB+FJGfRKQI+BC4180xqTpGRLYDpy9ovhdYXv58OTDIpUHVQtWMk7qAiBwXkb3lz3OBA4A/+plSF6iridcfOHLe66PlbapqAmw2xiQZY55wdzC1XGsROQ6O/0iBVm6OpzabYIz5pnwqWqdPz2OMaQ90Af6NfqbUBepq4jVVtOn3oqr3exHpimNqfrwx5nZ3B6TqvHigIxAFHAfmujec2sMY4wl8DEwUkRx3x6Nqn7qaeI8Cbc973QY45qZYaj0ROVb+bwawDsdUvaraSWOML0D5vxlujqdWEpGTIlIqImXAu+hnCgBjTH0cSfdvIvL38mb9TCkndTXx7gE6G2NuMsY0AIYDG90cU61kjGlijPldxXPgLuC7i291XdsIxJY/jwU2uDGWWqsikZS7D/1MYYwxwHvAARF547xF+plSTursX64q//rCPMADWCIir7g5pFrJGNMBx1kuOKpRrdSxcjDGrAKicZRuOwlMB9YDq4F2wM/A/SJyXd9YVM04ReOYZhYgDRhbcR3zemWMuQ3YAXwLlJU3v4jjOq9+ppSlziZepZRSqi6qq1PNSimlVJ2kiVcppZRyIU28SimllAtp4lVKKaVcSBOvUkop5UI2dwegVG1kjCnF8bWQCoNEJM1N4SilriH6dSKlqmCMyRMRTxfuzyYiJa7an1LKfXSqWanLYIzxNcZsL69F+50xpnd5e19jzF5jTLIxZkt5m7cxZn15QYFdxpiI8vYZxphFxpjNwPvGGA9jzGxjzJ7ydce68RCVUr8RnWpWqmqNjDFflz9PFZH7Llj+EPCZiLxSXh+6sTGmJY6/W3y7iKQaY7zL150JfCUig4wxdwLv4/irTwDdgNtEpKC8ctQvInKLMaYhsNMYs1lEUn/LA1VKuZYmXqWqViAiURdZvgdYUv5H8deLyNfGmGhge0WiPO/PAt4GDClv22qMaWGM8SpftlFECsqf3wVEGGOGlr/2AjoDmniVuoZo4lXqMojI9vLyiv2BD4wxs4EzVF2e8mJlLPMvWO9PIvLZVQ1WKVWr6DVepS6DMSYAyBCRd3FUpOkKJAJ9jDE3la9TMdW8HRhR3hYNZFZTp/Uz4Mnys2iMMTeXV5RSSl1D9IxXqcsTDUw2xhQDecBIETlVfp3278aYejjqrsYAM4ClxphvgLP8WiLuQouB9sDe8hJzp4BBv+VBKKVcT79OpJRSSrmQTjUrpZRSLqSJVymllHIhTbxKKaWUC2niVUoppVxIE69SSinlQpp4lVJKKRfSxKuUUkq50P8DvDJICq14xeYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from xgboost import plot_importance\n",
    "from xgboost import plot_tree\n",
    "\n",
    "model = opt_xgb.best_estimator_\n",
    " \n",
    "plot_importance(model, importance_type='gain', max_num_features=10); "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Conclusion\n",
    "\n",
    "Machine learning algorithms such as XGBoost based on decision trees can be used to detect fraud in finance with a high accuracy (more than 95%). \n"
   ]
  }
 ],
 "metadata": {
  "hide_input": false,
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}