NB_CIX-DNN_mnist_Keras.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Notebook 11: Introduction to Deep Neural Networks with Keras"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Learning Goals\n",
    "The goal of this notebook is to introduce deep neural networks (DNNs) using the high-level Keras package. The reader will become familiar with how to choose an architecture, cost function, and optimizer in Keras. We will also learn how to train neural networks.\n",
    "\n",
    "\n",
    "# MNIST with Keras\n",
    "\n",
    "We will once again work with the MNIST dataset of hand written digits introduced in *Notebook 7: Logistic Regression (MNIST)*. The goal is to find a statistical model which recognizes and distinguishes between the ten handwritten digits (0-9).\n",
    "\n",
    "The MNIST dataset comprises $70000$ handwritten digits, each of which comes in a square image, divided into a $28\\times 28$ pixel grid. Every pixel can take on $256$ nuances of the gray color, interpolating between white and black, and hence each data point assumes any value in the set $\\{0,1,\\dots,255\\}$. Since there are $10$ categories in the problem, corresponding to the ten digits, this problem represents a generic classification task. \n",
    "\n",
    "In this Notebook, we show how to use the Keras python package to tackle the MNIST problem with the help of deep neural networks.\n",
    "\n",
    "The following code is a slight modification of a Keras tutorial, see [https://github.com/fchollet/keras/blob/master/examples/mnist_cnn.py](https://github.com/fchollet/keras/blob/master/examples/mnist_cnn.py). We invite the reader to read Sec. IX of the review to acquire a broad understanding of what the separate parts of the code do."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from __future__ import print_function\n",
    "import keras,sklearn\n",
    "# suppress tensorflow compilation warnings\n",
    "import os\n",
    "os.environ['KMP_DUPLICATE_LIB_OK']='True'\n",
    "import tensorflow as tf\n",
    "os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'\n",
    "import numpy as np\n",
    "seed=0\n",
    "np.random.seed(seed) # fix random seed\n",
    "\n",
    "tf.random.set_seed(seed)\n",
    "\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Structure of the Procedure\n",
    "\n",
    "Constructing a Deep Neural Network to solve ML problems is a multiple-stage process. Quite generally, one can identify the key steps as follows:\n",
    "\n",
    "* ***step 1:*** Load and process the data\n",
    "* ***step 2:*** Define the model and its architecture\n",
    "* ***step 3:*** Choose the optimizer and the cost function\n",
    "* ***step 4:*** Train the model \n",
    "* ***step 5:*** Evaluate the model performance on the *unseen* test data\n",
    "* ***step 6:*** Modify the hyperparameters to optimize performance for the specific data set\n",
    "\n",
    "We would like to emphasize that, while it is always possible to view steps 1-5 as independent of the particular task we are trying to solve, it is only when they are put together in ***step 6*** that the real gain of using Deep Learning is revealed, compared to less sophisticated methods such as the regression models or bagging, described in Secs. VII and VIII of the review. With this remark in mind, we shall focus predominantly on steps 1-5 below. We show how one can use grid search methods to find optimal hyperparameters in ***step 6***."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 1: Load and Process the Data\n",
    "\n",
    "Keras can conveniently download the MNIST data from the web. All we need to do is import the `mnist` module and use the `load_data()` class, and it will create the training and test data sets or us.\n",
    "\n",
    "The MNIST set has pre-defined test and training sets, in order to facilitate the comparison of the performance of different models on the data.\n",
    "\n",
    "Once we have loaded the data, we need to format it in the correct shape. This differs from one package to the other and, as we see in the case of Keras, it can even be different depending on the backend used.\n",
    "\n",
    "While choosing the correct `datatype` can help improve the computational speed, we emphasize the rescaling step, which is necessary to avoid large variations in the minimal and maximal possible values of each feature. In other words, we want to make sure a feature is not being over-represented just because it is \"large\".\n",
    "\n",
    "Last, we cast the label vectors $y$ to binary class matrices (a.k.a. one-hot format), as explained in Sec. VII on SoftMax regression."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "an example of a data point with label [0. 0. 0. 0. 1. 0. 0. 0. 0. 0.]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaMAAAGkCAYAAACckEpMAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAapUlEQVR4nO3dbWxT5xnG8cvlxQWWuFCa2BlplFHQUGFMBRZALS/ViMhUxEunUapNibShMl4kFlC1FG2EVSIdG6gfWLuVbQw0GGwapUigQipIANF0gFKBWMXSEUYQSTMQtQNljijPPkS4dROgx9i5Y+f/k46Kj8+dc/PoqS+e+PjY55xzAgDA0APWDQAAQBgBAMwRRgAAc4QRAMAcYQQAMEcYAQDMEUYAAHOEEQDAHGEEADBHGAEAzKVVGL322msqLCzUgw8+qHHjxunIkSPWLXWryspK+Xy+uC0YDFq31S0OHz6sWbNmKS8vTz6fT7t374573jmnyspK5eXlacCAAZo2bZrOnDlj02wK3WscysrKOs2RiRMn2jSbQlVVVZowYYKysrKUk5OjOXPm6OzZs3HH9IY58WXGIV3mRNqE0c6dO7V8+XKtWrVK9fX1euqpp1RSUqILFy5Yt9atHn/8cTU3N8e206dPW7fULa5fv66xY8dq48aNXT6/bt06bdiwQRs3btTx48cVDAY1Y8YMtbW1dXOnqXWvcZCkmTNnxs2Rffv2dWOH3aO2tlZLlixRXV2dqqurdfPmTRUXF+v69euxY3rDnPgy4yClyZxwaeJb3/qWW7RoUdy+r3/96+6nP/2pUUfdb/Xq1W7s2LHWbZiT5N58883Y41u3brlgMOheeeWV2L7//e9/LhAIuN/+9rcGHXaPL46Dc86Vlpa62bNnm/RjqbW11UlytbW1zrneOye+OA7Opc+cSIuVUXt7u06ePKni4uK4/cXFxTp27JhRVzYaGhqUl5enwsJCPffcczp37px1S+YaGxvV0tISNz/8fr+mTp3a6+aHJNXU1CgnJ0cjR47UwoUL1draat1SyoXDYUnSkCFDJPXeOfHFcbgtHeZEWoTR5cuX9emnnyo3Nzduf25urlpaWoy66n5FRUXaunWr9u/fr02bNqmlpUWTJ0/WlStXrFszdXsO9Pb5IUklJSXatm2bDh48qPXr1+v48eN6+umnFY1GrVtLGeecysvL9eSTT2r06NGSeuec6GocpPSZE32tG/DC5/PFPXbOddqXyUpKSmJ/HjNmjCZNmqThw4dry5YtKi8vN+ysZ+jt80OS5s+fH/vz6NGjNX78eBUUFGjv3r2aN2+eYWeps3TpUp06dUpHjx7t9FxvmhN3God0mRNpsTIaOnSo+vTp0+lfNK2trZ3+5dObDBo0SGPGjFFDQ4N1K6ZuX1HI/OgsFAqpoKAgY+fIsmXLtGfPHh06dEjDhg2L7e9tc+JO49CVnjon0iKM+vfvr3Hjxqm6ujpuf3V1tSZPnmzUlb1oNKoPPvhAoVDIuhVThYWFCgaDcfOjvb1dtbW1vXp+SNKVK1fU1NSUcXPEOaelS5dq165dOnjwoAoLC+Oe7y1z4l7j0JUeOycML57wZMeOHa5fv37uD3/4g/vnP//pli9f7gYNGuTOnz9v3Vq3WbFihaupqXHnzp1zdXV17plnnnFZWVm9Ygza2tpcfX29q6+vd5Lchg0bXH19vfvPf/7jnHPulVdecYFAwO3atcudPn3aLViwwIVCIReJRIw7T667jUNbW5tbsWKFO3bsmGtsbHSHDh1ykyZNcl/96lczbhx+/OMfu0Ag4Gpqalxzc3Ns++STT2LH9IY5ca9xSKc5kTZh5Jxzv/nNb1xBQYHr37+/e+KJJ+IuX+wN5s+f70KhkOvXr5/Ly8tz8+bNc2fOnLFuq1scOnTISeq0lZaWOuc6LuVdvXq1CwaDzu/3uylTprjTp0/bNp0CdxuHTz75xBUXF7tHHnnE9evXzz366KOutLTUXbhwwbrtpOtqDCS5zZs3x47pDXPiXuOQTnPC55xz3bcOAwCgs7R4zwgAkNkIIwCAOcIIAGCOMAIAmCOMAADmCCMAgLm0CqNoNKrKysoed4M/C4xFB8ahA+PwGcaiQ7qNQ1p9zigSiSgQCCgcDis7O9u6HVOMRQfGoQPj8BnGokO6jUNarYwAAJmJMAIAmOtx32d069YtXbp0SVlZWZ2+dyQSicT9tzdjLDowDh0Yh88wFh16wjg459TW1qa8vDw98MDd1z497j2jixcvKj8/37oNAECSNDU13fN7lnrcyigrK0tSR/Pp8KYbAKBrkUhE+fn5sdf1u+lxYXT7V3PZ2dmEEQBkgC/zVe8pu4DhtddeU2FhoR588EGNGzdOR44cSdWpAABpLiVhtHPnTi1fvlyrVq1SfX29nnrqKZWUlOjChQupOB0AIM2l5AKGoqIiPfHEE3r99ddj+0aNGqU5c+aoqqrqrrXp9kEtAEDXvLyeJ31l1N7erpMnT6q4uDhuf3FxsY4dO9bp+Gg0qkgkErcBAHqXpIfR5cuX9emnnyo3Nzduf25urlpaWjodX1VVpUAgENu4rBsAep+UXcDwxasnnHNdXlFRUVGhcDgc25qamlLVEgCgh0r6pd1Dhw5Vnz59Oq2CWltbO62WJMnv98vv9ye7DQBAGkn6yqh///4aN26cqqur4/ZXV1dr8uTJyT4dACADpORDr+Xl5frBD36g8ePHa9KkSXrjjTd04cIFLVq0KBWnAwCkuZSE0fz583XlyhX94he/UHNzs0aPHq19+/apoKAgFacDAKS5HnejVD5nBACZwfRzRgAAeEUYAQDMEUYAAHOEEQDAHGEEADBHGAEAzBFGAABzhBEAwBxhBAAwRxgBAMwRRgAAc4QRAMAcYQQAMEcYAQDMEUYAAHOEEQDAHGEEADBHGAEAzBFGAABzhBEAwBxhBAAwRxgBAMwRRgAAc4QRAMAcYQQAMEcYAQDMEUYAAHOEEQDAHGEEADBHGAEAzBFGAABzhBEAwBxhBAAwRxgBAMwRRgAAc4QRAMAcYQQAMEcYAQDMEUYAAHN9rRsA0DP961//8lyzaNEizzXbtm3zXCNJoVAooTr0TKyMAADmCCMAgDnCCABgjjACAJgjjAAA5ggjAIA5wggAYI4wAgCYI4wAAOYIIwCAOcIIAGCOMAIAmONGqQloa2vzXHPt2jXPNYFAwHONJA0cODChOuDz9u3b57mmtrbWc83vf/97zzWSVFFR4bmmb19e8noqVkYAAHOEEQDAXNLDqLKyUj6fL24LBoPJPg0AIIOk5Beojz/+uN55553Y4z59+qTiNACADJGSMOrbty+rIQDAl5aS94waGhqUl5enwsJCPffcczp37twdj41Go4pEInEbAKB3SXoYFRUVaevWrdq/f782bdqklpYWTZ48WVeuXOny+KqqKgUCgdiWn5+f7JYAAD1c0sOopKREzz77rMaMGaNvf/vb2rt3ryRpy5YtXR5fUVGhcDgc25qampLdEgCgh0v5J8AGDRqkMWPGqKGhocvn/X6//H5/qtsAAPRgKf+cUTQa1QcffKBQKJTqUwEA0lTSw2jlypWqra1VY2Oj3nvvPX33u99VJBJRaWlpsk8FAMgQSf813cWLF7VgwQJdvnxZjzzyiCZOnKi6ujoVFBQk+1QAgAyR9DDasWNHsn8kACDDcQvbBPzyl7/0XFNVVeW55te//rXnGkn6yU9+klAd8Hnjxo3rlvNUVlYmVLdgwQLPNY899lhC50LqcaNUAIA5wggAYI4wAgCYI4wAAOYIIwCAOcIIAGCOMAIAmCOMAADmCCMAgDnCCABgjjACAJgjjAAA5rhRag+2Zs2ahOq+9rWvea6ZPXt2QudC5vroo4+sW0AvwsoIAGCOMAIAmCOMAADmCCMAgDnCCABgjjACAJgjjAAA5ggjAIA5wggAYI4wAgCYI4wAAOYIIwCAOW6U2oO1tbUlVFdWVua5prq62nPN+PHjPdfAxrVr1zzXrF+/PgWdJM9f//pXzzUvvfRSCjpBMrAyAgCYI4wAAOYIIwCAOcIIAGCOMAIAmCOMAADmCCMAgDnCCABgjjACAJgjjAAA5ggjAIA5wggAYI4wAgCY467dCSgsLLRu4a4ikYjnmp///Oeea7Zt2+a5RpIGDx6cUB0S19DQ4LnmH//4Rwo6AbrGyggAYI4wAgCYI4wAAOYIIwCAOcIIAGCOMAIAmCOMAADmCCMAgDnCCABgjjACAJgjjAAA5ggjAIA5bpSagLKyMs81ly5d8lxTWVnpuSZR+/fv91zz97//PaFz/ehHP0qoDonLzc31XDN8+HDPNf/+97891yTqe9/7XredC6nHyggAYI4wAgCY8xxGhw8f1qxZs5SXlyefz6fdu3fHPe+cU2VlpfLy8jRgwABNmzZNZ86cSVa/AIAM5DmMrl+/rrFjx2rjxo1dPr9u3Tpt2LBBGzdu1PHjxxUMBjVjxgy1tbXdd7MAgMzk+QKGkpISlZSUdPmcc06vvvqqVq1apXnz5kmStmzZotzcXG3fvl0vvPDC/XULAMhISX3PqLGxUS0tLSouLo7t8/v9mjp1qo4dO9ZlTTQaVSQSidsAAL1LUsOopaVFUufLSHNzc2PPfVFVVZUCgUBsy8/PT2ZLAIA0kJKr6Xw+X9xj51ynfbdVVFQoHA7HtqamplS0BADowZL6oddgMCipY4UUCoVi+1tbW+/4oTu/3y+/35/MNgAAaSapK6PCwkIFg0FVV1fH9rW3t6u2tlaTJ09O5qkAABnE88ro2rVr+vDDD2OPGxsb9f7772vIkCF69NFHtXz5cq1du1YjRozQiBEjtHbtWg0cOFDPP/98UhsHAGQOz2F04sQJTZ8+Pfa4vLxcklRaWqo//elPevHFF3Xjxg0tXrxYV69eVVFRkQ4cOKCsrKzkdQ0AyCg+55yzbuLzIpGIAoGAwuGwsrOzrdtJmnA47LmmqKgooXM1NDQkVOfVN77xjYTq3nnnHc81Dz/8cELnQof6+nrPNePHj09BJ8lz9uxZzzWPPfZYCjrBnXh5PefedAAAc4QRAMAcYQQAMEcYAQDMEUYAAHOEEQDAHGEEADBHGAEAzBFGAABzhBEAwBxhBAAwRxgBAMwl9cv1cGeBQMBzTaLfAdVdN0o9depUQnWJfJtvT79Rant7u+ea3/3udynopGt/+9vfuu1cQCJYGQEAzBFGAABzhBEAwBxhBAAwRxgBAMwRRgAAc4QRAMAcYQQAMEcYAQDMEUYAAHOEEQDAHGEEADBHGAEAzHHX7h4s0bt2b9myJcmdJNe7777rueab3/ym55pjx451S40kXbt2zXPNyy+/nNC5Ms2oUaMSqhs8eHCSO4ElVkYAAHOEEQDAHGEEADBHGAEAzBFGAABzhBEAwBxhBAAwRxgBAMwRRgAAc4QRAMAcYQQAMEcYAQDM+ZxzzrqJz4tEIgoEAgqHw8rOzrZuJy19//vf91yzffv2FHTSeyTyv5HP50tBJ73Hpk2bPNf88Ic/TEEnuBMvr+esjAAA5ggjAIA5wggAYI4wAgCYI4wAAOYIIwCAOcIIAGCOMAIAmCOMAADmCCMAgDnCCABgjjACAJjra90Akm/FihWea/7yl7+koBPcDTdKvT91dXWea7hRas/FyggAYI4wAgCY8xxGhw8f1qxZs5SXlyefz6fdu3fHPV9WViafzxe3TZw4MVn9AgAykOcwun79usaOHauNGzfe8ZiZM2equbk5tu3bt+++mgQAZDbPFzCUlJSopKTkrsf4/X4Fg8GEmwIA9C4pec+opqZGOTk5GjlypBYuXKjW1tY7HhuNRhWJROI2AEDvkvQwKikp0bZt23Tw4EGtX79ex48f19NPP61oNNrl8VVVVQoEArEtPz8/2S0BAHq4pH/OaP78+bE/jx49WuPHj1dBQYH27t2refPmdTq+oqJC5eXlsceRSIRAAoBeJuUfeg2FQiooKFBDQ0OXz/v9fvn9/lS3AQDowVL+OaMrV66oqalJoVAo1acCAKQpzyuja9eu6cMPP4w9bmxs1Pvvv68hQ4ZoyJAhqqys1LPPPqtQKKTz58/rpZde0tChQzV37tykNg4AyByew+jEiROaPn167PHt93tKS0v1+uuv6/Tp09q6das+/vhjhUIhTZ8+XTt37lRWVlbyugYAZBTPYTRt2jQ55+74/P79+++rIQBA78Ndu4EkGDFihOeaRO7a/Z3vfMdzjSQ99NBDnmvWrFmT0LmARHCjVACAOcIIAGCOMAIAmCOMAADmCCMAgDnCCABgjjACAJgjjAAA5ggjAIA5wggAYI4wAgCYI4wAAOa4USrSwsMPP+y5JpGvr1+5cqXnGklasGBBQnXdpb6+3nMNN0pFd2JlBAAwRxgBAMwRRgAAc4QRAMAcYQQAMEcYAQDMEUYAAHOEEQDAHGEEADBHGAEAzBFGAABzhBEAwBw3Ss1Aw4cP91xTWlrquebcuXOeayRp1KhRnmsWL17suWbMmDGea5A+Dhw44Lnm6tWrnmsGDx7suQbesTICAJgjjAAA5ggjAIA5wggAYI4wAgCYI4wAAOYIIwCAOcIIAGCOMAIAmCOMAADmCCMAgDnCCABgjhulZqDs7GzPNX/84x9T0AmQOhcvXvRc097enoJOkAysjAAA5ggjAIA5wggAYI4wAgCYI4wAAOYIIwCAOcIIAGCOMAIAmCOMAADmCCMAgDnCCABgjjACAJgjjAAA5rhrN9ALPPTQQ55rQqGQ55rm5mbPNd2poqLCc80bb7yR0Ln69uXl1QtWRgAAc4QRAMCcpzCqqqrShAkTlJWVpZycHM2ZM0dnz56NO8Y5p8rKSuXl5WnAgAGaNm2azpw5k9SmAQCZxVMY1dbWasmSJaqrq1N1dbVu3ryp4uJiXb9+PXbMunXrtGHDBm3cuFHHjx9XMBjUjBkz1NbWlvTmAQCZwdM7bG+//Xbc482bNysnJ0cnT57UlClT5JzTq6++qlWrVmnevHmSpC1btig3N1fbt2/XCy+80OlnRqNRRaPR2ONIJJLI3wMAkMbu6z2jcDgsSRoyZIgkqbGxUS0tLSouLo4d4/f7NXXqVB07dqzLn1FVVaVAIBDb8vPz76clAEAaSjiMnHMqLy/Xk08+qdGjR0uSWlpaJEm5ublxx+bm5sae+6KKigqFw+HY1tTUlGhLAIA0lfCF8EuXLtWpU6d09OjRTs/5fL64x865Tvtu8/v98vv9ibYBAMgACa2Mli1bpj179ujQoUMaNmxYbH8wGJSkTqug1tbWTqslAABu8xRGzjktXbpUu3bt0sGDB1VYWBj3fGFhoYLBoKqrq2P72tvbVVtbq8mTJyenYwBAxvH0a7olS5Zo+/bteuutt5SVlRVbAQUCAQ0YMEA+n0/Lly/X2rVrNWLECI0YMUJr167VwIED9fzzz6fkLwAASH+ewuj111+XJE2bNi1u/+bNm1VWViZJevHFF3Xjxg0tXrxYV69eVVFRkQ4cOKCsrKykNAwAyDw+55yzbuLzIpGIAoGAwuGwsrOzrdsBeq333nvPc83cuXM913z00Ueea7pTop99HDRoUJI7ST9eXs+5Nx0AwBxhBAAwRxgBAMwRRgAAc4QRAMAcYQQAMEcYAQDMEUYAAHOEEQDAHGEEADBHGAEAzBFGAABzCX/TK4DMVlRU5Lnmrbfe8lwza9YszzWS9N///jehOq9OnDiRUN3UqVOT3ElmY2UEADBHGAEAzBFGAABzhBEAwBxhBAAwRxgBAMwRRgAAc4QRAMAcYQQAMEcYAQDMEUYAAHOEEQDAHGEEADDHXbsBJM2ECRM812zYsCGhc/3qV7/yXPPMM894rhk/frznGnjHyggAYI4wAgCYI4wAAOYIIwCAOcIIAGCOMAIAmCOMAADmCCMAgDnCCABgjjACAJgjjAAA5ggjAIA5n3POWTfxeZFIRIFAQOFwWNnZ2dbtAAAS5OX1nJURAMAcYQQAMEcYAQDMEUYAAHOEEQDAHGEEADBHGAEAzBFGAABzhBEAwBxhBAAwRxgBAMwRRgAAc4QRAMAcYQQAMEcYAQDMeQqjqqoqTZgwQVlZWcrJydGcOXN09uzZuGPKysrk8/nitokTJya1aQBAZvEURrW1tVqyZInq6upUXV2tmzdvqri4WNevX487bubMmWpubo5t+/btS2rTAIDM0tfLwW+//Xbc482bNysnJ0cnT57UlClTYvv9fr+CwWByOgQAZLz7es8oHA5LkoYMGRK3v6amRjk5ORo5cqQWLlyo1tbWO/6MaDSqSCQStwEAehefc84lUuic0+zZs3X16lUdOXIktn/nzp36yle+ooKCAjU2NupnP/uZbt68qZMnT8rv93f6OZWVlVqzZk2n/V/mO9MBAD1XJBJRIBD4Uq/nCYfRkiVLtHfvXh09elTDhg2743HNzc0qKCjQjh07NG/evE7PR6NRRaPRuObz8/MJIwBIc17CyNN7RrctW7ZMe/bs0eHDh+8aRJIUCoVUUFCghoaGLp/3+/1drpgAAL2HpzByzmnZsmV68803VVNTo8LCwnvWXLlyRU1NTQqFQgk3CQDIbJ4uYFiyZIn+/Oc/a/v27crKylJLS4taWlp048YNSdK1a9e0cuVKvfvuuzp//rxqamo0a9YsDR06VHPnzk3JXwAAkP48vWfk8/m63L9582aVlZXpxo0bmjNnjurr6/Xxxx8rFApp+vTpevnll5Wfn/+lzuHld4wAgJ4rZe8Z3Su3BgwYoP3793v5kQAAcG86AIA9wggAYI4wAgCYI4wAAOYIIwCAOcIIAGCOMAIAmCOMAADmCCMAgDnCCABgjjACAJgjjAAA5ggjAIA5wggAYI4wAgCYI4wAAOYIIwCAOcIIAGCOMAIAmCOMAADmCCMAgDnCCABgjjACAJgjjAAA5ggjAIC5vtYNfJFzTpIUiUSMOwEA3I/br+O3X9fvpseFUVtbmyQpPz/fuBMAQDK0tbUpEAjc9Rif+zKR1Y1u3bqlS5cuKSsrSz6fL+65SCSi/Px8NTU1KTs726jDnoGx6MA4dGAcPsNYdOgJ4+CcU1tbm/Ly8vTAA3d/V6jHrYweeOABDRs27K7HZGdn9+pJ9nmMRQfGoQPj8BnGooP1ONxrRXQbFzAAAMwRRgAAc2kVRn6/X6tXr5bf77duxRxj0YFx6MA4fIax6JBu49DjLmAAAPQ+abUyAgBkJsIIAGCOMAIAmCOMAADmCCMAgDnCCABgjjACAJgjjAAA5v4PCfKgUV04va8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 480x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X_train shape: (60000, 784)\n",
      "Y_train shape: (60000, 10)\n",
      "\n",
      "60000 train samples\n",
      "10000 test samples\n"
     ]
    }
   ],
   "source": [
    "from keras.datasets import mnist\n",
    "from tensorflow.keras.preprocessing.image import ImageDataGenerator\n",
    "import elasticdeform\n",
    "\n",
    "# input image dimensions\n",
    "num_classes = 10 # 10 digits\n",
    "\n",
    "img_rows, img_cols = 28, 28 # number of pixels \n",
    "\n",
    "# the data, shuffled and split between train and test sets\n",
    "(X_train, Y_train), (X_test, Y_test) = mnist.load_data()\n",
    "\n",
    "# convert class vectors to binary class matrices\n",
    "Y_train = keras.utils.to_categorical(Y_train, num_classes)\n",
    "Y_test = keras.utils.to_categorical(Y_test, num_classes)\n",
    "\n",
    "# datagen = ImageDataGenerator(featurewise_center=True, featurewise_std_normalization=True, zca_whitening=True)\n",
    "# X_train = datagen.flow(X_train.reshape(60000, 28, 28, 1), Y_train, batch_size=9, shuffle=False)\n",
    "# shift = 0.2\n",
    "# datagen = ImageDataGenerator(width_shift_range=shift, height_shift_range=shift)\n",
    "# X_train = datagen.flow(X_train, Y_train, batch_size=9, shuffle=False)\n",
    "\n",
    "# reshape data, depending on Keras backend\n",
    "X_train = X_train.reshape(X_train.shape[0], img_rows*img_cols)\n",
    "X_test = X_test.reshape(X_test.shape[0], img_rows*img_cols)\n",
    "    \n",
    "# cast floats to single precesion\n",
    "X_train = X_train.astype('float32')\n",
    "X_test = X_test.astype('float32')\n",
    "\n",
    "# rescale data in interval [0,1]\n",
    "X_train /= 255\n",
    "X_test /= 255\n",
    "\n",
    "mean_px = X_train.mean().astype(np.float32)\n",
    "std_px = X_train.std().astype(np.float32)\n",
    "X_train = (X_train - mean_px)/(std_px)\n",
    "\n",
    "mean_px = X_test.mean().astype(np.float32)\n",
    "std_px = X_test.std().astype(np.float32)\n",
    "X_test = (X_test - mean_px)/(std_px)\n",
    "\n",
    "# look at an example of data point\n",
    "print('an example of a data point with label', Y_train[20])\n",
    "plt.matshow(X_train[20,:].reshape(28,28),cmap='binary')\n",
    "plt.show()\n",
    "\n",
    "print('X_train shape:', X_train.shape)\n",
    "print('Y_train shape:', Y_train.shape)\n",
    "print()\n",
    "print(X_train.shape[0], 'train samples')\n",
    "print(X_test.shape[0], 'test samples')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 2: Define the Neural Net and its Architecture\n",
    "\n",
    "We can now move on to construct our deep neural net. We shall use Keras's `Sequential()` class to instantiate a model, and will add different deep layers one by one.\n",
    "\n",
    "At this stage, we refrain from using convolutional layers. This is done further below.\n",
    "\n",
    "Let us create an instance of Keras' `Sequential()` class, called `model`. As the name suggests, this class allows us to build DNNs layer by layer. We use the `add()` method to attach layers to our model. For the purposes of our introductory example, it suffices to focus on `Dense` layers for simplicity. Every `Dense()` layer accepts as its first required argument an integer which specifies the number of neurons. The type of activation function for the layer is defined using the `activation` optional argument, the input of which is the name of the activation function in `string` format. Examples include `relu`, `tanh`, `elu`, `sigmoid`, `softmax`. \n",
    "\n",
    "In order for our DNN to work properly, we have to make sure that the numbers of input and output neurons for each layer match. Therefore, we specify the shape of the input in the first layer of the model explicitly using the optional argument `input_shape=(N_features,)`. The sequential construction of the model then allows Keras to infer the correct input/output dimensions of all hidden layers automatically. Hence, we only need to specify the size of the softmax output layer to match the number of categories."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model architecture created successfully!\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential\n",
    "from keras.layers import Dense, Dropout, Flatten, BatchNormalization\n",
    "from keras.layers import Conv2D, MaxPooling2D\n",
    "\n",
    "# 400 100 0.5 64 10 / 0.9829999804496765\n",
    "# 300 100 0.5 80 15 / 0.9836000204086304\n",
    "\n",
    "def create_DNN(lys, dropout):\n",
    "    # instantiate model\n",
    "    model = Sequential()\n",
    "    # add a dense all-to-all relu layer\n",
    "    \n",
    "    first = True\n",
    "    for ly in lys:\n",
    "        if first:\n",
    "            model.add(Dense(ly,input_shape=(img_rows*img_cols,), activation='relu'))\n",
    "            model.add(BatchNormalization())\n",
    "            first = False\n",
    "        else:\n",
    "            model.add(Dense(ly, activation='relu'))\n",
    "            model.add(BatchNormalization())\n",
    "\n",
    "    model.add(Dropout(dropout))\n",
    "    # soft-max layer\n",
    "    model.add(Dense(num_classes, activation='softmax'))\n",
    "    \n",
    "    return model\n",
    "\n",
    "print('Model architecture created successfully!')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 3: Choose the Optimizer and the Cost Function\n",
    "\n",
    "Next, we choose the loss function according to which to train the DNN. For classification problems, this is the cross entropy, and since the output data was cast in categorical form, we choose the `categorical_crossentropy` defined in Keras' `losses` module. Depending on the problem of interest one can pick any other suitable loss function. To optimize the weights of the net, we choose SGD. This algorithm is already available to use under Keras' `optimizers` module, but we could use `Adam()` or any other built-in one as well. The parameters for the optimizer, such as `lr` (learning rate) or `momentum` are passed using the corresponding optional arguments of the `SGD()` function. All available arguments can be found in Keras' online documentation at [https://keras.io/](https://keras.io/). While the loss function and the optimizer are essential for the training procedure, to test the performance of the model one may want to look at a particular `metric` of performance. For instance, in categorical tasks one typically looks at their `accuracy`, which is defined as the percentage of correctly classified data points. To complete the definition of our model, we use the `compile()` method, with optional arguments for the `optimizer`, `loss`, and the validation `metric` as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model compiled successfully and ready to be trained.\n"
     ]
    }
   ],
   "source": [
    "from tensorflow.keras.optimizers import schedules\n",
    "\n",
    "def compile_model_(dropout):\n",
    "    def _compile_model_(optimizer=keras.optimizers.RMSprop(), ly1=320, ly2=270, ly3=20):\n",
    "        model=create_DNN(ly1, ly2, ly3, dropout)\n",
    "        model.compile(loss=keras.losses.categorical_crossentropy,\n",
    "                      optimizer=optimizer,\n",
    "                      metrics=['accuracy'])\n",
    "        return model    \n",
    "    return _compile_model_\n",
    "    \n",
    "\n",
    "def compile_model(lys, dropout, learning_rate, c_optimizer):\n",
    "    learning_r = 10**learning_rate\n",
    "    lr_schedule = schedules.ExponentialDecay(\n",
    "    initial_learning_rate=learning_r,\n",
    "    decay_steps=10000,\n",
    "    decay_rate=0.9)\n",
    "    rand = round(c_optimizer) # random.choice(foo)\n",
    "    if rand == 0:\n",
    "        optimizer=keras.optimizers.SGD(learning_rate=lr_schedule)\n",
    "    elif rand == 1:\n",
    "        optimizer=keras.optimizers.RMSprop(learning_rate=learning_r)\n",
    "    elif rand == 2:\n",
    "        optimizer=keras.optimizers.Adagrad(learning_rate=lr_schedule)\n",
    "    elif rand == 3:\n",
    "        optimizer=keras.optimizers.Adadelta(learning_rate=lr_schedule)\n",
    "    elif rand == 4:\n",
    "        optimizer=keras.optimizers.Adam() # learning_rate=lr_schedule\n",
    "    elif rand == 5:\n",
    "        optimizer=keras.optimizers.Adamax(learning_rate=lr_schedule)\n",
    "    else:\n",
    "        optimizer=keras.optimizers.Nadam(learning_rate=learning_r)\n",
    "\n",
    "    # create the mode\n",
    "    model=create_DNN(lys, dropout)\n",
    "    # compile the model\n",
    "    model.compile(loss=keras.losses.categorical_crossentropy,\n",
    "                  optimizer=optimizer,\n",
    "                  metrics=['accuracy'])\n",
    "    return model\n",
    "\n",
    "print('Model compiled successfully and ready to be trained.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 4: Train the model\n",
    "\n",
    "We train our DNN in minibatches, the advantages of which were explained in Sec. IV. \n",
    "\n",
    "Shuffling the training data during training improves stability of the model. Thus, we train over a number of training epochs. \n",
    "\n",
    "Training the DNN is a one-liner using the `fit()` method of the `Sequential` class. The first two required arguments are the training input and output data. As optional arguments, we specify the mini-`batch_size`, the number of training `epochs`, and the test or `validation_data`. To monitor the training procedure for every epoch, we set `verbose=True`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# # training parameters\n",
    "# batch_size = 80 # 64\n",
    "# epochs = 15 # 10\n",
    "\n",
    "# # create the deep neural net\n",
    "# model_DNN=compile_model()\n",
    "\n",
    "# # train DNN and store training info in history\n",
    "# history=model_DNN.fit(X_train, Y_train,\n",
    "#           batch_size=batch_size,\n",
    "#           epochs=epochs,\n",
    "#           verbose=1,\n",
    "#           validation_data=(X_test, Y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 5: Evaluate the Model Performance on the *Unseen* Test Data\n",
    "\n",
    "Next, we evaluate the model and read of the loss on the test data, and its accuracy using the `evaluate()` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scipy.optimize import curve_fit\n",
    "# best_model = None\n",
    "# best_accuracy = 0\n",
    "\n",
    "def fine_fitting(ly1, ly2, ly3, c_optimizer, batch_size, epochs):\n",
    "    def fine_fitting_(p,  dropout, learning_rate):\n",
    "        model_DNN=compile_model(ly1, ly2, ly3, dropout, learning_rate, c_optimizer)\n",
    "\n",
    "        history=model_DNN.fit(X_train, Y_train,\n",
    "              batch_size=round(batch_size),\n",
    "              epochs=round(epochs),\n",
    "              verbose=1,\n",
    "              validation_data=(X_test, Y_test))\n",
    "        score = model_DNN.evaluate(X_test, Y_test, verbose=0)\n",
    "        print(\"*****Score: \", score[1], \"  with:\", round(ly1), round(ly2), round(ly3), dropout, learning_rate, round(c_optimizer), round(batch_size), round(epochs))\n",
    "        return [1 - score[1]]\n",
    "    return  fine_fitting_\n",
    "\n",
    "def fitting(lys, c_optimizer, batch_size, epochs, dropout, learning_rate):\n",
    "    model_DNN=compile_model(lys, dropout, learning_rate, c_optimizer)\n",
    "\n",
    "    history=model_DNN.fit(X_train, Y_train,\n",
    "          batch_size=round(batch_size),\n",
    "          epochs=round(epochs),\n",
    "          verbose=1,\n",
    "          validation_data=(X_test, Y_test))\n",
    "    \n",
    "    score = model_DNN.evaluate(X_test, Y_test, verbose=0)\n",
    "    return score[1], model_DNN, history\n",
    "\n",
    "B = ( [0,0,0,   0,     60,10, 0.2,-5], [500, 150, 50,    6,     60.1, 10.1, 0.6, -2] )\n",
    "Bf = ( [0.2,-6], [0.6, -1] )\n",
    "def getRandomP0(bounds):\n",
    "    params0 = []\n",
    "    n_param = len(bounds[0])\n",
    "    for i in range(n_param):\n",
    "        params0.append( np.random.random()*( bounds[1][i] - bounds[0][i] ) + bounds[0][i]  )\n",
    "    return params0\n",
    "\n",
    "# param123, cov = curve_fit(fitting, [1], [0], maxfev = 2000, bounds=B, p0=getRandomP0(B)) #  getRandomP0(B) 200000"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/30\n",
      "1875/1875 [==============================] - 125s 66ms/step - loss: 0.7289 - accuracy: 0.7878 - val_loss: 0.1887 - val_accuracy: 0.9501\n",
      "Epoch 2/30\n",
      "1875/1875 [==============================] - 126s 67ms/step - loss: 0.4516 - accuracy: 0.8610 - val_loss: 0.1193 - val_accuracy: 0.9666\n",
      "Epoch 3/30\n",
      "1875/1875 [==============================] - 126s 67ms/step - loss: 0.3635 - accuracy: 0.8813 - val_loss: 0.1056 - val_accuracy: 0.9743\n",
      "Epoch 4/30\n",
      "1875/1875 [==============================] - 126s 67ms/step - loss: 0.3194 - accuracy: 0.8972 - val_loss: 0.0945 - val_accuracy: 0.9775\n",
      "Epoch 5/30\n",
      "1875/1875 [==============================] - 127s 68ms/step - loss: 0.2962 - accuracy: 0.9045 - val_loss: 0.0918 - val_accuracy: 0.9774\n",
      "Epoch 6/30\n",
      "1875/1875 [==============================] - 126s 67ms/step - loss: 0.2720 - accuracy: 0.9102 - val_loss: 0.0879 - val_accuracy: 0.9809\n",
      "Epoch 7/30\n",
      "1875/1875 [==============================] - 126s 67ms/step - loss: 0.2595 - accuracy: 0.9158 - val_loss: 0.1004 - val_accuracy: 0.9789\n",
      "Epoch 8/30\n",
      "1875/1875 [==============================] - 127s 68ms/step - loss: 0.2476 - accuracy: 0.9215 - val_loss: 0.0971 - val_accuracy: 0.9812\n",
      "Epoch 9/30\n",
      "1875/1875 [==============================] - 126s 67ms/step - loss: 0.2350 - accuracy: 0.9247 - val_loss: 0.0980 - val_accuracy: 0.9785\n",
      "Epoch 10/30\n",
      "1875/1875 [==============================] - 126s 67ms/step - loss: 0.2316 - accuracy: 0.9247 - val_loss: 0.0906 - val_accuracy: 0.9819\n",
      "Epoch 11/30\n",
      "1875/1875 [==============================] - 126s 67ms/step - loss: 0.2235 - accuracy: 0.9270 - val_loss: 0.0855 - val_accuracy: 0.9841\n",
      "Epoch 12/30\n",
      "1875/1875 [==============================] - 126s 67ms/step - loss: 0.2204 - accuracy: 0.9275 - val_loss: 0.1138 - val_accuracy: 0.9802\n",
      "Epoch 13/30\n",
      "1875/1875 [==============================] - 128s 68ms/step - loss: 0.2090 - accuracy: 0.9303 - val_loss: 0.1003 - val_accuracy: 0.9821\n",
      "Epoch 14/30\n",
      "1875/1875 [==============================] - 126s 67ms/step - loss: 0.2077 - accuracy: 0.9311 - val_loss: 0.0887 - val_accuracy: 0.9839\n",
      "Epoch 15/30\n",
      "1875/1875 [==============================] - 127s 68ms/step - loss: 0.1984 - accuracy: 0.9352 - val_loss: 0.0856 - val_accuracy: 0.9843\n",
      "Epoch 16/30\n",
      "1875/1875 [==============================] - 126s 67ms/step - loss: 0.1895 - accuracy: 0.9379 - val_loss: 0.0888 - val_accuracy: 0.9823\n",
      "Epoch 17/30\n",
      "1875/1875 [==============================] - 138s 73ms/step - loss: 0.1979 - accuracy: 0.9368 - val_loss: 0.0860 - val_accuracy: 0.9843\n",
      "Epoch 18/30\n",
      "1875/1875 [==============================] - 126s 67ms/step - loss: 0.1883 - accuracy: 0.9383 - val_loss: 0.0847 - val_accuracy: 0.9838\n",
      "Epoch 19/30\n",
      "1875/1875 [==============================] - 127s 68ms/step - loss: 0.1806 - accuracy: 0.9413 - val_loss: 0.0868 - val_accuracy: 0.9840\n",
      "Epoch 20/30\n",
      "1875/1875 [==============================] - 126s 67ms/step - loss: 0.1735 - accuracy: 0.9413 - val_loss: 0.1087 - val_accuracy: 0.9853\n",
      "Epoch 21/30\n",
      "1875/1875 [==============================] - 127s 68ms/step - loss: 0.1762 - accuracy: 0.9419 - val_loss: 0.1020 - val_accuracy: 0.9838\n",
      "Epoch 22/30\n",
      "1875/1875 [==============================] - 127s 68ms/step - loss: 0.1731 - accuracy: 0.9435 - val_loss: 0.0903 - val_accuracy: 0.9859\n",
      "Epoch 23/30\n",
      "1875/1875 [==============================] - 127s 68ms/step - loss: 0.1760 - accuracy: 0.9436 - val_loss: 0.0964 - val_accuracy: 0.9847\n",
      "Epoch 24/30\n",
      "1875/1875 [==============================] - 129s 69ms/step - loss: 0.1685 - accuracy: 0.9446 - val_loss: 0.0983 - val_accuracy: 0.9851\n",
      "Epoch 25/30\n",
      "1875/1875 [==============================] - 131s 70ms/step - loss: 0.1726 - accuracy: 0.9445 - val_loss: 0.1003 - val_accuracy: 0.9863\n",
      "Epoch 26/30\n",
      "1875/1875 [==============================] - 131s 70ms/step - loss: 0.1658 - accuracy: 0.9464 - val_loss: 0.1068 - val_accuracy: 0.9852\n",
      "Epoch 27/30\n",
      "1875/1875 [==============================] - 150s 80ms/step - loss: 0.1607 - accuracy: 0.9466 - val_loss: 0.1230 - val_accuracy: 0.9834\n",
      "Epoch 28/30\n",
      "1875/1875 [==============================] - 170s 91ms/step - loss: 0.1570 - accuracy: 0.9492 - val_loss: 0.1591 - val_accuracy: 0.9838\n",
      "Epoch 29/30\n",
      "1875/1875 [==============================] - 157s 84ms/step - loss: 0.1645 - accuracy: 0.9458 - val_loss: 0.1115 - val_accuracy: 0.9850\n",
      "Epoch 30/30\n",
      "1875/1875 [==============================] - 132s 70ms/step - loss: 0.1558 - accuracy: 0.9484 - val_loss: 0.1217 - val_accuracy: 0.9843\n",
      "Score:  0.9843000173568726\n"
     ]
    }
   ],
   "source": [
    "for i in range(1):\n",
    "    score_, model_, history_ = fitting([2500, 2000, 1500, 1000, 500, 10], c_optimizer=1, \n",
    "                             batch_size=32, epochs=30, dropout=0.5, learning_rate=-3)\n",
    "    print(\"Score: \", score_)\n",
    "    # Score:  0.9835000038146973   with: 320 270 20 0.25 -3 1 100 15\n",
    "\n",
    "    if score_ > best_accuracy:\n",
    "        best_model = model_\n",
    "        best_accuracy = score_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9857000112533569"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "best_accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for i in range(100):\n",
    "    myrand = getRandomP0(B)\n",
    "    res = fitting(1, *myrand)\n",
    "    if res[0] < 0.03:\n",
    "        myrand2 = np.copy(myrand)\n",
    "        for j in [-5,-4,-3,-2,-1]:\n",
    "            for k in [0,1,2,3,4,5,6]:\n",
    "                myrand2[3] = k\n",
    "                myrand2[-1] = j\n",
    "                fine_fitting(*myrand2[0:6])(1, *myrand2[-2:]) #  getRandomP0(B) 200000\n",
    "# Score:  0.9811999797821045   with: 306 308 472 0.6818202991034834 78 10\n",
    "# Score:  0.982699990272522   with: 404 285 204 0.06916699545513805 143 15\n",
    "# Score:  0.9822999835014343   with: 496 18 47 0.12217255924633919 -3.3010651833370104 1 60 10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "313/313 [==============================] - 3s 11ms/step - loss: 0.1104 - accuracy: 0.9857\n",
      "\n",
      "Test loss: 0.11040810495615005\n",
      "Test accuracy: 0.9857000112533569\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAksAAAGwCAYAAAC5ACFFAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAA9hAAAPYQGoP6dpAABjgUlEQVR4nO3de1xUZf4H8M8wzAwXYQBBLoKAN7yApKgIaKUlZmpqa6G7eSm13G3XXLPdSC01k9J0rUw3TUt3LS2zy/4yFStNI0XJK5qSNy4OICAMF2Fg5vz+ODAygsjgwJmBz/u15zUzZ86c853TrOfDc57zHJkgCAKIiIiIqF52UhdAREREZM0YloiIiIgawLBERERE1ACGJSIiIqIGMCwRERERNYBhiYiIiKgBDEtEREREDbCXugBbZTAYcO3aNbi4uEAmk0ldDhERETWCIAgoLi6Gn58f7Owa12bEsNRE165dQ0BAgNRlEBERURNkZGTA39+/UcsyLDWRi4sLAHFnu7q6SlwNERERNYZWq0VAQIDxON4YDEtNVHPqzdXVlWGJiIjIxpjThYYdvImIiIgawLBERERE1ACGJSIiIqIGMCwRERERNYBhiYiIiKgBDEtEREREDWBYIiIiImoAwxIRERFRAxiWiIiIiBrAsERERETUAIYlIiIiogYwLBERERE1gGGJiIiouQgCUJoPlGulroTugb3UBRARETWoNA/IOAIUZwNunQC3QPFR4SB1ZSJdKVCYDty4Aty4Kj4WXr31vLJUXM5BDagDxMktAFD7Vz/vJD537gDYsQ3DGjEsERFRw8oKgILLQMElQK8DPLsDXt3Fg7+lGQxA3gUg4zCQfkQMSQUX61/WxVcMTu5BgHv1Y81rF1/LBA99lRh2bhZWB6ArdQNRaW7j1lVeJE45Z+p/X66sDlD+gLrTrUDlFgh49wacPO79+1CTMCwREbV1glAdiC6Kgahmyq9+XV5Y/+dc/ACvkFpTD3Ey56CuKwWyfhXDUUayONW3Pa8eYggqyhSDiq4EKNaIU8bhusvLlbdaodwDxUc7OaArE8OPrgyoLBO3ryu99byyrNYypWI4bAyV+lZgMwa36ufqAMBQKdZelCm2QhVlVD/PEJ8Xa8Rt1ez7+rh1AnzDq6e+4mM7r8bV11wEAbh5A7hxWQyOtwfKqnLAqT3g6C4+OnlUP7YHHGueu996rXIBZDJpv1M9ZIIgCFIXYYu0Wi3UajWKiorg6uoqdTlE5tOVigcmvQ7oPBSwV0pdkXmqKsS/9ht94Kt1ANSVievwCQX8BwAd+zfvQUcQAG0WkHkUyDwGXP8NsHcQDwx1JldA2a7uPJULYK+684HEoBf/W+orxclQafparxPnVd4UD9b5tYPRZaCiqOHv4OILeHQG7OzFlp9izZ2XdfYSw41n9+oAVR2k2nUQP5d+WGwxyjgCZJ8GDFWmn1c4AR0jgIDI6mmAeLCtvT/LCoDCK/W09FwRQ8jt67xXJuEr6LaWrEDT+ppCXwlor1UHqoxbIaooQ/xvdONK/Z9z8bsVoPzuEx9dfC0bOGp+M3c6zagrtty27BS3ApWjB9DlQeD+lyy3fjTt+M2w1EQMS2RzygqA9F+Aq0ni47UTgKAX33PuAPR/Goh4GnD1lbTMelWUiAdVzclb0/XfbtVvCe5BYmjyHyBOPmFND5C6UnH/Zh69FZBKsu+9RjuFGJrkiuogVFX9qANggX/KXTuKgaj21L6LuG+UzqbL3iwE8tLE/w7XfwOunwfyzosH1TtROIkB9nYufkCnSCBgEBAwUNz3ckXTv4e+SgyntQ/oNXUpnQCFc/WjkxhMjc+dqx9rlnG+Na+hoNoSbhYC2adM/z+Ql4Z6/7s7e9VqgQoXX5v8sVBPC9qdWtwqiht3mrGdT/2nQ5VO4r89ZQXAzQKgLL96qn5+s+DW86ryuuvtEwc8vv6edt3tGJZaEMMSWb2irOpw9DNw9Rfg+rm6y6gDxL9oaw7kdvZAz8eAyOfEv+ilODiYc1CA7NZBrs5B0LnWe7cfBJ3EgHHt+K2WntvJVYBvn+rw1F8MUm6d6u4TgwHI/10MRVnHxMecs3WDnEwutmR17C+u16AXD0S6EvGxohio0NZ6XmvSlZi/H2V2YmuInUIMHnLlrUe1f/2BSOFo/nZupysVW56un681/SaephEMYl0+YbVajSLFvjlkvooSsf+T5qQYzpvjj4gaKtdbrWjuQaatam6dLPTbKasVqKofXf2AwOh7X3ctDEstiGGJrIogiAfsq0nVLUdJ9f+F7xkCBEYBnaLFR7dOYlg69z8geb0Yrmr49AEGPguETbDMP4T1Kc0HNCdqBaMTjT/d4NNH/IfUEoHuZiFw7VcgM+VWa9DNgrrLOXeoDk8R4mnAzGNiQCqv5xSWi++toOU/APC9TwxpTWHQ1wpVJWLQs1eJ4bZ2CJIrboUjO3nTttVcKsvF36SrH6BqJ3U1rVflTTGs1/7/VUXxHf5oaOiPjOoWN6Wz+EeVo7tV9iVqCoalFsSwRJKrqgDOfwek7hQDUul10/dldmKwqAlGnaIAZ8+G16k5JYam05/fahJ39AD6TQEGTBfDVVOVF4l//V77VezQe+242B+jPsaOrPdVT33E/i4tRRDEfiI1YSjzaP19a2rYOwB+fW8Fo479AXXHlquXiBqNYakFMSwRALHZODMZyEkVTy10irq3vhZ3IwjiAfzkp8CZL0yvGpKrxIN1YLRYR8BAsX9LU5QVAMf/Axz98FYLlcwOCHkUGDgTCH6g4b8yK2+K4SLr11vhKD+t/mU9uph2TvXpY52XSFfeFP9KzzwGZKWILTk14ci7d/P+dycii2FYakEMS1aiSgdc2A2c2CpeYePZrfov+wjxsb4+JveiJhxdOSROmcfEK4xqqFyBLsOA7iOArsMtd4VVUSZwcps41Q4dLn5AnyeBkJFiy4a9yjLbq2HQAxf2AMkfAJf235rv1UMMTX0mitvMPVerxehX8XV9rTDqTkDHvoBfP6BjPzEcNcdYPUREd8Cw1IIYliSWkwoc3wqc2iZ2AryT2n1M/AeIgcKc1pa7hSNAvILIO1RsbSjLq/WGTAwE3R8BusWKwcCc4FZRIvYlOvkJcPkgjB2c7R2BXo8B4RPFFp6W6pty/TyQvEFs1arpcKxsJ4ai+q5icfa6FYr8+on7XuoxYYiozWNYakEMSxK4eQM4vUNsRbp2/Nb8dt5A+CSgxyixc3BNB936+pjI7IAOvW61PPkPEMeCqRnptzHhyMUPCB4CBA0WJ/dgMQQZ9GJdF3aLrTHZp0w/184H6DZcbHXq/GD9oc1gAK4cFAPJ2W9u3SYBAIKGiAGp19imn16zhPIi4MSnYt+mmpGVVa7iabTa4Ujt32o6hBJR68Gw1IIYllqIwQBc3g8c/y9w7v8AfYU4304BhDwC9J0MdHkIkNczGH3lTbHDsvGS7mP1dyhWuYoH+CqduNztI/beKRzdjVYDpO0Vp4s/mgYfuRIIjBGDU7dYsS/SyU+BU9tNa/ToLAbBPnHiJbrWxGAQr7hRuYj9jnhPKyKyAQxLLYhhqZkVXAZOfCIGiNrhoUNvoO9TYj+du13ZVZ/ibDE01QwUeO3XuoPkufjdCkbBQxofjhpSVSG2VKXtFVudbly+87IqNRA6Hgj/o9hJm60zREQWw7DUghiW6qErFcf3AMRWBplc7E8jk4unv+xqP9723E4utq78nii2Il05eGu9Dmog7Emg75/Ey8gtGR70VeJgjZnHxBoCY8TWnOYMKDVjItWcrkv/RZzX9SGxFSlkZPONa0RE1MYxLLUghqVqVTrg4vdiX6Lzu+q/lUGTyIAuQ8VWpJBRgMLBQuu1QhXFYn8nRzepKyEiavWacvyWvJPB2rVrERwcDAcHB0RERODgwYMNLv/++++jZ8+ecHR0REhICLZs2WLy/oMPPgiZTFZnGjVqlHGZRYsW1Xnfx8enWb5fq2TQA5cOAN/8DXi7G/DpRODMDjEoOXcQL9dXB4ins9p5i1dFOXqILURKF3G0WHsHsd+R7LafoHswMHQBMOc0MPlLIPQPrTsoAWKfHwYlIiKrVU+v2Jazfft2zJkzB2vXrkVMTAw++OADjBw5EmfPnkWnTnVHCl63bh3i4+OxYcMGDBgwAMnJyZg5cybc3d0xZswYAMDOnTuh093qoJufn4/w8HA88cQTJuvq3bs39u3bZ3wtl1vZrQGsjSCIY+ic2QGc2Wl6U9B2PkDo40DoBLGjdFNOYRkM4v2MOLAfERFZGUnD0qpVqzB9+nTMmDEDALB69Wrs2bMH69atQ0JCQp3l//Of/+C5555DXFwcAKBz5844fPgw3nrrLWNY8vAwHfl327ZtcHJyqhOW7O3t2ZrUGLnnxFNsZ74w7ZTs4CaO9RP2hNjP517H+rGzgxU0dBIREdUhWVjS6XRISUnByy+/bDI/NjYWSUlJ9X6moqICDg6mp2QcHR2RnJyMyspKKBR1WyU2btyIiRMnwtnZ2WR+Wloa/Pz8oFKpEBkZiWXLlqFz5853rLeiogIVFRXG11qt9q7f0WbduCKGo9NfALmpt+YrnMTbXYRNEC/Xt1dKViIREVFLkSws5eXlQa/Xw9vb22S+t7c3srOz6/3MiBEj8OGHH2LcuHHo168fUlJSsGnTJlRWViIvLw++vr4myycnJ+PMmTPYuHGjyfzIyEhs2bIF3bt3R05ODpYuXYro6Gikpqaiffv29W47ISEBixcvvodvbAMKLgNf/xW4eujWPDuFOJBi6B/Eq7SUznf+PBERUSsk6Wk4AJDd1r9FEIQ682osXLgQ2dnZGDRoEARBgLe3N6ZNm4bly5fX2+do48aNCA0NxcCBA03mjxw50vg8LCwMUVFR6NKlCzZv3oy5c+fWu+34+HiT97RaLQICAhr9Pa3e1SRg25+AmwUAZEDw/WILUs8xgKO71NURERFJRrJOIp6enpDL5XVakXJzc+u0NtVwdHTEpk2bUFZWhitXriA9PR1BQUFwcXGBp6fpAIVlZWXYtm2bsT9UQ5ydnREWFoa0tDvcFR2ASqWCq6urydRqHN8KbH5MDEp+fYE5p4Cp3wD9pjAoERFRmydZWFIqlYiIiEBiYqLJ/MTERERHRzf4WYVCAX9/f8jlcmzbtg2jR4+G3W23Wvjss89QUVGBp5566q61VFRU4Ny5c3VO47V6Bj2wdyHw9V/E+5/1GgdM2yVe+k9EREQAJD4NN3fuXEyePBn9+/dHVFQU1q9fj/T0dMyaNQuAeOorKyvLOJbShQsXkJycjMjISNy4cQOrVq3CmTNnsHnz5jrr3rhxI8aNG1dvH6R58+ZhzJgx6NSpE3Jzc7F06VJotVpMnTq1eb+wNakoAXbOFAeSBID7/wE8GM/7exEREd1G0rAUFxeH/Px8LFmyBBqNBqGhodi1axcCA8Ubhmo0GqSnpxuX1+v1WLlyJc6fPw+FQoGhQ4ciKSkJQUFBJuu9cOECDh06hL1799a73czMTEyaNAl5eXnw8vLCoEGDcPjwYeN2W73CDODTSUDOaUCuAsa+D/R54u6fIyIiaoN4u5MmstnbnWQcBbb9ESjNFUfbnvgJEDBA6qqIiIhaRFOO35JfDUct6PQO4Ku/APoKwDsMmPQp4NaKrugjIiJqBgxLbYHBAOxPAH5aLr4OeRR4fAOgaidtXURERDaAYam105UBX/0ZOPuV+DrmBeCh1+799iRERERtBMNSa6bVANsmAdeOiyNxj3kH6PsnqasiIiKyKQxLrdW1E8CnE4FiDeDoAUzcCgQ2PH4VERER1cWw1Bqd/RrY+RxQdRPw6gFM2gZ4BEtdFRERkU1iWGptkjcAu+aJz7s+DEzYBDiopa2JiIjIhnG45takMAPYM198PvA5YNJ2BiUiIqJ7xJal1uT7JeIYSkFDgJFvATKZ1BURERHZPLYstRZZvwKnPxOfxy5lUCIiIrIQhqXWQBCAvQvE530mAn73SVoOERFRa8Kw1Bqc3wVc/RmwdwAeWih1NURERK0Kw5Kt01cCia+Kz6OeB9T+0tZDRETUyjAs2bpjHwH5vwPOXsDgv0tdDRERUavDsGTLbhaKN8gFgAfjAZWLpOUQERG1RgxLtuzgSuBmAeAZAvSbKnU1RERErRLDkq26cRU48m/xeezrgJxDZhERETUHhiVb9f1iQK8Dgh8AusVKXQ0REVGrxbBkizKPAWe+ACDjAJRERETNjGHJ1gjCrfu/3fdHwLePtPUQERG1cgxLtubc/4CMw4C9IzBsgdTVEBERtXoMS7akSgfse018Hv03wNVP2nqIiIjaAIYlW3JsI1BwCXDuAMS8IHU1REREbQLDkq24eQM48Jb4fNh8QNVO2nqIiIjaCIYlW/HT22Jg6tAL6DtZ6mqIiIjaDIYlW1BwGUheLz4f/jpgJ5e2HiIiojaEYckW7FskDkDZZRjQ7WGpqyEiImpTGJasXUYycPYrADKxVYmIiIhaFMOSNRMEYM8r4vO+TwE+odLWQ0RE1AYxLFmzs18BmUcBhRMwdL7U1RAREbVJDEvWqqoCSKwegDLmBcDVV9p6iIiI2iiGJWuVvAEovAq08xFH6yYiIiJJSB6W1q5di+DgYDg4OCAiIgIHDx5scPn3338fPXv2hKOjI0JCQrBlyxaT9z/++GPIZLI6U3l5+T1tt0WVFQA/LRefD1sAKJ2lrYeIiKgNkzQsbd++HXPmzMH8+fNx/PhxDBkyBCNHjkR6enq9y69btw7x8fFYtGgRUlNTsXjxYjz//PP43//+Z7Kcq6srNBqNyeTg4NDk7ba4n1YA5UWAdyhw3x+lroaIiKhNkwmCIEi18cjISPTr1w/r1q0zzuvZsyfGjRuHhISEOstHR0cjJiYGK1asMM6bM2cOjh07hkOHDgEQW5bmzJmDwsJCi20XACoqKlBRUWF8rdVqERAQgKKiIri6ujb6O99V/kXg/UjAUAlM/lIcW4mIiIgsQqvVQq1Wm3X8lqxlSafTISUlBbGxsSbzY2NjkZSUVO9nKioqTFqIAMDR0RHJycmorKw0zispKUFgYCD8/f0xevRoHD9+/J62CwAJCQlQq9XGKSAgoNHf1SxnvxKDUtfhDEpERCSZzBtlSDybg7ySirsv3MrZS7XhvLw86PV6eHt7m8z39vZGdnZ2vZ8ZMWIEPvzwQ4wbNw79+vVDSkoKNm3ahMrKSuTl5cHX1xc9evTAxx9/jLCwMGi1WrzzzjuIiYnByZMn0a1btyZtFwDi4+Mxd+5c4+ualiWLG/Ii4NcPcPWz/LqJiIjuoLxSj+TLBThw4Tr2n8/FxeulAAB7OxkeDOmACRH+GNajA5T2LdPOUlJRBV2VAR7OyhbZXkMkC0s1ZDKZyWtBEOrMq7Fw4UJkZ2dj0KBBEAQB3t7emDZtGpYvXw65XLxf2qBBgzBo0CDjZ2JiYtCvXz+89957ePfdd5u0XQBQqVRQqVRmf78m6TK0ZbZDRERt2pW8UmM4+uVSPsorDcb35HYy+Ls74mp+Gfady8G+czlwd1Jg7H0d8Yd+/gjt6NrgcdNcgiAgLbcE+8/nYv/56zh6pQDPDA5G/MieFttGU0kWljw9PSGXy+u05uTm5tZp9anh6OiITZs24YMPPkBOTg58fX2xfv16uLi4wNPTs97P2NnZYcCAAUhLS2vydomIiFqDmzo9frmUhwPnr2P/heu4ml9m8r6PqwMe6O6FB0K8ENPVE2pHBX7PLcaOlCzs/DUTucUV+DjpCj5OuoIQbxdMiPDH2L5+6ODicIctNqy0ogpJF/Px4/lcHDh/HVmFN03eT8spafJ3tSTJwpJSqURERAQSExMxfvx44/zExESMHTu2wc8qFAr4+/sDALZt24bRo0fDzq7+ZkFBEHDixAmEhYXd83aJiIhsiSAIuHi9BPvPX8eBC9dx5HIBdFW3Wo8Uchn6B3rgwRAxIIV4u9RpLerawQUvj+yBebHdcej3PHzxaxb2pGbjfE4x3th1Dm/u/g0PdPfChAh/PNSzA1T28kbV8+P5XBy9fAM6/a16lPZ2GNS5PYaGeOHBkA4I9rSOoXMkPQ03d+5cTJ48Gf3790dUVBTWr1+P9PR0zJo1C4DYTygrK8s4ltKFCxeQnJyMyMhI3LhxA6tWrcKZM2ewefNm4zoXL16MQYMGoVu3btBqtXj33Xdx4sQJvP/++43eLhERkZSKyipxIrMQJzMK8XtuCSqq9KjSC6g0CKjSG6qfVz/qDajUG1BlEIyvqwy35tc+tQYAHd0cxXDU3QvRXT3RTtW4KGAvt8ODIR3wYEgHFN2sxP+duoYvUjLxa3ohfvgtFz/8lgu1owKPhfvhDxH+CPdXQyaToUxXhaTf87H/Qi5+/K1u61EnDyc8GOKFoSEdMKhzezgq7xy2pCJpWIqLi0N+fj6WLFkCjUaD0NBQ7Nq1C4GBgQAAjUZjMvaRXq/HypUrcf78eSgUCgwdOhRJSUkICgoyLlNYWIhnn30W2dnZUKvV6Nu3L3766ScMHDiw0dslIiJqKRVVepy9psXJjEKczCzCiYxCXM4rtdj6lfZ2iAz2wIMhHfBAdy908XK+575GakcF/hQZiD9FBuLi9RJ8kZKJnb9mIVtbjv8cvor/HL6Krh3awcfVAcmXC0xbj+R2iOzsUR28vNDZ897raW6SjrNky5oyTgMREbVtBoOAy/mlOJlRiBMZYsvRWY0Wlfq6h+Kg9k4ID3BDbz9XOCntoZDLYG9nB3u5DAq5HRTy6ufGeabv29uJj14uKjgomr+1Rm8QkHQxDztSMrH7TDYqap3u83d3xNDqcBTVpT2clNK11TTl+C351XBERETWQhAEFN2sxLXCcmiKbuJaUTk0hTdRUKqDnZ0MCjsZ5NWBRG4ng73drYBS81puJ4N9dViR28lgJ5PhSl4pTlafVtOWV9XZroezEvcFuCHc3w3hAWqE+7vB3QoumTeH3E6GId28MKSbF7Tlldh9JhtlFVUY3M0yrVlSYlgiIqI2o0xXZQxCmsJyXCu6iWuFN6EpKjc+lun0zVqDyt4OYR3VCA9wQ3iAG/oGuMHf3dGmw8TtXB0UeLJ/Mw3eLAGGJSIisjqCICDzxk38ll2M3zRa/JZdDE3RTRgEQBAXgCA+QIAgPla/V9O7pPZ7BkFAfqkOhWWVDWz1Fg9nJXzVDvBzc4Sf2gHt26kgCECVQew8ra/uTK03GFBpEKDXC9Xzb3W0rv3a29UB9wW44b4AN4T4uEAhl/w+9mQGhiUiIpJUcXklLuQU45ymGL9la/Gbphjns4tRXFH3dJUlOCvl8HNzhG91EPJzc7wVjKqft0QfH7IdDEtERNQi9AYBV/JLq8OQFueyxXCUUXCz3uUVchm6dnBBTx8X9PB1QScPZ8jtZJABkMmqJ8hQ/T8A4t0ZjO9DVv0IQFbTWuQIVwf7VnXKi5ofwxIRETWawSCguKIK2puVKKqeTJ6X1zy/tYy21vz6rvoCxJGje/i6oIePK3pWP3b2cubpKrIKDEtERK1ceaUeB9PysPtMNn65mIfyKgOE2n1+qp+jVp+f2v2BgFv9gSr1BtzLgDMOCjuEeIthqCYc9fBxsbkrv6htYVgiImqFissr8cNvudiTmo39569b/Aovlb0d1I4K4+R623NXB/t63/d2dYDcjqfAyLYwLBERtRL5JRVIPJuDPanZ+Pn3fJNRk/3UDojt7YPhvbzh5aIy9usR3erbU1+fH9RaVmlvB1cHBTtAU5vCsEREZMOyCm9ib2o2dp/JxtErBTDUOkXW2csZj/T2wSOhPgjrqGanZqImYlgiIrIxv+eWYE9qNvakZuNUZpHJe6EdXY0BqWsHF4kqJGpdGJaIiMyUeaMM5ZUG+Lk5NOs9rorLK5GWW4Lfc0qQlluMtNwSXMguxrWicuMyMhkwINADI0J9ENvLGwEeTs1WD1FbxbBERHQXVXoDUq7ewA+/5WLfuRxcvH7rjvDuTgp0dHeEn1oc0LBj9cCGHd0d4efmAE9nFezu0qG56GYlfs8tRlpOCS5UB6Pfc0ugqRWKalPIZYju4olHQn3wcE+xDxIRNR+GJSKiehSW6XDgwnV8fy4X+8/nmtz81N5OBgeFHCUVVbhRVokbZZU4k6Wtdz1KuR183RxuhSl3R7g7KXA1vwy/55bgQk4xcosr7liHt6sK3Tq4oJt3O+NjDx8XuDgoLP6diah+DEtERBDHFrp4vQTfn8vF97/lIuXqDehr9ZZ2d1JgaEgHDOvZAUO6eUHtqIC2vBJZN8QbsV4rvImswvLqR/F1jrYcOr0BV/PLcDW/rMHt+6kd0NXbBd06tBMnbxd07dAOakeGIiKpMSwRUZulqzIg+XIBvv8tB9+fy0V6gWmgCfF2wbCeHfBQjw7o28m9zvhArg4KuPoq0NPXtd71V+oNyNGW41qtEJVVeBMFJTp0au+ErtXBqGuHdmwpIrJiDEtE1KYYDAJ+PJ+LHSmZOJiWh5JaN2tVyu0wqEt7PNSjA4b16HDPnaUVcjv4uzvB352drolsGcMSEbUJ5ZV6fPFrJjYeuoxLtTpoe7ZTieGoZwcM7uoJZxX/WSQiU/xXgYhatevFFfjPL1fw3yPpKCjVAQBcVPaYODAAo/v4Iayj+q5XqxFR28awRESt0oWcYmw8eBlfnsiCrkq87UdHN0c8MzgYcQMC0I4tSETUSPzXgohaDUEQ8PPv+dhw8BIOXLhunH9fgBtmDumMEb29YS+3k7BCIrJFDEtEZPN0VQZ8c/IaPjx4Cb9lFwMQR7Ye0csHM+8PRr9O7rwvGhE1GcMSEdmswjIdth5Jx+akK8aBHZ2UcjzZPwBPxwQhsL2zxBUSUWvAsEREVkkQBHGE7NJKFJTpcKNUh4JSHW6UiY+aonLsPpONm5V6AOJI11Ojg/CngYFQO3HMIiKyHIYlImpxBoOA4xmFOJVZKIagMp0YimqFoRtlOlTqhbuuq6evK2YOCcboPn5Q2rM/EhFZHsMSEbWI8ko9fv49D4lnc7DvXC7ySu58P7TaHBVyeDgr4e6sgLuTUnxe/dg/yB1RnduzPxIRNSuGJSJqNgWlOvzwWy4Sz2bjpwt5xlNmgDjW0aAu7eHj6gB3ZyU8nBTiY60w5O6khKNSLuE3ICJiWCIiC7uaX4rEsznYezYHx64UoNa9aOGrdsDDPb0xvJc3BnVuz9NmRGQTGJaI6J4YDAJOZRUh8Ww2Es/m4EJOicn7PX1dMbyXN2J7eaO3nytPmRGRzWFYIqI6Kqr0KC6vQkl5FYrLq1BcXglt9WNx9bySikrkl+pwKC3PeNk+AMjtZIgM9sDwXt54uKf3Pd+MlohIagxLRG1QfkkFjlwuwJFL+fj9eokxANWEoprbgzSWs1KOB0M6YHgvbwwN6cBL94moVZE8LK1duxYrVqyARqNB7969sXr1agwZMuSOy7///vtYs2YNrly5gk6dOmH+/PmYMmWK8f0NGzZgy5YtOHPmDAAgIiICy5Ytw8CBA43LLFq0CIsXLzZZr7e3N7Kzsy387YisQ15JBY5cKsDhS/k4cjm/zqmyO3FWyuHioICLgz1cHOzRrvq5q4O9OF9ljzB/NaK6tIfKnh2xiah1kjQsbd++HXPmzMHatWsRExODDz74ACNHjsTZs2fRqVOnOsuvW7cO8fHx2LBhAwYMGIDk5GTMnDkT7u7uGDNmDABg//79mDRpEqKjo+Hg4IDly5cjNjYWqamp6Nixo3FdvXv3xr59+4yv5XL+Q0+tR25xOY5cKsCRy/k4fKkAv+fWDUc9fFwwqHN7hHVUw81JYRKKXBwUaKeyh9yO/YuIiGSCINx91LdmEhkZiX79+mHdunXGeT179sS4ceOQkJBQZ/no6GjExMRgxYoVxnlz5szBsWPHcOjQoXq3odfr4e7ujjVr1hhboBYtWoSvvvoKJ06caHLtWq0WarUaRUVFcHV1bfJ6iCwhV1uOw5erW44u5ePi9dI6y/T0dUVksAcGdW6PgcEe8HBWSlApEZG0mnL8lqxlSafTISUlBS+//LLJ/NjYWCQlJdX7mYqKCjg4OJjMc3R0RHJyMiorK6FQ1O0nUVZWhsrKSnh4eJjMT0tLg5+fH1QqFSIjI7Fs2TJ07tz5jvVWVFSgouJWJ1atVnvX70jUkGuFN/HNyWv45sQ1XMorgZ1MBjuZDDIZqp8Dstse61tGV2XAtaJyk3XLZEBPH1cM6twekZ09MDDIA+4MR0RETSJZWMrLy4Ner4e3t7fJ/Ib6Do0YMQIffvghxo0bh379+iElJQWbNm1CZWUl8vLy4OvrW+czL7/8Mjp27IiHH37YOC8yMhJbtmxB9+7dkZOTg6VLlyI6Ohqpqalo3759vdtOSEio08+JyFxFZZX49rQGX53IQvLlAoutVyYDevlWh6NgDwwM9oCbE8MREZElSN7B+/YxVwRBuOM4LAsXLkR2djYGDRoEQRDg7e2NadOmYfny5fX2OVq+fDk+/fRT7N+/36RFauTIkcbnYWFhiIqKQpcuXbB582bMnTu33m3Hx8ebvKfVahEQEGDWd6W2qbxSj+/P5eKrE1nYfz7X5H5nkcEeGNe3I6I6t4edTAaDIEAAxEdBgEEQnxsMgAABQs1r4dYyANDVy4VXoBERNRPJwpKnpyfkcnmdVqTc3Nw6rU01HB0dsWnTJnzwwQfIycmBr68v1q9fDxcXF3h6epos+/bbb2PZsmXYt28f+vTp02Atzs7OCAsLQ1pa2h2XUalUUKlUjfx21NbpDQIOX8rHV8ezsPtMNoorqozv9fBxwbi+HfFYuB/83BwlrJKIiBpDsrCkVCoRERGBxMREjB8/3jg/MTERY8eObfCzCoUC/v7+AIBt27Zh9OjRsLO7dduEFStWYOnSpdizZw/69+9/11oqKipw7ty5BocsILobQRBwJkuLr05k4X8nr5kM1NjRzRGP3eeHcfd1RIiPi4RVEhGRuSQ9DTd37lxMnjwZ/fv3R1RUFNavX4/09HTMmjULgHjqKysrC1u2bAEAXLhwAcnJyYiMjMSNGzewatUqnDlzBps3bzauc/ny5Vi4cCE++eQTBAUFGVuu2rVrh3bt2gEA5s2bhzFjxqBTp07Izc3F0qVLodVqMXXq1BbeA9QaFJVVYssvV/DliSxcqnUVmtpRgUfDfDHuPj8MCPKAHS/DJyKySZKGpbi4OOTn52PJkiXQaDQIDQ3Frl27EBgYCADQaDRIT083Lq/X67Fy5UqcP38eCoUCQ4cORVJSEoKCgozLrF27FjqdDhMmTDDZ1muvvYZFixYBADIzMzFp0iTk5eXBy8sLgwYNwuHDh43bJWoMQRDw1YksLP2/c8gv1QEAVPZ2eLinN8be54cHQzrwRrFERK2ApOMs2TKOs9S2XbpegoVfn8HPv+cDALp4OWPWA13wSKgPXBzY0ZqIyFrZ1DhLRLaovFKPfx+4iLU/XoROb4DK3g5/G9YVz97fha1IREStFMMSUSMl/Z6HBV+dwaU8sV/SkG6eWDouFIHtnSWujIiImhPDEtFd5JVU4I1vz+HL41kAAC8XFV4d3Quj+/jecUwwIiJqPRiWiO7AYBCw/VgG3vzuNxTdrIRMBjwVGYh5I0KgdmS/JCKitsLssBQUFIRnnnkG06ZNQ6dOnZqjJiLJnc8uxvwvT+PY1RsAxFuJLHs8DPcFuElbGBERtTize6S++OKL+Prrr9G5c2cMHz4c27ZtM7nBLJEtu6nT483vfsOodw/i2NUbcFLKsWBUT3zz1xgGJSKiNqrJQwecPHkSmzZtwqeffoqqqir88Y9/xDPPPIN+/fpZukarxKEDWp8ffsvBq1+nIvPGTQBAbC9vLHqsN29JQkTUijTl+H3P4yxVVlZi7dq1+Oc//4nKykqEhobihRdewNNPP92qO78yLLUemqKbWPK/s/jujDjau5/aAYvHhmJ4r/rvUUhERLarRcdZqqysxJdffomPPvoIiYmJGDRoEKZPn45r165h/vz52LdvHz755JOmrp6o2VXqDfjo58tYvS8NZTo95HYyTB8cjBce6gZnFa99ICIikdlHhF9//RUfffQRPv30U8jlckyePBn/+te/0KNHD+MysbGxuP/++y1aKJElHbmUj4Vfn8GFnBIAQESgO14fG4pefmwlJCIiU2aHpQEDBmD48OFYt24dxo0bB4Wi7iXUvXr1wsSJEy1SIJElXS+uQMKuc9hZPWaSh7MSL4/sgQn9/HmjWyIiqpfZYenSpUt3veGss7MzPvrooyYXRWRpeoOAT45cxfI951FcXgWZDJg0sBP+MSIEbk5KqcsjIiIrZnZYys3NRXZ2NiIjI03mHzlyBHK5HP3797dYcUSWcCKjEAu/OoPTWUUAgNCOrlg6jmMmERFR45g9ztLzzz+PjIyMOvOzsrLw/PPPW6QoIksoLNPhlS9PY/zan3E6qwguDvZYMrY3vn5+MIMSERE1mtktS2fPnq13LKW+ffvi7NmzFimK6F4YDAJ2/JqJN7/7DQWlOgDA4307Iv7RnvByUUlcHRER2Rqzw5JKpUJOTg46d+5sMl+j0cDenpdbk7TOabRY+NUZ421KunVoh9fHhWJQ5/YSV0ZERLbK7HQzfPhwxMfH4+uvv4ZarQYAFBYW4pVXXsHw4cMtXiBRY5RUVOFfiRfwcdIV6A0CnJRyzHm4G56OCYZCbvbZZiIiIiOzw9LKlStx//33IzAwEH379gUAnDhxAt7e3vjPf/5j8QKJGiIIAvak5mDRN6nI1pYDAB4N88HC0b3gq+ZtSoiI6N6ZHZY6duyIU6dOYevWrTh58iQcHR3x9NNPY9KkSfWOuUTUXLIKb+K1r1Ox71wOAKCThxOWjO2NB0M6SFwZERG1Jk3qZOTs7Ixnn33W0rUQNUqV3oCPk65gVeIFlOn0UMhleO7+LvjrsK5wUMilLo+IiFqZJvfIPnv2LNLT06HT6UzmP/bYY/dcFNGdnMwoRPzO0zir0QIABgS5443xYeju7SJxZURE1Fo1aQTv8ePH4/Tp05DJZBAEAQAgk4m3itDr9ZatkAhAcXkl3t5zHlsOX4UgAGpHBV55tAeeiAjgbUqIiKhZmX2Z0AsvvIDg4GDk5OTAyckJqamp+Omnn9C/f3/s37+/GUqktkwQBHx3WoOHVx3A5l/EoDS+b0d8/+IDiBvQiUGJiIiandktS7/88gt++OEHeHl5wc7ODnZ2dhg8eDASEhIwe/ZsHD9+vDnqpDYo80YZXvs6Fd//lgsACGrvhKXjwjC4m6fElRERUVtidljS6/Vo164dAMDT0xPXrl1DSEgIAgMDcf78eYsXSG1Ppd6Aj36+jH8lpuFmpdiB+88PdMFfhrIDNxERtTyzw1JoaChOnTqFzp07IzIyEsuXL4dSqcT69evrjOpNZK7j6TfwypdncK66A/fAIA8sezwUXTuwAzcREUnD7LC0YMEClJaWAgCWLl2K0aNHY8iQIWjfvj22b99u8QKpbSiv1CNh1zljB243JwVeGdkTEyL82S+JiIgkJRNqLme7BwUFBXB3dzdeEdcWaLVaqNVqFBUVwdXVVepybFrmjTL8+b+/4nRWEQDg8X4dMf/Rnmjfjje9JSIiy2rK8duslqWqqio4ODjgxIkTCA0NNc738PAwr1KiaofS8vC3T3/FjbJKuDsp8K+4+zgCNxERWRWzwpK9vT0CAwM5lhLdM0EQsO7ARby95zwMAhDWUY11T/WDv7uT1KURERGZMHucpQULFiA+Ph4FBQXNUQ+1AcXllfjzf3/F8t1iUHoiwh+fz4piUCIiIqtkdlh69913cfDgQfj5+SEkJAT9+vUzmcy1du1aBAcHw8HBARERETh48GCDy7///vvo2bMnHB0dERISgi1bttRZ5osvvkCvXr2gUqnQq1cvfPnll/e8XbKM33NLMO79n7E7NRsKuQxvjA/F8gl9OCQAERFZLbOvhhs3bpzFNr59+3bMmTMHa9euRUxMDD744AOMHDkSZ8+eRadOneosv27dOsTHx2PDhg0YMGAAkpOTMXPmTLi7u2PMmDEAxEEz4+Li8Prrr2P8+PH48ssv8eSTT+LQoUOIjIxs0nbJMnaf0eDFz06iVKeHj6sD1j3VD307uUtdFhERUYMscjVcU0VGRqJfv35Yt26dcV7Pnj0xbtw4JCQk1Fk+OjoaMTExWLFihXHenDlzcOzYMRw6dAgAEBcXB61Wi++++864zCOPPAJ3d3d8+umnTdpufXg1XOPpDQLe3nse6/ZfBABEBntgzR/7wcuFV7sREVHLasrx2+zTcJai0+mQkpKC2NhYk/mxsbFISkqq9zMVFRVwcHAwmefo6Ijk5GRUVlYCEFuWbl/niBEjjOtsynZrtq3Vak0muruCUh2mbko2BqUZg4OxdUYkgxIREdkMs8OSnZ0d5HL5HafGysvLg16vh7e3t8l8b29vZGdn1/uZESNG4MMPP0RKSgoEQcCxY8ewadMmVFZWIi8vDwCQnZ3d4Dqbsl0ASEhIgFqtNk4BAQGN/q5t1anMQox57xAO/Z4HJ6Uc703qiwWje8FeLllGJyIiMpvZfZZu7yxdWVmJ48ePY/PmzVi8eLHZBdw+kKUgCHcc3HLhwoXIzs7GoEGDIAgCvL29MW3aNCxfvtwkqDVmneZsFwDi4+Mxd+5c42utVsvA1IDPjmZgwddnoKsyINjTGf9+KgIhPrxlCRER2R6zw9LYsWPrzJswYQJ69+6N7du3Y/r06Y1aj6enJ+RyeZ3WnNzc3DqtPjUcHR2xadMmfPDBB8jJyYGvry/Wr18PFxcXeHqKd6L38fFpcJ1N2S4AqFQqqFQ8dXQ3FVV6LP7fWXxyJB0A8HBPb6yKC4erg0LiyoiIiJrGYudDIiMjsW/fvkYvr1QqERERgcTERJP5iYmJiI6ObvCzCoUC/v7+kMvl2LZtG0aPHg07O/GrREVF1Vnn3r17jeu8l+1Sw3K05Yj74DA+OZIOmQx4cXh3rJ8cwaBEREQ2zeyWpfrcvHkT7733Hvz9/c363Ny5czF58mT0798fUVFRWL9+PdLT0zFr1iwA4qmvrKws41hKFy5cQHJyMiIjI3Hjxg2sWrUKZ86cwebNm43rfOGFF3D//ffjrbfewtixY/H1119j3759xqvlGrNdMp8gCPjzf1NwIqMQakcF3pnI25YQEVHrYHZYuv2GuYIgoLi4GE5OTvjvf/9r1rri4uKQn5+PJUuWQKPRIDQ0FLt27UJgYCAAQKPRID093bi8Xq/HypUrcf78eSgUCgwdOhRJSUkICgoyLhMdHY1t27ZhwYIFWLhwIbp06YLt27cbx1hqzHbJfF+fuIZf0wvhpJTj6+djEOTpLHVJREREFmH2OEsff/yxSViys7ODl5cXIiMj4e7edgYY5DhLt5TpqjDs7QPI1pbjpREheH5oV6lLIiIiqldTjt9mtyxNmzbN3I9QK/fv/ReRrS1HgIcjpg8OlrocIiIiizK7g/dHH32Ezz//vM78zz//3KTvELUNGQVl+OCnSwCA+Y/25D3eiIio1TE7LL355pvGy/Rr69ChA5YtW2aRosh2vPndb6ioMiCqc3uM6O0jdTlEREQWZ3ZYunr1KoKD655qCQwMNOmMTa3f4Uv5+Pa0BnYy4NUxvRoc1JOIiMhWmR2WOnTogFOnTtWZf/LkSbRv394iRZH10xsELPnfWQDAHyM7oadv2+7kTkRErZfZYWnixImYPXs2fvzxR+j1euj1evzwww944YUXMHHixOaokazQZ8cycFajhauDPeYOD5G6HCIiomZj9tVwS5cuxdWrV/HQQw/B3l78uMFgwJQpU9hnqY0oulmJt/ecBwDMebg7PJyVEldERETUfMwOS0qlEtu3b8fSpUtx4sQJODo6IiwsjAM6tiHvfZ+G/FIdung5Y3IU/7sTEVHr1uTbnXTr1g3dunWzZC1kAy5eL8HHSVcAAK+O6Q2F3GK3FyQiIrJKZh/pJkyYgDfffLPO/BUrVuCJJ56wSFFkvd749hyqDAKG9eiAB7p7SV0OERFRszM7LB04cACjRo2qM/+RRx7BTz/9ZJGiyDr9eD4XP/yWC3s7GRaM6il1OURERC3C7LBUUlICpbJuh16FQgGtVmuRosj6VOoNWPp/4lABT8cEobNXO4krIiIiahlmh6XQ0FBs3769zvxt27ahV69eFimKrM9/frmKi9dL0d5Zib89xL5qRETUdpjdwXvhwoX4wx/+gIsXL2LYsGEAgO+//x6ffvppvfeMI9uXX1KBf+27AACYNyIErg4KiSsiIiJqOWaHpcceewxfffUVli1bhh07dsDR0RF9+vTBvn378MADDzRHjSSxVYkXUFxehV6+rniyf4DU5RAREbWoJg0dMGrUqHo7eVPrc06jxafJ4j3/XhvTC3I73v+NiIjaFg6SQ3ckCOL93wwCMKqPLyI7895/RETU9pjdsqTX6/Gvf/0Ln332GdLT06HT6UzeLygosFhxJK09qdn45VI+VPZ2iB/ZQ+pyiIiIJGF2y9LixYuxatUqPPnkkygqKsLcuXPx+OOPw87ODosWLWqGEkkK5ZV6LP32HADgufs7w9/dSeKKiIiIpGF2WNq6dSs2bNiAefPmwd7eHpMmTcKHH36IV199FYcPH26OGkkCGw9dRuaNm/BxdcCsB7tIXQ4REZFkzA5L2dnZCAsLAwC0a9cORUVFAIDRo0fj22+/tWx1JIkcbTne//F3AED8oz3gpGzyLQSJiIhsntlhyd/fHxqNBgDQtWtX7N27FwBw9OhRqFQqy1ZHknhr928o0+kREeiOx8L9pC6HiIhIUmaHpfHjx+P7778HALzwwgtYuHAhunXrhilTpuCZZ56xeIHUso6n38DOX7MAAK+O7gWZjEMFEBFR22b2+ZU333zT+HzChAkICAjAzz//jK5du+Kxxx6zaHHUsgwGAYv/J97/bUKEP8ID3KQtiIiIyArcc2eUyMhIREZGWqIWkljiuRycyCiEs1KOf4wIkbocIiIiq8BBKclow0+XAABTo4PQwdVB4mqIiIisA8MSAQBSrt7Asas3oJTbYVp0kNTlEBERWQ2GJQIAfHhQbFUae58fW5WIiIhqYVgiXM0vxZ7UbADAzPs7S1wNERGRdWFYImw6dBkGAXgwxAvdvV2kLoeIiMiqNOpqOHd390aPt8Mb6dqWG6U6fHYsEwDw7BC2KhEREd2uUWFp9erVzVwGSWXrkau4WalHbz9XRHVpL3U5REREVqdRYWnq1KnNVsDatWuxYsUKaDQa9O7dG6tXr8aQIUPuuPzWrVuxfPlypKWlQa1W45FHHsHbb7+N9u3FA/2DDz6IAwcO1Pnco48+arx33aJFi7B48WKT9729vZGdnW3Bb2b9yiv1+DjpKgDg2fs7c7RuIiKiejSpz9LFixexYMECTJo0Cbm5uQCA3bt3IzU11az1bN++HXPmzMH8+fNx/PhxDBkyBCNHjkR6enq9yx86dAhTpkzB9OnTkZqais8//xxHjx7FjBkzjMvs3LkTGo3GOJ05cwZyuRxPPPGEybp69+5tstzp06fN3Au27+sTWcgrqYCv2gGPhvlKXQ4REZFVMjssHThwAGFhYThy5Ah27tyJkpISAMCpU6fw2muvmbWuVatWYfr06ZgxYwZ69uyJ1atXIyAgAOvWrat3+cOHDyMoKAizZ89GcHAwBg8ejOeeew7Hjh0zLuPh4QEfHx/jlJiYCCcnpzphyd7e3mQ5Ly+vBmutqKiAVqs1mWyZwSBgw8HLAIBnYoKhkLOvPxERUX3MPkK+/PLLWLp0KRITE6FUKo3zhw4dil9++aXR69HpdEhJSUFsbKzJ/NjYWCQlJdX7mejoaGRmZmLXrl0QBAE5OTnYsWMHRo0adcftbNy4ERMnToSzs7PJ/LS0NPj5+SE4OBgTJ07EpUuXGqw3ISEBarXaOAUEBDTym1qn/Rdy8XtuCVxU9pg40La/CxERUXMyOyydPn0a48ePrzPfy8sL+fn5jV5PXl4e9Ho9vL29TeY31HcoOjoaW7duRVxcHJRKJXx8fODm5ob33nuv3uWTk5Nx5swZk9N0gHg/uy1btmDPnj3YsGEDsrOzER0d3WD98fHxKCoqMk4ZGRmN/q7WaH31rU0mRXaCi4NC4mqIiIisl9lhyc3NDRqNps7848ePo2PHjmYXcHunYkEQ7tjR+OzZs5g9ezZeffVVpKSkYPfu3bh8+TJmzZpV7/IbN25EaGgoBg4caDJ/5MiR+MMf/oCwsDA8/PDDxo7fmzdvvmOdKpUKrq6uJpOtOp1ZhMOXCmBvJ+OtTYiIiO7C7LD0xz/+Ef/85z+RnZ0NmUwGg8GAn3/+GfPmzcOUKVMavR5PT0/I5fI6rUi5ubl1WptqJCQkICYmBi+99BL69OmDESNGYO3atdi0aVOdAFdWVoZt27bVaVWqj7OzM8LCwpCWltbo+m3Zhupbm4wJ94Ofm6PE1RAREVk3s8PSG2+8gU6dOqFjx44oKSlBr169cP/99yM6OhoLFixo9HqUSiUiIiKQmJhoMj8xMRHR0dH1fqasrAx2dqYly+VyAGKLVG2fffYZKioq8NRTT921loqKCpw7dw6+vq3/irDMG2X49rQYLGcMCZa4GiIiIuvXqHGWalMoFNi6dSuWLFmC48ePw2AwoG/fvujWrZvZG587dy4mT56M/v37IyoqCuvXr0d6errxtFp8fDyysrKwZcsWAMCYMWMwc+ZMrFu3DiNGjIBGo8GcOXMwcOBA+Pn5max748aNGDdunHH8pdrmzZuHMWPGoFOnTsjNzcXSpUuh1WqbdTwpa/HRz1egNwiI6doevf3UUpdDRERk9cwOSzW6dOmCLl263NPG4+LikJ+fjyVLlkCj0SA0NBS7du1CYGAgAECj0ZiMuTRt2jQUFxdjzZo1ePHFF+Hm5oZhw4bhrbfeMlnvhQsXcOjQIezdu7fe7WZmZmLSpEnIy8uDl5cXBg0ahMOHDxu321oV3azEtmRxf87krU2IiIgaRSbcfv6qHnPnzm30CletWnVPBdkKrVYLtVqNoqIim+ns/e8DF/Hmd78hxNsFu+cM4YjdRETU5jTl+N2olqXjx4+bvE5JSYFer0dISAgAsSVHLpcjIiLCzJKppeiqDPjoZ3EQyhlDghmUiIiIGqlRYenHH380Pl+1ahVcXFywefNmuLu7AwBu3LiBp59+usF7upG0/nfyGnK0FejgosJj9/nd/QNEREQEoAlXw61cuRIJCQnGoAQA7u7uWLp0KVauXGnR4sgyBEEwDhcwLSYIKnu5xBURERHZDrPDklarRU5OTp35ubm5KC4utkhRZFmHfs/Db9nFcFLK8aeBrbsTOxERkaWZHZbGjx+Pp59+Gjt27EBmZiYyMzOxY8cOTJ8+HY8//nhz1Ej3qObWJk/2D4Daibc2ISIiMofZQwf8+9//xrx58/DUU0+hsrJSXIm9PaZPn44VK1ZYvEC6N+c0WhxMy4OdDJg+mINQEhERmcvssOTk5IS1a9dixYoVuHjxIgRBQNeuXeHs7Nwc9dE9qumrNDLMFwEeThJXQ0REZHuaPCils7MzPDw8IJPJGJSslKboJr45cQ0A8CwHoSQiImoSs/ssGQwGLFmyBGq1GoGBgejUqRPc3Nzw+uuvw2AwNEeN1EQfJ11BlUHAwGAPhAe4SV0OERGRTTK7ZWn+/PnYuHEj3nzzTcTExEAQBPz8889YtGgRysvL8cYbbzRHnWSm4vJKfHJYvLUJW5WIiIiazuywtHnzZnz44Yd47LHHjPPCw8PRsWNH/OUvf2FYshLbj2aguKIKnb2cMaxHB6nLISIisllmn4YrKChAjx496szv0aMHCgoKLFIU3ZsqvQEf/XwFADBjcGfY2fHWJkRERE1ldlgKDw/HmjVr6sxfs2YNwsPDLVIU3ZtdZ7KRVXgT7Z2VeLxfR6nLISIismlmn4Zbvnw5Ro0ahX379iEqKgoymQxJSUnIyMjArl27mqNGMoMgCFj/00UAwJSoIDgoeGsTIiKie2F2y9IDDzyACxcuYPz48SgsLERBQQEef/xxnD9/njfStQKHLxXgTJYWKns7TI7irU2IiIjuVZPGWfLz82NHbiv1cdJlAMAT/f3h4ayUuBoiIiLb16SwVF5ejlOnTiE3N7fO2Eq1r5KjlncmSwsAeCycfZWIiIgsweywtHv3bkyZMgV5eXl13pPJZNDr9RYpjMxXpTcgW1sOAAhsz1ubEBERWYLZfZb++te/4oknnoBGo4HBYDCZGJSkpSkqh94gQGlvB692KqnLISIiahXMDku5ubmYO3cuvL29m6MeugeZN24CAPzdHDm2EhERkYWYHZYmTJiA/fv3N0MpdK8yb5QBADq6O0pcCRERUethdp+lNWvW4IknnsDBgwcRFhYGhUJh8v7s2bMtVhyZx9iy5M7+SkRERJZidlj65JNPsGfPHjg6OmL//v2QyW6d7pHJZAxLEsqoblnyZ8sSERGRxZgdlhYsWIAlS5bg5Zdfhp2d2WfxqBndalliWCIiIrIUs9OOTqdDXFwcg5IVyuJpOCIiIoszO/FMnToV27dvb45a6B5U6g3QFIlhKYAtS0RERBZj9mk4vV6P5cuXY8+ePejTp0+dDt6rVq2yWHHUeNlF5TAIgNLeDp4cY4mIiMhizA5Lp0+fRt++fQEAZ86cMXmvdmdvalm1O3dzjCUiIiLLMTss/fjjj81RB90jDhtARETUPNhLu5XglXBERETNg2GplcjkGEtERETNQvKwtHbtWgQHB8PBwQERERE4ePBgg8tv3boV4eHhcHJygq+vL55++mnk5+cb3//4448hk8nqTOXl5fe0XWvH03BERETNQ9KwtH37dsyZMwfz58/H8ePHMWTIEIwcORLp6en1Ln/o0CFMmTIF06dPR2pqKj7//HMcPXoUM2bMMFnO1dUVGo3GZHJwcGjydm1BZgFbloiIiJqDpGFp1apVmD59OmbMmIGePXti9erVCAgIwLp16+pd/vDhwwgKCsLs2bMRHByMwYMH47nnnsOxY8dMlpPJZPDx8TGZ7mW71k5XZUC2Vmw5Y1giIiKyLMnCkk6nQ0pKCmJjY03mx8bGIikpqd7PREdHIzMzE7t27YIgCMjJycGOHTswatQok+VKSkoQGBgIf39/jB49GsePH7+n7QJARUUFtFqtyWQtasZYUtnbwYtjLBEREVmUZGEpLy8Per0e3t7eJvO9vb2RnZ1d72eio6OxdetWxMXFQalUwsfHB25ubnjvvfeMy/To0QMff/wxvvnmG3z66adwcHBATEwM0tLSmrxdAEhISIBarTZOAQEBTf3qFlfTubujuyPHuiIiIrIwyTt4335wFwThjgf8s2fPYvbs2Xj11VeRkpKC3bt34/Lly5g1a5ZxmUGDBuGpp55CeHg4hgwZgs8++wzdu3c3CVTmbhcA4uPjUVRUZJwyMjLM/arNhp27iYiImo/Zg1JaiqenJ+RyeZ3WnNzc3DqtPjUSEhIQExODl156CQDQp08fODs7Y8iQIVi6dCl8fX3rfMbOzg4DBgwwtiw1ZbsAoFKpoFJZ5ymumpYl3hOOiIjI8iRrWVIqlYiIiEBiYqLJ/MTERERHR9f7mbKyMtjZmZYsl8sBiC1D9REEASdOnDAGqaZs19qxZYmIiKj5SNayBABz587F5MmT0b9/f0RFRWH9+vVIT083nlaLj49HVlYWtmzZAgAYM2YMZs6ciXXr1mHEiBHQaDSYM2cOBg4cCD8/PwDA4sWLMWjQIHTr1g1arRbvvvsuTpw4gffff7/R27U1HL2biIio+UgaluLi4pCfn48lS5ZAo9EgNDQUu3btQmBgIABAo9GYjH00bdo0FBcXY82aNXjxxRfh5uaGYcOG4a233jIuU1hYiGeffRbZ2dlQq9Xo27cvfvrpJwwcOLDR27U1HL2biIio+ciEO52/ogZptVqo1WoUFRXB1dVVsjp0VQaELPwOggAcnf8wvFyss18VERGRNWjK8Vvyq+Ho3miKbkKoHmPJs51S6nKIiIhaHYYlG1e7vxLHWCIiIrI8hiUbd6u/Eq+EIyIiag4MSzaOV8IRERE1L4YlG1cTlgI82LJERETUHBiWbByHDSAiImpeDEs2jqN3ExERNS+GJRumqzIgW1sOgC1LREREzYVhyYbVjLHkoLBDe2eOsURERNQcGJZsWEbBrVNwHGOJiIioeTAs2TB27iYiImp+DEs2jGMsERERNT+GJRvG0buJiIiaH8OSDTMOSMmwRERE1GwYlmwYT8MRERE1P4YlG1VRpUdOMcdYIiIiam4MSzZKU1gOQQAcFXJ4cIwlIiKiZsOwZKNqn4LjGEtERETNh2HJRmVwjCUiIqIWwbBkozhsABERUctgWLJRvBKOiIioZTAs2ahbYYktS0RERM2JYclG8b5wRERELYNhyQZVVOmRo60AAAR4sGWJiIioOTEs2aBrheJglE5KOdydFBJXQ0RE1LoxLNmg2qfgOMYSERFR82JYskHs3E1ERNRyGJZsEDt3ExERtRyGJRuUUcAxloiIiFoKw5IN4ujdRERELYdhyQZx9G4iIqKWI3lYWrt2LYKDg+Hg4ICIiAgcPHiwweW3bt2K8PBwODk5wdfXF08//TTy8/ON72/YsAFDhgyBu7s73N3d8fDDDyM5OdlkHYsWLYJMJjOZfHx8muX7WVp5pR65xeIYS2xZIiIian6ShqXt27djzpw5mD9/Po4fP44hQ4Zg5MiRSE9Pr3f5Q4cOYcqUKZg+fTpSU1Px+eef4+jRo5gxY4Zxmf3792PSpEn48ccf8csvv6BTp06IjY1FVlaWybp69+4NjUZjnE6fPt2s39VSrhWKrUrOHGOJiIioRUgallatWoXp06djxowZ6NmzJ1avXo2AgACsW7eu3uUPHz6MoKAgzJ49G8HBwRg8eDCee+45HDt2zLjM1q1b8Ze//AX33XcfevTogQ0bNsBgMOD77783WZe9vT18fHyMk5eXV7N+V0upPWwAx1giIiJqfpKFJZ1Oh5SUFMTGxprMj42NRVJSUr2fiY6ORmZmJnbt2gVBEJCTk4MdO3Zg1KhRd9xOWVkZKisr4eHhYTI/LS0Nfn5+CA4OxsSJE3Hp0qUG662oqIBWqzWZpMD+SkRERC1LsrCUl5cHvV4Pb29vk/ne3t7Izs6u9zPR0dHYunUr4uLioFQq4ePjAzc3N7z33nt33M7LL7+Mjh074uGHHzbOi4yMxJYtW7Bnzx5s2LAB2dnZiI6ONun7dLuEhASo1WrjFBAQYOY3tgyOsURERNSyJO/gffupJEEQ7nh66ezZs5g9ezZeffVVpKSkYPfu3bh8+TJmzZpV7/LLly/Hp59+ip07d8LBwcE4f+TIkfjDH/6AsLAwPPzww/j2228BAJs3b75jnfHx8SgqKjJOGRkZ5n5Vi+Do3URERC3LXqoNe3p6Qi6X12lFys3NrdPaVCMhIQExMTF46aWXAAB9+vSBs7MzhgwZgqVLl8LX19e47Ntvv41ly5Zh37596NOnT4O1ODs7IywsDGlpaXdcRqVSQaVSNfbrNZsMtiwRERG1KMlalpRKJSIiIpCYmGgyPzExEdHR0fV+pqysDHZ2piXL5XIAYotUjRUrVuD111/H7t270b9//7vWUlFRgXPnzpmELWvFliUiIqKWJelpuLlz5+LDDz/Epk2bcO7cOfz9739Henq68bRafHw8pkyZYlx+zJgx2LlzJ9atW4dLly7h559/xuzZszFw4ED4+fkBEE+9LViwAJs2bUJQUBCys7ORnZ2NkpIS43rmzZuHAwcO4PLlyzhy5AgmTJgArVaLqVOntuwOMFN5pR7XjWMssWWJiIioJUh2Gg4A4uLikJ+fjyVLlkCj0SA0NBS7du1CYGAgAECj0ZiMuTRt2jQUFxdjzZo1ePHFF+Hm5oZhw4bhrbfeMi6zdu1a6HQ6TJgwwWRbr732GhYtWgQAyMzMxKRJk5CXlwcvLy8MGjQIhw8fNm7XWmXVGmPJjWMsERERtQiZUPv8FTWaVquFWq1GUVERXF1dW2SbBy5cx9RNyejh44Ldc+5vkW0SERG1Jk05fkt+NRw1HocNICIiankMSzaEnbuJiIhaHsOSDeHo3URERC2PYcmG8DQcERFRy2NYsiEZBTwNR0RE1NIYlmxEeaUeeSUcY4mIiKilMSzZiJr+Su1U9lA7cowlIiKilsKwZCNq91e6042GiYiIyPIYlmwEr4QjIiKSBsOSjeAYS0RERNJgWLIRHDaAiIhIGgxLNoItS0RERNJgWLIR7LNEREQkDYYlG1B7jKUAtiwRERG1KIYlG1DTX8lFZQ9XR3uJqyEiImpbGJZsQEb1KbiOHGOJiIioxTEs2QB27iYiIpIOw5IN4LABRERE0mFYsgE1LUsBHmxZIiIiamnsLWwDOGwAEVHbYjAYoNPppC7DJikUCsjlcouuk2HJBmTxNBwRUZuh0+lw+fJlGAwGqUuxWW5ubvDx8bHYRVEMS1bupk6PvBLxrwt28CYiat0EQYBGo4FcLkdAQADs7NhbxhyCIKCsrAy5ubkAAF9fX4usl2HJymUVVo+x5GAPtaNC4mqIiKg5VVVVoaysDH5+fnBy4h/ITeHoKJ6Fyc3NRYcOHSxySo6R1cplFHDYACKitkKv1wMAlEqlxJXYtpqgWVlZaZH1MSxZOQ4bQETU9nAA4ntj6f3HsGTleCUcERGRtBiWrBxH7yYiorYmKCgIq1evlroMI3bwtnI1p+EC2LJERERW7MEHH8R9991nkZBz9OhRODs733tRFsKwZOXYskRERK2BIAjQ6/Wwt7979PDy8mqBihqPp+GsWJmuCvml4hhLHdmyRETU5giCgDJdlSSTIAiNrnPatGk4cOAA3nnnHchkMshkMnz88ceQyWTYs2cP+vfvD5VKhYMHD+LixYsYO3YsvL290a5dOwwYMAD79u0zWd/tp+FkMhk+/PBDjB8/Hk5OTujWrRu++eYbS+3mu2LLkhXLqm5VcuUYS0REbdLNSj16vbpHkm2fXTICTsrGxYR33nkHFy5cQGhoKJYsWQIASE1NBQD84x//wNtvv43OnTvDzc0NmZmZePTRR7F06VI4ODhg8+bNGDNmDM6fP49OnTrdcRuLFy/G8uXLsWLFCrz33nv405/+hKtXr8LDw+Pev+xdsGXJivEUHBER2QK1Wg2lUgknJyf4+PjAx8fHOBjkkiVLMHz4cHTp0gXt27dHeHg4nnvuOYSFhaFbt25YunQpOnfufNeWomnTpmHSpEno2rUrli1bhtLSUiQnJ7fE15O+ZWnt2rVYsWIFNBoNevfujdWrV2PIkCF3XH7r1q1Yvnw50tLSoFar8cgjj+Dtt99G+/btjct88cUXWLhwIS5evIguXbrgjTfewPjx4+9pu1LgGEtERG2bo0KOs0tGSLZtS+jfv7/J69LSUixevBj/93//h2vXrqGqqgo3b95Eenp6g+vp06eP8bmzszNcXFyMtzVpbpK2LG3fvh1z5szB/Pnzcfz4cQwZMgQjR4684w47dOgQpkyZgunTpyM1NRWff/45jh49ihkzZhiX+eWXXxAXF4fJkyfj5MmTmDx5Mp588kkcOXKkyduVSgZbloiI2jSZTAYnpb0kk6UGdrz9qraXXnoJX3zxBd544w0cPHgQJ06cQFhYGHQ6XYPrUShMu6PIZLIWu9mwpGFp1apVmD59OmbMmIGePXti9erVCAgIwLp16+pd/vDhwwgKCsLs2bMRHByMwYMH47nnnsOxY8eMy6xevRrDhw9HfHw8evTogfj4eDz00EMmHcXM3S4AVFRUQKvVmkzNjS1LRERkK5RKpfF2LQ05ePAgpk2bhvHjxyMsLAw+Pj64cuVK8xd4DyQLSzqdDikpKYiNjTWZHxsbi6SkpHo/Ex0djczMTOzatQuCICAnJwc7duzAqFGjjMv88ssvddY5YsQI4zqbsl0ASEhIgFqtNk4BAQFmfd+m4OjdRERkK4KCgnDkyBFcuXIFeXl5d2z16dq1K3bu3IkTJ07g5MmT+OMf/9hiLURNJVlYysvLg16vh7e3t8l8b29vZGdn1/uZ6OhobN26FXFxcVAqlfDx8YGbmxvee+894zLZ2dkNrrMp2wWA+Ph4FBUVGaeMjAyzvm9TsIM3ERHZinnz5kEul6NXr17w8vK6Y9eWf/3rX3B3d0d0dDTGjBmDESNGoF+/fi1crXkk7+B9+zlRQRDueJ707NmzmD17Nl599VWMGDECGo0GL730EmbNmoWNGzeatU5ztgsAKpUKKpWqUd/JEkorqlBQPcaSvwdbloiIyLp1794dv/zyi8m8adOm1VkuKCgIP/zwg8m8559/3uT17afl6hvzqbCwsEl1NoVkYcnT0xNyubxOa05ubm6dVp8aCQkJiImJwUsvvQRA7Bnv7OyMIUOGYOnSpfD19YWPj0+D62zKdqWQVSi2KqkdFXB14BhLREREUpHsNJxSqURERAQSExNN5icmJiI6Orrez5SVlcHOzrTkmnEcalJnVFRUnXXu3bvXuM6mbFcK7NxNRERkHSQ9DTd37lxMnjwZ/fv3R1RUFNavX4/09HTMmjULgNhPKCsrC1u2bAEAjBkzBjNnzsS6deuMp+HmzJmDgQMHws/PDwDwwgsv4P7778dbb72FsWPH4uuvv8a+fftw6NChRm/XGrBzNxERkXWQNCzFxcUhPz8fS5YsgUajQWhoKHbt2oXAwEAAgEajMekgNm3aNBQXF2PNmjV48cUX4ebmhmHDhuGtt94yLhMdHY1t27ZhwYIFWLhwIbp06YLt27cjMjKy0du1BuzcTUREZB1kgjl3yiMjrVYLtVqNoqIiuLq6Wnz9f/5vCr47k43XxvTC0zHBFl8/ERFZn/Lycly+fBnBwcFwcHCQuhyb1dB+bMrxm/eGs1JsWSIiIrIODEtWih28iYiIrAPDkhUqqajCjbJKAAxLREREUmNYskJZ1afg3JwUcOEYS0RERJJiWLJCPAVHRERkPRiWrJCxc7cbO3cTEZFtePDBBzFnzhyLrW/atGkYN26cxdZ3LxiWrBBbloiIiKwHw5IV4ujdREQEABAEQFcqzWTGMIzTpk3DgQMH8M4770Amk0Emk+HKlSs4e/YsHn30UbRr1w7e3t6YPHky8vLyjJ/bsWMHwsLC4OjoiPbt2+Phhx9GaWkpFi1ahM2bN+Prr782rm///v3NsIMbR9IRvKl+HGOJiIgAAJVlwDI/abb9yjVA6dyoRd955x1cuHABoaGhWLJkCQBAr9fjgQcewMyZM7Fq1SrcvHkT//znP/Hkk0/ihx9+gEajwaRJk7B8+XKMHz8excXFOHjwIARBwLx583Du3DlotVp89NFHAAAPD49m+6p3w7BkhTJqTsN5sGWJiIisn1qthlKphJOTE3x8fAAAr776Kvr164dly5YZl9u0aRMCAgJw4cIFlJSUoKqqCo8//rjxdmNhYWHGZR0dHVFRUWFcn5QYlqxMcXklCqvHWOroxrBERNSmKZzEFh6ptn0PUlJS8OOPP6Jdu3Z13rt48SJiY2Px0EMPISwsDCNGjEBsbCwmTJgAd3f3e9puc2BYsjJZhRxjiYiIqslkjT4VZm0MBgPGjBljcrP7Gr6+vpDL5UhMTERSUhL27t2L9957D/Pnz8eRI0cQHGxd90RlB28rk1kghqUA9lciIiIbolQqodfrja/79euH1NRUBAUFoWvXriaTs7MYAGUyGWJiYrB48WIcP34cSqUSX375Zb3rkxLDkpUprqiEs1LOK+GIiMimBAUF4ciRI7hy5Qry8vLw/PPPo6CgAJMmTUJycjIuXbqEvXv34plnnoFer8eRI0ewbNkyHDt2DOnp6di5cyeuX7+Onj17Gtd36tQpnD9/Hnl5eaisrJTsuzEsWZnxff1xZvEI/CvuPqlLISIiarR58+ZBLpejV69e8PLygk6nw88//wy9Xo8RI0YgNDQUL7zwAtRqNezs7ODq6oqffvoJjz76KLp3744FCxZg5cqVGDlyJABg5syZCAkJQf/+/eHl5YWff/5Zsu8mEwQzBlIgI61WC7VajaKiIri6ukpdDhERtQLl5eW4fPkygoOD4eDgIHU5Nquh/diU4zdbloiIiIgawLBERERE1ACGJSIiIqIGMCwRERERNYBhiYiIyMrw2qt7Y+n9x7BERERkJeRyOQBAp9NJXIltKysT77GqUFjmThi83QkREZGVsLe3h5OTE65fvw6FQgE7O7ZpmEMQBJSVlSE3Nxdubm7G8HmvGJaIiIishEwmg6+vLy5fvoyrV69KXY7NcnNzg4+Pj8XWx7BERERkRZRKJbp168ZTcU2kUCgs1qJUg2GJiIjIytjZ2XEEbyvCk6FEREREDWBYIiIiImoAwxIRERFRA9hnqYlqBrzSarUSV0JERESNVXPcNmfgSoalJiouLgYABAQESFwJERERmau4uBhqtbpRy8oEjqneJAaDAdeuXYOLiwtkMplF163VahEQEICMjAy4urpadN2tFfdZ03C/NQ33W9Nwv5mP+6xpGtpvgiCguLgYfn5+jR70ky1LTWRnZwd/f/9m3Yarqyv/z2Em7rOm4X5rGu63puF+Mx/3WdPcab81tkWpBjt4ExERETWAYYmIiIioAQxLVkilUuG1116DSqWSuhSbwX3WNNxvTcP91jTcb+bjPmsaS+83dvAmIiIiagBbloiIiIgawLBERERE1ACGJSIiIqIGMCwRERERNYBhycqsXbsWwcHBcHBwQEREBA4ePCh1SVZt0aJFkMlkJpOPj4/UZVmdn376CWPGjIGfnx9kMhm++uork/cFQcCiRYvg5+cHR0dHPPjgg0hNTZWmWCtyt/02bdq0Or+/QYMGSVOslUhISMCAAQPg4uKCDh06YNy4cTh//rzJMvy91dWY/cbfm6l169ahT58+xoEno6Ki8N133xnft+TvjGHJimzfvh1z5szB/Pnzcfz4cQwZMgQjR45Eenq61KVZtd69e0Oj0Rin06dPS12S1SktLUV4eDjWrFlT7/vLly/HqlWrsGbNGhw9ehQ+Pj4YPny48R6IbdXd9hsAPPLIIya/v127drVghdbnwIEDeP7553H48GEkJiaiqqoKsbGxKC0tNS7D31tdjdlvAH9vtfn7++PNN9/EsWPHcOzYMQwbNgxjx441BiKL/s4EshoDBw4UZs2aZTKvR48ewssvvyxRRdbvtddeE8LDw6Uuw6YAEL788kvja4PBIPj4+AhvvvmmcV55ebmgVquFf//73xJUaJ1u32+CIAhTp04Vxo4dK0k9tiI3N1cAIBw4cEAQBP7eGuv2/SYI/L01hru7u/Dhhx9a/HfGliUrodPpkJKSgtjYWJP5sbGxSEpKkqgq25CWlgY/Pz8EBwdj4sSJuHTpktQl2ZTLly8jOzvb5LenUqnwwAMP8LfXCPv370eHDh3QvXt3zJw5E7m5uVKXZFWKiooAAB4eHgD4e2us2/dbDf7e6qfX67Ft2zaUlpYiKirK4r8zhiUrkZeXB71eD29vb5P53t7eyM7Olqgq6xcZGYktW7Zgz5492LBhA7KzsxEdHY38/HypS7MZNb8v/vbMN3LkSGzduhU//PADVq5ciaNHj2LYsGGoqKiQujSrIAgC5s6di8GDByM0NBQAf2+NUd9+A/h7q8/p06fRrl07qFQqzJo1C19++SV69epl8d+ZvUWqJYuRyWQmrwVBqDOPbhk5cqTxeVhYGKKiotClSxds3rwZc+fOlbAy28Pfnvni4uKMz0NDQ9G/f38EBgbi22+/xeOPPy5hZdbhr3/9K06dOoVDhw7VeY+/tzu7037j762ukJAQnDhxAoWFhfjiiy8wdepUHDhwwPi+pX5nbFmyEp6enpDL5XUSb25ubp1kTHfm7OyMsLAwpKWlSV2Kzai5epC/vXvn6+uLwMBA/v4A/O1vf8M333yDH3/8Ef7+/sb5/L017E77rT78vQFKpRJdu3ZF//79kZCQgPDwcLzzzjsW/50xLFkJpVKJiIgIJCYmmsxPTExEdHS0RFXZnoqKCpw7dw6+vr5Sl2IzgoOD4ePjY/Lb0+l0OHDgAH97ZsrPz0dGRkab/v0JgoC//vWv2LlzJ3744QcEBwebvM/fW/3utt/qw99bXYIgoKKiwvK/Mwt0PicL2bZtm6BQKISNGzcKZ8+eFebMmSM4OzsLV65ckbo0q/Xiiy8K+/fvFy5duiQcPnxYGD16tODi4sJ9dpvi4mLh+PHjwvHjxwUAwqpVq4Tjx48LV69eFQRBEN58801BrVYLO3fuFE6fPi1MmjRJ8PX1FbRarcSVS6uh/VZcXCy8+OKLQlJSknD58mXhxx9/FKKiooSOHTu26f325z//WVCr1cL+/fsFjUZjnMrKyozL8PdW1932G39vdcXHxws//fSTcPnyZeHUqVPCK6+8ItjZ2Ql79+4VBMGyvzOGJSvz/vvvC4GBgYJSqRT69etnctko1RUXFyf4+voKCoVC8PPzEx5//HEhNTVV6rKszo8//igAqDNNnTpVEATxcu7XXntN8PHxEVQqlXD//fcLp0+flrZoK9DQfisrKxNiY2MFLy8vQaFQCJ06dRKmTp0qpKenS122pOrbXwCEjz76yLgMf2913W2/8fdW1zPPPGM8Xnp5eQkPPfSQMSgJgmV/ZzJBEIQmtHQRERERtQnss0RERETUAIYlIiIiogYwLBERERE1gGGJiIiIqAEMS0REREQNYFgiIiIiagDDEhEREVEDGJaIiIiIGsCwRERkIfv374dMJkNhYaHUpRCRBTEsERERETWAYYmIiIioAQxLRNRqCIKA5cuXo3PnznB0dER4eDh27NgB4NYpsm+//Rbh4eFwcHBAZGQkTp8+bbKOL774Ar1794ZKpUJQUBBWrlxp8n5FRQX+8Y9/ICAgACqVCt26dcPGjRtNlklJSUH//v3h5OSE6OhonD9/vnm/OBE1K4YlImo1FixYgI8++gjr1q1Damoq/v73v+Opp57CgQMHjMu89NJLePvtt3H06FF06NABjz32GCorKwGIIefJJ5/ExIkTcfr0aSxatAgLFy7Exx9/bPz8lClTsG3bNrz77rs4d+4c/v3vf6Ndu3YmdcyfPx8rV67EsWPHYG9vj2eeeaZFvj8RNQ+ZIAiC1EUQEd2r0tJSeHp64ocffkBUVJRx/owZM1BWVoZnn30WQ4cOxbZt2xAXFwcAKCgogL+/Pz7++GM8+eST+NOf/oTr169j7969xs//4x//wLfffovU1FRcuHABISEhSExMxMMPP1ynhv3792Po0KHYt28fHnroIQDArl27MGrUKNy8eRMODg7NvBeIqDmwZYmIWoWzZ8+ivLwcw4cPR7t27YzTli1bcPHiReNytYOUh4cHQkJCcO7cOQDAuXPnEBMTY7LemJgYpKWlQa/X48SJE5DL5XjggQcarKVPnz7G576+vgCA3Nzce/6ORCQNe6kLICKyBIPBAAD49ttv0bFjR5P3VCqVSWC6nUwmAyD2eap5XqN247ujo2OjalEoFHXWXVMfEdketiwRUavQq1cvqFQqpKeno2vXriZTQECAcbnDhw8bn9+4cQMXLlxAjx49jOs4dOiQyXqTkpLQvXt3yOVyhIWFwWAwmPSBIqLWjy1LRNQquLi4YN68efj73/8Og8GAwYMHQ6vVIikpCe3atUNgYCAAYMmSJWjfvj28vb0xf/58eHp6Yty4cQCAF198EQMGDMDrr7+OuLg4/PLLL1izZg3Wrl0LAAgKCsLUqVPxzDPP4N1330V4eDiuXr2K3NxcPPnkk1J9dSJqZgxLRNRqvP766+jQoQMSEhJw6dIluLm5oV+/fnjllVeMp8HefPNNvPDCC0hLS0N4eDi++eYbKJVKAEC/fv3w2Wef4dVXX8Xrr78OX19fLFmyBNOmTTNuY926dXjllVfwl7/8Bfn5+ejUqRNeeeUVKb4uEbUQXg1HRG1CzZVqN27cgJubm9TlEJENYZ8lIiIiogYwLBERERE1gKfhiIiIiBrAliUiIiKiBjAsERERETWAYYmIiIioAQxLRERERA1gWCIiIiJqAMMSERERUQMYloiIiIgawLBERERE1ID/B+HgZ4zNpeKRAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjkAAAGwCAYAAABLvHTgAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAA9hAAAPYQGoP6dpAABatklEQVR4nO3de1zT9f4H8NcYbNzHfYBcvSMoKqiBmZZFmV3UU5p2vJR18ledMqtzMjtldrFjaXYyLbOLVqestJOlmZR3MW9h3i+pCMpgch3XDbbv748vGyKgA7Z9Ybyej8ceG999t71Zy734XGWCIAggIiIicjIuUhdAREREZA8MOUREROSUGHKIiIjIKTHkEBERkVNiyCEiIiKnxJBDRERETokhh4iIiJySq9QFOJrJZEJubi58fHwgk8mkLoeIiIisIAgCysrKEB4eDhcX69poOl3Iyc3NRWRkpNRlEBERUSvk5OQgIiLCqnM7Xcjx8fEBIL5Jvr6+EldDRERE1tDpdIiMjLR8j1uj04UccxeVr68vQw4REVEH05KhJhx4TERERE6JIYeIiIicEkMOEREROaVONyaHiIjInoxGI2pqaqQuo0NSKBRWTw+3BkMOERGRDQiCgLy8PJSUlEhdSofl4uKC2NhYKBQKmzwfQw4REZENmANOSEgIPD09ueBsC5kX69VoNIiKirLJ+8eQQ0RE1EZGo9EScAIDA6Uup8MKDg5Gbm4uamtr4ebm1ubn48BjIiKiNjKPwfH09JS4ko7N3E1lNBpt8nwMOURERDbCLqq2sfX7x5BDRERETokhh4iIiJwSQw4RERHZRExMDBYvXix1GRacXWUjgiCgqMKAogoDeqit3yGViIhISiNGjED//v1tEk727dsHLy+vthdlIww5NnKuoAI3LdwGL4UcR+fdJnU5RERENiEIAoxGI1xdrx0ZgoODHVCR9dhdZSOhKncAQIXBiLJqLudNRNTZCYKASkOtwy+CIFhd47Rp07Bt2za88847kMlkkMlk+PTTTyGTyfDzzz8jOTkZSqUSO3bswJkzZ3D33XdDrVbD29sbgwYNwi+//NLg+a7srpLJZFixYgXGjh0LT09P9OjRA+vWrbPVW3xNbMmxEU+FK3zdXaGrrkVeaTV83Nu+iBEREXVcVTVG9HnxZ4e/7rF5t8JTYd3X+zvvvINTp04hISEB8+bNAwAcPXoUAPCPf/wDb731Frp27Qo/Pz9cuHABt99+O1599VW4u7tj5cqVuPPOO3Hy5ElERUU1+xovv/wyFixYgDfffBPvvvsu7r//fpw/fx4BAQFt/2WvgS05NhSm8gAA5OmqJa6EiIjo2lQqFRQKBTw9PREaGorQ0FDI5XIAwLx583DLLbegW7duCAwMRGJiIh555BH07dsXPXr0wKuvvoquXbtes2Vm2rRpmDhxIrp3747XX38dFRUV2Lt3ryN+Pbbk2JJa5Y6T+WXQlDLkEBF1dh5uchybd6skr2sLycnJDX6uqKjAyy+/jB9//NGy9UJVVRWys7Ov+jz9+vWz3Pby8oKPjw+0Wq1NarwWhhwbCvMVx+XkM+QQEXV6MpnM6m6j9ujKWVLPPvssfv75Z7z11lvo3r07PDw8cM8998BgMFz1ea7cg0omk8FkMtm83qZ03He/HVLXDT7WsLuKiIg6CIVCYdVeUTt27MC0adMwduxYAEB5eTmysrLsXF3bcEyODYWp2JJDREQdS0xMDPbs2YOsrCwUFBQ028rSvXt3rF27FgcPHsQff/yBSZMmOaxFprUYcmwotK67imNyiIioo3jmmWcgl8vRp08fBAcHNzvG5u2334a/vz9SU1Nx55134tZbb8XAgQMdXG3LsLvKhsxr5eSzu4qIiDqInj17Yvfu3Q2OTZs2rdF5MTEx2Lx5c4Njjz32WIOfr+y+amrNnpKSklbV2RpsybEhc0tOYYUB+tpr928SERGR/TDk2JCfpxuUruJbqtXpJa6GiIioc5M85CxduhSxsbFwd3dHUlISduzY0ey506ZNsyw7ffklPj7egRU3TyaTWbqsOC6HiIhIWpKGnNWrV2PmzJmYM2cOMjMzMWzYMIwaNarZQU/vvPMONBqN5ZKTk4OAgADce++9Dq68efWDj6skroSIiKhzkzTkLFq0CNOnT8dDDz2EuLg4LF68GJGRkVi2bFmT56tUKsuy06Ghodi/fz+Ki4vxwAMPOLjy5nHwMRERUfsgWcgxGAw4cOAA0tLSGhxPS0tDRkaGVc/x0Ucf4eabb0Z0dHSz5+j1euh0ugYXe2J3FRERUfsgWcgpKCiA0WiEWq1ucFytViMvL++aj9doNPjpp5/w0EMPXfW8+fPnQ6VSWS6RkZFtqvtazN1VbMkhIiKSluQDj2UyWYOfBUFodKwpn376Kfz8/DBmzJirnjd79myUlpZaLjk5OW0p95rC2JJDRETULki2GGBQUBDkcnmjVhutVtuodedKgiDg448/xuTJk6FQKK56rlKphFKpbHO91gpVeQDg1g5ERERSk6wlR6FQICkpCenp6Q2Op6enIzU19aqP3bZtG/78809Mnz7dniW2iqW7qkwPo6nxSo9ERETtyYgRIzBz5kybPd+0adOu2cviKJJu6zBr1ixMnjwZycnJSElJwfLly5GdnY0ZM2YAELuaLl68iFWrVjV43EcffYQhQ4YgISFBirKvKthHCbmLDEaTgMJyPULqQg8RERE5lqRjciZMmIDFixdj3rx56N+/P7Zv344NGzZYZktpNJpGa+aUlpZizZo17bIVBwDkLjIEe4vdYxyXQ0RE7dm0adOwbds2vPPOO5YFdrOysnDs2DHcfvvt8Pb2hlqtxuTJk1FQUGB53Lfffou+ffvCw8MDgYGBuPnmm1FRUYG5c+di5cqV+P777y3Pt3XrVsl+P5nQ1O5ZTkyn00GlUqG0tBS+vr52eY0x7+3CwZwSfDA5CbfGh9rlNYiIqP2orq7GuXPnLCv4AwAEAaipdHwxbp6AFRN4ALHhYNSoUUhISMC8efMAAEajEf3798fDDz+MKVOmoKqqCv/85z9RW1uLzZs3Q6PRICoqCgsWLMDYsWNRVlaGHTt2YMqUKQCA6dOnQ6fT4ZNPPgEABAQEXHP8rFmT72Od1nx/cxdyOzCPy8ljSw4RUedVUwm8Hu74130+F1B4WXWqSqWCQqGAp6cnQkPFP8pffPFFDBw4EK+//rrlvI8//hiRkZE4deoUysvLUVtbi3Hjxll6Xvr27Ws518PDA3q93vJ8UmLIsQPzgoB5XCuHiIg6mAMHDmDLli3w9vZudN+ZM2eQlpaGkSNHom/fvrj11luRlpaGe+65B/7+/hJUe3UMOXZgCTlsySEi6rzcPMVWFSletw1MJhPuvPNO/Pvf/250X1hYGORyOdLT05GRkYFNmzbh3XffxZw5c7Bnzx7Exsa26bVtjSHHDsIYcoiISCazuttISgqFAkaj0fLzwIEDsWbNGsTExMDVtemYIJPJMHToUAwdOhQvvvgioqOj8d1332HWrFmNnk9Kkq947IzUvuyuIiKijiEmJgZ79uxBVlYWCgoK8Nhjj6GoqAgTJ07E3r17cfbsWWzatAkPPvggjEYj9uzZg9dffx379+9HdnY21q5di0uXLiEuLs7yfIcOHcLJkydRUFCAmpoayX43hhw7uLwlp5NNXiMiog7mmWeegVwuR58+fRAcHAyDwYBdu3bBaDTi1ltvRUJCAp588kmoVCq4uLjA19cX27dvx+23346ePXvihRdewMKFCzFq1CgAwMMPP4xevXohOTkZwcHB2LVrl2S/G7ur7MDcklNVY4SuqhYqTzeJKyIiImpaz549sXv37kbH165d2+T5cXFx2LhxY7PPFxwcjE2bNtmsvrZgS44duLvJ4V8XbNhlRUREJA2GHDsxt+ZoSqskroSIiKhzYsixE/O4nHy25BAREUmCIcdOzGvlcP8qIiIiaTDk2EmorwcAtuQQEXUmnFHbNrZ+/xhy7CRUxZ3IiYg6Czc3cbJJZaUEG3I6EYPBAACQy+U2eT5OIbeTUJXYksNVj4mInJ9cLoefnx+0Wi0AwNPTEzIrdwInkclkwqVLl+Dp6dnsSsstxZBjJ6Fc9ZiIqFMx77ptDjrUci4uLoiKirJZQGTIsRPzwOOSyhpU1xjh7mabpjciImqfZDIZwsLCEBISIulWBh2ZQqGAi4vtRtIw5NiJr7srPNzkqKoxIq+0GjFB7X+TNiIiaju5XG6zMSXUNhx4bCcymcyyVg4HHxMRETkeQ44dmVc95jRyIiIix2PIsSO25BAREUmHIceOQrm1AxERkWQYcuyofmsHbtJJRETkaAw5dlS/Vo5e4kqIiIg6H4YcOzK35OSxJYeIiMjhGHLsyBxyLpXpUWs0SVwNERFR58KQY0dBXkq4ushgEoBL5eyyIiIiciSGHDtycZFZ1srhRp1ERESOxZBjZ2pfJQCGHCIiIkdjyLGzMJUHAO5GTkRE5GgMOXbG7ioiIiJpMOTYmXlrB7bkEBERORZDjp2puX8VERGRJBhy7CyM+1cRERFJgiHHzsxbO2hKqyEIgsTVEBERdR4MOXYWUjeF3FBrQklljcTVEBERdR4MOXamdJUj0EsBgONyiIiIHIkhxwEsG3XquFEnERGRozDkOECoZa0c7l9FRETkKAw5DmBpySllSw4REZGjMOQ4gKUlh9PIiYiIHIYhxwFCuSAgERGRwzHkOEAoFwQkIiJyOMlDztKlSxEbGwt3d3ckJSVhx44dVz1fr9djzpw5iI6OhlKpRLdu3fDxxx87qNrWCWNLDhERkcO5Svniq1evxsyZM7F06VIMHToUH3zwAUaNGoVjx44hKiqqyceMHz8e+fn5+Oijj9C9e3dotVrU1tY6uPKWMe9EXlZdiwp9LbyUkr7tREREnYJMkHCvgSFDhmDgwIFYtmyZ5VhcXBzGjBmD+fPnNzp/48aNuO+++3D27FkEBARY9Rp6vR56ff3UbZ1Oh8jISJSWlsLX17ftv4SVEl76GeX6Wvz69HB0C/Z22OsSERE5A51OB5VK1aLvb8m6qwwGAw4cOIC0tLQGx9PS0pCRkdHkY9atW4fk5GQsWLAAXbp0Qc+ePfHMM8+gqqr5qdnz58+HSqWyXCIjI236e1jLMi6HXVZEREQOIVnIKSgogNFohFqtbnBcrVYjLy+vycecPXsWO3fuxJEjR/Ddd99h8eLF+Pbbb/HYY481+zqzZ89GaWmp5ZKTk2PT38Nal2/USURERPYn+eAQmUzW4GdBEBodMzOZTJDJZPjiiy+gUqkAAIsWLcI999yD9957Dx4eHo0eo1QqoVQqbV94C9Vv7cCQQ0RE5AiSteQEBQVBLpc3arXRarWNWnfMwsLC0KVLF0vAAcQxPIIg4MKFC3att63qt3ZgyCEiInIEyUKOQqFAUlIS0tPTGxxPT09Hampqk48ZOnQocnNzUV5ebjl26tQpuLi4ICIiwq71thVbcoiIiBxL0nVyZs2ahRUrVuDjjz/G8ePH8dRTTyE7OxszZswAII6nmTJliuX8SZMmITAwEA888ACOHTuG7du349lnn8WDDz7YZFdVe8KWHCIiIseSdEzOhAkTUFhYiHnz5kGj0SAhIQEbNmxAdHQ0AECj0SA7O9tyvre3N9LT0/H3v/8dycnJCAwMxPjx4/Hqq69K9StYjS05REREjiXpOjlSaM08e1soKNcj+dVfIJMBp14dBTe55ItNExERdRgdap2czibAUwGF3AWCAGjL9Nd+ABEREbUJQ46DuLjIEOIrTmXPK21+8UIiIiKyDYYcBzJv1JlXypYcIiIie2PIcSC1ZdVjtuQQERHZG0OOA5lbcvI5w4qIiMjuGHIcSM39q4iIiByGIceBwlTigoVcEJCIiMj+GHIcKFRVN7uK3VVERER2x5DjQKF1LTn5umqYTJ1qDUYiIiKHY8hxoBAfJWQyoMYooKjSIHU5RERETo0hx4Hc5C4I8jYvCMguKyIiIntiyHEw7kZORETkGAw5DmbejVzDwcdERER2xZDjYOaWnHy25BAREdkVQ46DWVpyGHKIiIjsiiHHwbi1AxERkWMw5DhYKDfpJCIicgiGHAcLtbTk6CWuhIiIyLkx5DiYOeSU62tRVl0jcTVERETOiyHHwTwVrvB1dwXAcTlERET2xJAjAc6wIiIisj+GHAmYN+rkqsdERET2w5AjgVBf7l9FRERkbww5ErC05HBMDhERkd0w5EiAm3QSERHZH0OOBMyrHrMlh4iIyH4YciSgZksOERGR3THkSMDcklNYYYC+1ihxNURERM6JIUcCfp5uULiKb72W2zsQERHZBUOOBGQyGcflEBER2RlDjkTUvlz1mIiIyJ4YciRiackprZK4EiIiIufEkCOR+rVyOCaHiIjIHhhyJBJqGZPDlhwiIiJ7YMiRCFc9JiIisi+GHIlYWnIYcoiIiOyCIUci5pCjLdPDaBIkroaIiMj5MORIJNhbCRcZUGsSUFjOwcdERES2xpAjEVe5C4J9lAC4ICAREZE9MORIKFTlAYALAhIREdkDQ46EwupmWOWzJYeIiMjmJA85S5cuRWxsLNzd3ZGUlIQdO3Y0e+7WrVshk8kaXU6cOOHAim3HPPiYLTlERES2J2nIWb16NWbOnIk5c+YgMzMTw4YNw6hRo5CdnX3Vx508eRIajcZy6dGjh4Mqti1zyMlnyCEiIrI5SUPOokWLMH36dDz00EOIi4vD4sWLERkZiWXLll31cSEhIQgNDbVc5HJ5s+fq9XrodLoGl/YilJt0EhER2Y1kIcdgMODAgQNIS0trcDwtLQ0ZGRlXfeyAAQMQFhaGkSNHYsuWLVc9d/78+VCpVJZLZGRkm2u3FUtLDsfkEBER2ZxkIaegoABGoxFqtbrBcbVajby8vCYfExYWhuXLl2PNmjVYu3YtevXqhZEjR2L79u3Nvs7s2bNRWlpqueTk5Nj092iLy1tyBIELAhIREdmSq9QFyGSyBj8LgtDomFmvXr3Qq1cvy88pKSnIycnBW2+9hRtuuKHJxyiVSiiVStsVbEPmlpyqGiN01bVQebhJXBEREZHzkKwlJygoCHK5vFGrjVarbdS6czXXXXcdTp8+bevyHMLdTQ4/TzHYcA8rIiIi25Is5CgUCiQlJSE9Pb3B8fT0dKSmplr9PJmZmQgLC7N1eQ5j2Y2c43KIiIhsStLuqlmzZmHy5MlITk5GSkoKli9fjuzsbMyYMQOAOJ7m4sWLWLVqFQBg8eLFiImJQXx8PAwGAz7//HOsWbMGa9askfLXaJNQlTtO5JUhr7RK6lKIiIiciqQhZ8KECSgsLMS8efOg0WiQkJCADRs2IDo6GgCg0WgarJljMBjwzDPP4OLFi/Dw8EB8fDzWr1+P22+/Xapfoc3C6sbl5JVyk04iIiJbkgmdbFqPTqeDSqVCaWkpfH19pS4Hi385hcW/nMbEwZGYP66f1OUQERG1S635/pZ8W4fOrr4lh2NyiIiIbIkhR2JqrnpMRERkFww5EgtTeQDg7CoiIiJbY8iRmHkKeUllDaprjBJXQ0RE5DwYciTm6+EKDzdxg1GOyyEiIrIdhhyJyWQyy/YO7LIiIiKynRaHnJycHFy4cMHy8969ezFz5kwsX77cpoV1JpZVj9mSQ0REZDMtDjmTJk3Cli1bAAB5eXm45ZZbsHfvXjz//POYN2+ezQvsDNiSQ0REZHstDjlHjhzB4MGDAQBff/01EhISkJGRgf/+97/49NNPbV1fpxDKtXKIiIhsrsUhp6amBkqlEgDwyy+/4K677gIA9O7dGxqNxrbVdRLsriIiIrK9Foec+Ph4vP/++9ixYwfS09Nx2223AQByc3MRGBho8wI7A3NLjobdVURERDbT4pDz73//Gx988AFGjBiBiRMnIjExEQCwbt06SzcWtYx5a4d8tuQQERHZTIt3IR8xYgQKCgqg0+ng7+9vOf63v/0Nnp6eNi2uszB3V2nLqlFrNMFVzpn9REREbdXib9Oqqiro9XpLwDl//jwWL16MkydPIiQkxOYFdgaB3kq4ushgEoCCcoPU5RARETmFFoecu+++G6tWrQIAlJSUYMiQIVi4cCHGjBmDZcuW2bzAzkDuIkOIjziYW1NaJXE1REREzqHFIef333/HsGHDAADffvst1Go1zp8/j1WrVuE///mPzQvsLMyDj/M5+JiIiMgmWhxyKisr4ePjAwDYtGkTxo0bBxcXF1x33XU4f/68zQvsLCwzrDj4mIiIyCZaHHK6d++O//3vf8jJycHPP/+MtLQ0AIBWq4Wvr6/NC+wswlUeAIA/teUSV0JEROQcWhxyXnzxRTzzzDOIiYnB4MGDkZKSAkBs1RkwYIDNC+wshvYIAgBsOpYPo0mQuBoiIqKOr8VTyO+55x5cf/310Gg0ljVyAGDkyJEYO3asTYvrTK7vHgSVhxsulemx51whUrsFSV0SERFRh9aqBVlCQ0MxYMAA5Obm4uLFiwCAwYMHo3fv3jYtrjNxk7vgtvhQAMD6Q9weg4iIqK1aHHJMJhPmzZsHlUqF6OhoREVFwc/PD6+88gpMJpM9auw07kgMAwBsPJKHWiPfSyIiorZocXfVnDlz8NFHH+GNN97A0KFDIQgCdu3ahblz56K6uhqvvfaaPersFFK6BiLAS4HCCgN+O1uE63uwy4qIiKi1WhxyVq5ciRUrVlh2HweAxMREdOnSBY8++ihDThu4yl1wW0Io/rsnGz8eymXIISIiaoMWd1cVFRU1Ofamd+/eKCoqsklRndkd/eq6rI7moYZdVkRERK3W4pCTmJiIJUuWNDq+ZMmSBrOtqHWGxAYiyFuJksoa7PqzQOpyiIiIOqwWd1ctWLAAo0ePxi+//IKUlBTIZDJkZGQgJycHGzZssEeNnYrcRYbb+4Zi1e7z+PGQBiN6cdNTIiKi1mhxS87w4cNx6tQpjB07FiUlJSgqKsK4ceNw8uRJy55W1Daj+4pdVj8fzYO+1ihxNURERB1Ti1tyACA8PJwDjO1oUEwA1L5K5Ov02Hm6ACPj1FKXRERE1OFYFXIOHTpk9RP269ev1cWQyMVFhtv7huGTXVn48ZCGIYeIiKgVrAo5/fv3h0wmgyBcfU8lmUwGo5HdK7ZwRz8x5KQfy0d1jRHubnKpSyIiIupQrAo5586ds3cddIUBkf4IV7kjt7Qa205dwq11Wz4QERGRdawKOdHR0faug65g7rJasfMcfjykYcghIiJqoVZt0EmOcUdiOADg1+P5qDKwG5CIiKglGHLascQIFSL8PVBpMGLLSa3U5RAREXUoDDntmEwmw+i6bR7WH9JIXA0REVHHwpDTzt3Zr67L6kQ+KvS1EldDRETUcTDktHPx4b6IDvREdY0Jv55glxUREZG1rJpd5e/vD5lMZtUTcidy25LJZLijXxje23IG6w/l4q66wchERER0dVaFnMWLF9u5DLqa0X3D8d6WM9hy8hLKqmvg4+4mdUlERETtnlUhZ+rUqfaug64iLswHXYO9cPZSBX45no+xAyKkLomIiKjda9WYnDNnzuCFF17AxIkTodWK40Q2btyIo0ePtvi5li5ditjYWLi7uyMpKQk7duyw6nG7du2Cq6sr+vfv3+LX7GjELiuxm4qzrIiIiKzT4pCzbds29O3bF3v27MHatWtRXl4OQNzE86WXXmrRc61evRozZ87EnDlzkJmZiWHDhmHUqFHIzs6+6uNKS0sxZcoUjBw5sqXld1h31E0l33bqEkqraiSuhoiIqP1rcch57rnn8OqrryI9PR0KhcJy/MYbb8Tu3btb9FyLFi3C9OnT8dBDDyEuLg6LFy9GZGQkli1bdtXHPfLII5g0aRJSUlKu+Rp6vR46na7BpSPqqfZBT7U3aowC0o/lS10OERFRu9fikHP48GGMHTu20fHg4GAUFhZa/TwGgwEHDhxAWlpag+NpaWnIyMho9nGffPIJzpw5Y3Wr0fz586FSqSyXyMhIq2tsb0b3FbusfjyUK3ElRERE7V+LQ46fnx80msbjQjIzM9GlSxern6egoABGoxFqtbrBcbVajby8vCYfc/r0aTz33HP44osv4Opq1ZhpzJ49G6WlpZZLTk6O1TW2N3ckil1WO08XoLjCIHE1RERE7VuLQ86kSZPwz3/+E3l5eZDJZDCZTNi1axeeeeYZTJkypcUFXLn+jiAITa7JYzQaMWnSJLz88svo2bOn1c+vVCrh6+vb4NJRdQv2RlyYL2pNAjYdazoIEhERkajFIee1115DVFQUunTpgvLycvTp0wc33HADUlNT8cILL1j9PEFBQZDL5Y1abbRabaPWHQAoKyvD/v378fjjj8PV1RWurq6YN28e/vjjD7i6umLz5s0t/VU6JPMA5B85y4qIiOiqWhxy3Nzc8MUXX+DUqVP4+uuv8fnnn+PEiRP47LPPIJfLrX4ehUKBpKQkpKenNzienp6O1NTURuf7+vri8OHDOHjwoOUyY8YM9OrVCwcPHsSQIUNa+qt0SKP7iiEn40whCsv1EldDRETUflk3sKUJ3bp1Q7du3dr04rNmzcLkyZORnJyMlJQULF++HNnZ2ZgxYwYAcTzNxYsXsWrVKri4uCAhIaHB40NCQuDu7t7ouDOLCfJCQhdfHLmow8ajebh/SLTUJREREbVLVoWcWbNmWf2EixYtsvrcCRMmoLCwEPPmzYNGo0FCQgI2bNiA6Gjxi1uj0VxzzZzO6I5+4ThyUYcf/9Aw5BARETVDJgiCcK2TbrzxxgY/HzhwAEajEb169QIAnDp1CnK5HElJSe1+bIxOp4NKpUJpaWmHHYScU1SJYQu2wEUG/Pb8SIT4uEtdEhERkV215vvbqpacLVu2WG4vWrQIPj4+WLlyJfz9/QEAxcXFeOCBBzBs2LBWlE0tFRngicRIP/yRU4KNR/IwJSVG6pKIiIjanRYPPF64cCHmz59vCTgA4O/vj1dffRULFy60aXHUvDvNs6z+4CwrIiKiprQ45Oh0OuTnN95WQKvVoqyszCZF0bXdXjfLat/5IuSVVktcDRERUfvT4pAzduxYPPDAA/j2229x4cIFXLhwAd9++y2mT5+OcePG2aNGakK4nweSov0hCMCGw2zNISIiulKLQ87777+P0aNH469//Suio6MRHR2N+++/H6NGjcLSpUvtUSM1w7ww4HqGHCIiokasml3VlIqKCpw5cwaCIKB79+7w8vKydW124Qyzq8zyddW4bv6vEAQg47mbEO7nIXVJREREdtGa7+8Wt+SYeXl5ISAgAEFBQR0m4Dgbta87BsUEAADWc5sHIiKiBlocckwmE+bNmweVSoXo6GhERUXBz88Pr7zyCkwmkz1qpKuw7GXFLisiIqIGWhxy5syZgyVLluCNN95AZmYmfv/9d7z++ut499138a9//cseNdJVjEoIg4sM+COnBDlFlVKXQ0RE1G60eO+qlStXYsWKFbjrrrssxxITE9GlSxc8+uijeO2112xaIF1dsI8S13UNRMaZQqw/rMGM4W3bT4yIiMhZtLglp6ioCL179250vHfv3igqKrJJUdQyo81dVodyJa6EiIio/WhxyElMTMSSJUsaHV+yZAkSExNtUhS1zKiEMMhdZDhyUYdfjzdeqJGIiKgzanF31YIFCzB69Gj88ssvSElJgUwmQ0ZGBnJycrBhwwZ71EjXEOClwAOpMVix8xxmrz2MTU/5w89TIXVZREREkmpxS87w4cNx6tQpjB07FiUlJSgqKsK4ceNw8uRJbtApoWdu7YVuwV7Qlukxd91RqcshIiKSXKsXA+yonGkxwCtlZhfjL8syYBKA9/+ahNsSQqUuiYiIyCZa8/3d4u4qAKiursahQ4eg1WobrY1z+awrcqwBUf54ZHg3LNt6BnO+O4xBMf4I9FZKXRYREZEkWhxyNm7ciClTpqCgoKDRfTKZDEaj0SaFUevMvLkHNh/X4mR+Gf71/RG8N2kgZDKZ1GURERE5XIvH5Dz++OO49957odFoYDKZGlwYcKSndJVj4fhEuLrIsOFwHn7kdg9ERNRJtTjkaLVazJo1C2q12h71kA0kdFHhsRu7AwD+9f0RaMuqJa6IiIjI8Voccu655x5s3brVDqWQLT1+U3f0CfNFSWUNnl97BJ1sfDkREVHLZ1dVVlbi3nvvRXBwMPr27Qs3N7cG9z/xxBM2LdDWnHl21ZVO5Olw57s7UWMUsPDeRPwlKULqkoiIiFqlNd/fLQ45K1aswIwZM+Dh4YHAwMAGg1plMhnOnj3bsqodrDOFHAB4b8ufePPnk/Bxd8Wmp25AmMpD6pKIiIhazCEhJzQ0FE888QSee+45uLi0uLdLcp0t5NQaTfjL+7vxR04JhvcMxqcPDOJsKyIi6nBa8/3d4pRiMBgwYcKEDhlwOiNXuQsW3tsPClcXbDt1Cav35UhdEhERkUO0OKlMnToVq1evtkctZCfdQ3zwbFovAMCr64/jQnGlxBURERHZX4sXAzQajViwYAF+/vln9OvXr9HA40WLFtmsOLKdB6+Pxc9H87D/fDH+8e0hfD59CFxc2G1FRETOq8Uh5/DhwxgwYAAA4MiRIw3u41iP9kvuIsOb9yZi1DvbkXGmEJ/vOY8pKTFSl0VERGQ33KCzk/l01znM/eEYPNzk2DhzGKIDvaQuiYiI6JocMvCYOrYpKTFI6RqIqhojnv3mEEymTpVxiYioE2HI6WRcXGRYcE8/eCnk2JtVhI93nZO6JCIiIrtgyOmEIgM8MWd0HwDAmz+fxJlL5RJXREREZHsMOZ3UxMGRGNYjCPpaE57++g/UGk1Sl0RERGRTDDmdlEwmw7//0g8+SlcczCnBhzvYbUVERM6FIacTC/fzwIt3it1Wb6efwnGNTuKKiIiIbIchp5O7JykCI3uHwGA0YcIHu/Hr8XypSyIiIrIJhpxOTiYTZ1sNjPKDrroW01fux1s/n4SRU8uJiKiDY8ghBHor8dXfUjAtNQYAsGTLn5j68V4UluulLYyIiKgNGHIIAKBwdcHcu+Lxzn394eEmx84/C3DnuzuRmV0sdWlEREStwpBDDdzdvwu+f3wougZ5Ibe0GuM/2I3Pdmehk+3+QUREToAhhxrpqfbB948PxaiEUNQYBfzr+6OY9fUfqDIYpS6NiIjIagw51CQfdzcsvX8g5tweB7mLDN9lXsTYpbtwrqBC6tKIiIisInnIWbp0KWJjY+Hu7o6kpCTs2LGj2XN37tyJoUOHIjAwEB4eHujduzfefvttB1bbuchkMjx8Q1d88dAQBHkrcSKvDHe9uxM/H82TujQiIqJrkjTkrF69GjNnzsScOXOQmZmJYcOGYdSoUcjOzm7yfC8vLzz++OPYvn07jh8/jhdeeAEvvPACli9f7uDKO5frugZi/RPXY1CMP8r0tXjkswOY/9NxbgVBRETtmkyQcETpkCFDMHDgQCxbtsxyLC4uDmPGjMH8+fOteo5x48bBy8sLn332mVXn63Q6qFQqlJaWwtfXt1V1d1Y1RhPe+OkEPtopbgGR0jUQ/5k4AME+SokrIyIiZ9ea72/JWnIMBgMOHDiAtLS0BsfT0tKQkZFh1XNkZmYiIyMDw4cPb/YcvV4PnU7X4EKt4yZ3wb/u6IP3Jg2El0KO3WcLcce7O3DgfJHUpRERETUiWcgpKCiA0WiEWq1ucFytViMv7+pjPiIiIqBUKpGcnIzHHnsMDz30ULPnzp8/HyqVynKJjIy0Sf2d2eh+Yfj+8aHoHuKNfJ0eEz74DZ/sOsdp5kRE1K5IPvBYJpM1+FkQhEbHrrRjxw7s378f77//PhYvXowvv/yy2XNnz56N0tJSyyUnJ8cmdXd23UN88P1jQ3FHvzDUmgS8/MMxPP3NH6iu4TRzIiJqH1yleuGgoCDI5fJGrTZarbZR686VYmNjAQB9+/ZFfn4+5s6di4kTJzZ5rlKphFLJMSP24KV0xbsTB2BAlD9e33Aca3+/iDOXKvDBX5MQqnKXujwiIurkJGvJUSgUSEpKQnp6eoPj6enpSE1Ntfp5BEGAXs89lqQik8kw/fpYrHpwMPw83fBHTgnuWrITv3M7CCIikpik3VWzZs3CihUr8PHHH+P48eN46qmnkJ2djRkzZgAQu5qmTJliOf+9997DDz/8gNOnT+P06dP45JNP8NZbb+Gvf/2rVL8C1RnaPQjrHrsevdQ+0Jbpcd8Hv+Hr/ewaJCIi6UjWXQUAEyZMQGFhIebNmweNRoOEhARs2LAB0dHRAACNRtNgzRyTyYTZs2fj3LlzcHV1Rbdu3fDGG2/gkUcekepXoMtEBXpi7aOpmPX1Qfx8NB//+PYQjuXqMGd0HNzkkg//IiKiTkbSdXKkwHVy7M9kEvDu5j/x9i+nAACp3QLx3qSB8PdSSFwZERF1VB1qnRxyXi4uMjx5cw98MDkJXgo5Ms4U4q73duJEHtcoIiIix2HIIbu5NT4Uax8diqgAT+QUVWHc0gz8dFgjdVlERNRJMOSQXfUK9cG6x4fi+u5BqDQY8X9f/I5Fm07CZOpUvaRERCQBhhyyOz9PBT59YBCmXy+ub/SfzX/ib58dQFl1jcSVERGRM2PIIYdwrdv3auG9iVC4uuCX4/kYtzQDWQUVUpdGREROiiGHHOovSRH4+pEUqH2VOK0tx11LdmL7qUtSl0VERE6IIYccrn+kH354/HoMiPKDrroW0z7Zi5d/OMpWHSIisimuk0OS0dca8a//HcHX+y8AAGQyYHjPYExNicHwnsFwcbn6Rq1ERNR5tOb7myGHJCUIAraeuoSVGVnYerK+2yo60BOTr4vGvUmRUHm6SVghERG1Bww5VmDIab+yCirw2W/n8c3+HOiqawEA7m4uGNO/CyanRCM+XCVxhUREJBWGHCsw5LR/lYZafH8wFyszsnAir8xyPDnaH1NSY3BbfCgUrhxORkTUmTDkWIEhp+MQBAH7soqxancWNh7JQ23dAoLBPkpMGhyFSUOioPZ1l7hKIiJyBIYcKzDkdEz5umr8d082/rs3G5fK9AAAVxcZbk0IxZTrojE4NgAyGQcqExE5K4YcKzDkdGyGWhM2Hs3DZ7uzsC+r2HK8e4g37hsUib8MjOBu50REToghxwoMOc7jaG4pPtt9Ht8fzEVVjREAoJC7YFTfUNw3KArXdWXrDhGRs2DIsQJDjvMpq67B9wdz8eXebBzN1VmOdw3ywn2DxdadQG+lhBUSEVFbMeRYgSHHuR2+UIr/7s3GuoMXUWEQW3fc5DKkxYdi0uAopHQN5CKDREQdEEOOFRhyOocKfS1++ENs3fnjQqnleFSAJ+4bHIl7kiIQ4sOZWUREHQVDjhUYcjqfo7ml+GpvDv6XeRFlenGRQVcXGW6OU2PikCgM6x7E1h0ionaOIccKDDmdV6WhFusPafDl3mz8nl1iOR4VIG4hMT6ZW0gQEbVXDDlWYMghADiZV4Yv92Zj7e8XGmwhMXZAF0xJiUFcGD8bRETtCUOOFRhy6HJVBiO+P3gRK3efx3FN/cyswTEBmJoag7R4Ndzk3EKCiEhqDDlWYMihpgiCgP3ni7Eyo+EWEmpfJe4fEo37BkdyoDIRkYQYcqzAkEPXkldajf/uzcZ/92SjoFzcQsJNLsPovmGYkhqDAZF+XGSQiMjBGHKswJBD1jLUmvDTEQ1WZmQ1GKjct4sKU1NjcEe/MLi7yaUrkIioE2HIsQJDDrXG4QulWLk7C+v+yIWh1gQA8Pd0w18GRqCn2gfhfh4I83NHuMoDHgoGHyIiW2PIsQJDDrVFYbkeq/fn4PPd55FbWt3kOf6ebghTeSDcz73u+vLb7lD7unMwMxFRCzHkWIEhh2yh1mjCL8e12HZKi4sl1cgtqYKmpMqylcTVyGRAiI8SYSoPxIX54oYeQUjtFsQ1eoiIroIhxwoMOWQvgiBAV10rBp7SKuSWVFuuxWPizzXGxv/LuciAvhF+uKFHEK7vHoQBUf5QuLK1h4jIjCHHCgw5JCWTSUBBhR6akmpcKK7C/vNF2HG6AH9qyxuc56mQI6VrIK7vEYRhPYLQLdibM7qIqFNjyLECQw61R5rSKuw8XYAdpwuw688CFFYYGtwfpnLH9d2DMKxnMIZ2C0Sgt1KiSomIpMGQYwWGHGrvTCYBx/N02HG6ADtPF2BvVpFlRpdZfLgvhnYPQlSAJ9S+7gj1dYdapUSglxJybjZKRE6IIccKDDnU0VTXGLH3XBF2/im29Fy+/cSV5C4yhPgoofZ1h9pXWRd+6kJQ3SVU5Q5vpasDfwMiorZjyLECQw51dJfK9Nj1ZwEOnC9Gnq4a+bpq5JVWo6BcD5OV/zd7KeSID1fhL0ldMLpfOEMPEbV7DDlWYMghZ1VrNKGg3IC8utCjLROvzUEoX6dHfmk1yvS1DR7n4SbH6H5huDcpAoNjAzjAmYjaJYYcKzDkUGdXoa+FprQK6ce0+GZ/Ds4WVFjuiwn0xL3JkRg3sAvCVB4SVklE1BBDjhUYcojqCYKA37OL8fW+C/jxUK5lMUMXGTCsRzDGJ0fi5j4hULpyqwoikhZDjhUYcoiaVqGvxU9H8vD1/hzsPVdkOe7n6YYx/bvg3uQIxIerJKyQiDozhhwrMOQQXVtWQQW+PXAB3x64gDxd/R5dfcJ8MT45Anf37wJ/L4WEFRJRZ8OQYwWGHCLrGU0Cdpy+hG8OXED60XwYjOJ6PXIXGVQebvBwk8NLKYeHwhWeV9z2VMrhqZDDU+Fad11/O9zPA71DfTjImYis1prvb84bJaJmyV1kGNErBCN6haC4woDvD17ENwcu4GiuDkVXrMrcUl38PHBLHzXS4tUYHBMAV+7MTkQ2JnlLztKlS/Hmm29Co9EgPj4eixcvxrBhw5o8d+3atVi2bBkOHjwIvV6P+Ph4zJ07F7feeqvVr8eWHKK2yyuthq66BhX6WlQZjKg0GFFhqL9daaitu2769qn8MlTX1K/i7Ofphpt6hyCtTyhu6BkETwX//iKihjpcd9Xq1asxefJkLF26FEOHDsUHH3yAFStW4NixY4iKimp0/syZMxEeHo4bb7wRfn5++OSTT/DWW29hz549GDBggFWvyZBDJL0qgxE7Tl/CpmP5+PV4Poorayz3KV1dMKxHMNLi1RjZO4T7dBERgA4YcoYMGYKBAwdi2bJllmNxcXEYM2YM5s+fb9VzxMfHY8KECXjxxRetOp8hh6h9qTWasP98MTYdzcemY3m4UFxluc9FBiTHBCCtjxppfUIRFegpYaVEJKUONSbHYDDgwIEDeO655xocT0tLQ0ZGhlXPYTKZUFZWhoCAgGbP0ev10Ov1lp91uub3/SEix3OVu+C6roG4rmsg/nVHHE7klVkCz9FcHfaeK8Lec0V4df1x9A71QVp8KG6JUyM+3BcuDt6MtKBcD193NyhcOX6IqCOQLOQUFBTAaDRCrVY3OK5Wq5GXl2fVcyxcuBAVFRUYP358s+fMnz8fL7/8cptqJSLHkMlkiAvzRVyYL568uQcuFFci/Vg+Nh3Nx96sIpzIK8OJvDL859fTUPsqcVPvEIzsrcbQ7kHwUNh+wcLqGiN2ny3EtpOXsO3UJZwrqICPuytujlPjtoRQDO8ZDHc3LpRI1F5J1l2Vm5uLLl26ICMjAykpKZbjr732Gj777DOcOHHiqo//8ssv8dBDD+H777/HzTff3Ox5TbXkREZGsruKqIMprjBg8wktNh3Lw47TBaisW50ZEMfxDO0ehJFxYugJVbm36jUEQcDZggpsrQs1e84WQl9ravZ8Dzc5buwdjNsSwnBjr2D4uLu16nWJ6No6VHdVUFAQ5HJ5o1YbrVbbqHXnSqtXr8b06dPxzTffXDXgAIBSqYRSyYGLRB2dv5cCf0mKwF+SIqCvNeK3s0X49Xg+fj2uxcWSKmw+ocXmE1rMwRHEh/tiZJwaN8eFICFcddVurQp9LTLOFGLbKS22nrzUYEwQAISr3DG8VwiG9wxGSrdAnMovw8Yjedh4JA8XS6qw4XAeNhzOg0LugmE9gnBbQihujlNzsUSidkDygcdJSUlYunSp5VifPn1w9913Nzvw+Msvv8SDDz6IL7/8EmPGjGnxa3LgMZFzEQQBJ/PL8OtxLX49no/MnBJc/q9aiE9dt1acGtd3D4K7mwtO5Zdj60kttp26hH1ZRagx1j9AIXfB4NgADO8ZjBG9gtE9xLvJRQsFQcCRizr8dESDjUfyGmx0KneRIaVrIG5NCMWt8WqE+LSuZYmI6nW42VXmKeTvv/8+UlJSsHz5cnz44Yc4evQooqOjMXv2bFy8eBGrVq0CIAacKVOm4J133sG4ceMsz+Ph4QGVyro9dewWcgQBOLMZKMkGkh+w3fMSUYsUlOux5YQWvx7XYsfpS5ZNRwGxW0vl4QZtmb7BY6ICPDGiV7Cltaal6/QIgoDT2nL8dDgPG4/m4bimfoKDTAYkR/vj1vhQ3BofisgAzhAjao0OF3IAcTHABQsWQKPRICEhAW+//TZuuOEGAMC0adOQlZWFrVu3AgBGjBiBbdu2NXqOqVOn4tNPP7Xq9ewWcrJ2AZ/eDrh5AU8fB9y5kSGR1PS1Ruyp69b6pa5bCxDDTkq3QIzoGYzhvUIQE+hp0y0msgoq8PPRPPx0JA8Hc0oa3BcT6InU7kFI7RaIlK6BXAeIyEodMuQ4ml1bcpZeB1w6Adw6H0h51HbPTURtJggCTuWXo7jSgP6Rfg6bFZVbUoVNdYFnX1YRTFf8i9s71AdD60LP4NgADl4magZDjhXsOiZn/8fAj08B/jHA338HXDi1lIjq6aprsPdsETLOFCLjTAFO5JU1uF/uIkO/CBVSuwViaLcgDIz25xR1ojoMOVawa8gxVACL4oDqUmDiaqDXbbZ9fiJyKgXlevx2thC7/izE7jMFyCqsbHC/wtUFSVH+GNo9ECndgtAvQgU3bmRKnRRDjhXsPrtq0wtAxrtA1xuBKf+z/fMTkdO6UFyJ3WcKkXGmELv+LGg0QNpTIceAKD8MignAoJgADIjy42am1Gkw5FjB7iGn+Dzwn/6AYAIe3QOE9Lb9axCR0xMEAWcuVWD3mQKxpedsIUqrahqc4+oiQ3wXFQZF+2NQrBh8Arg+DzkphhwrOGSdnK/uB078CCRPB+5YZJ/XIKJOxWQSp6nvzSrCvnNF2JdVBE1pdaPzugV7YXBd4BkUE4AIf4+rzhwzmQQUVxpQUG7ApTI9LpVXo6DMgEvlelwq06Og7tpVLkOorztCfN0RWndRq9yh9lUi1NcdKg+3Vs9QM5oEFFboodXpLa97+cVTIceQroG4rmsAIvw5Bb+zYsixgkNCzrntwMo7ATdPYNZxwMPPPq9DRJ3aheJK7Msqwr6sYuw7V4TT2vJG54T6umNQbADiwnxQVl2LgjJ9gwBTUG6A8copX62gdHWB+rLwE+qrhNrXHWpfdwR4KVBcabCEFu1lAUZbpkdRhb7RrLPmRPh7WDZ0ZejpXBhyrOCQkCMIwLJUQHsMSHsNSH3cPq9DRHSZogoDDpwvxr4scef2IxdLUWtlegjwUiDIW4FgHyWCvZUI8laKt33E2zVGE/J1euTpqpFfWi1e112KK2uu/QLXIJMBgV7i64X41L92sLcSl+oGaB+6UNookEX4e2BIrBh4rusa2KrFFksqDcgqrMT5wgpkF1Yiq7AS2UUVOF9YCQHAgEg/JEX7IznGHwldVFC6csabFBhyrOCwbR0OfAr88CTgFw08kcnp5ETkcFUGIzJzirE/qxhnLpXD37M+xFweYAK9FW2atVVdY8SlMjEA5ZXWh588nR75pdUoqjTA39OtLsC419fgK16H+CgR4KWA6zVqKNfX4sD5Yvx2trDZ0NPFz8PSymMOPSaTAG2ZHucLxeByvi7AnK8LNrrqWqt/V4XcBX0jVEiO9kdS3YULOjoGQ44VHBZyDJV108lLgPu+BHrfbr/XIiLqhCr0tdhfF3r21IWeK1uugryVKKuuuepu8gCg9lUiOsALUYGeiAn0RFSgF6IDPFFjNOHA+WLLpbDC0OixsUFelsCTHO2PbsHeV90UllqHIccKDt2gM/1FYNc7QOxwYOo6+74WEVEnV9FES4859MhdZIjw90BUgCeiAz0RE+hVd1u89lBcu7VdEARkFVbWBZ4i7M8qbnIclMrDDQOj/JAcE4A+4b7opfZBmMrdpluHdEYMOVZwaMgpyQbeSaybTv4bEBJn39cjIiKLCn0tTuaXIdBLgXA/D7sspFhaWYPfs8VWnv3ni3AwpwTVNY1bjXyUruih9kavUB/0CPFBr1Af9FT7IMhb0W7CT0mlAXvOFeG3s4XIK63G4NgA3NQ7BNGBXlKXBoAhxyoODTnAZdPJHwTueNv+r0dERJKpMZpwXKPD/qxi/J5djFP5ZTh7qaLZAeABXgr0CPG2hB7x4g0/T/uvd1RaWYO9WUXYfUZs+Tqep0NTiaBrsBdu6hWCm3qHIDkmAApXaVbdZsixgsNDzrkdwMo76qaTHwM8/O3/mkRE1G4Yak04V1CBU/llOJVfhpN54vX5osomQwUgjhHqHuKNSH9PRPh7oIu/ByLqbof4uEPeijE/5r3TfjtbiN/OFeJobuNQ0y3YC9d1DUS4nwd2nL6E/VnFDQKat9IV13cPwk29QzCidzBCfNxbXEdrMeRYweEhRxCAZUMB7VEg7VUg9e/2f00iImr3qgxGnLlULoYebRlO5ZXhVH45LpZUXfVxbnIZwv08EOHvgQg/MfhEBFwWggwXIV/7IPRd07Ar8uG6lpoiHM0tbbQeUde6UGOekXZlaNFV12Dn6QJsPqHF1pNaFJQ3HHjdt4sKN/YOwY29gpEY4WfXAdcMOVZweMgBgAMrgR+eAPyigCcOcjo5ERE1q6y6Bqe15TijFQPPxeIqXCiuwoWSSuSWVF918UZfVOA75UvoJssFANyjfxH7hfrthWKDvCzT66/rGgi1r/UtMSaTgMMXSy2B548LpQ3uD/RSYHivYNzUOwTDegRD5eHWwt/86hhyrCBJyDFUAm/3AaqKgfv+C/Qe7ZjXJSIip1JrNCG/TI8LRZVi8CmuwoVi8bamuAzzKubhBpdDlvPPySLxYdxKDO4RiiFdAxCm8rBZLZfK9Nh6UostJ7XYcaoAZfr69YaUri7IfPEWm24g25rvb25f6wgKT2DgVGDXYmDP+ww5RETUKq5yF3Tx80AXPw8MufLOjc8Dvx2CydUDp2/6EN13PInYqhy8HroFGPCMzWsJ9lHi3uRI3JsciRqjCfuyirDlhBZbTl5CgJfCpgGntdiS4yglOcA7/cTp5P+3G1D3cdxrExGRc/v9M2Bd3RZC934KxI8F/lgNfPc3QK4EHt0NBHZzWDmVhlqbh5zWfH9LMw+sM/KLBHrfId7e+4G0tRARkfM4vxv48Snx9vDnxIADAP3GA11HAEY9sH4Wmp3KZQftoRUHYMhxrCEzxOs/VgOVRdLWQkREHV9JNrD6r4CpBoi7Cxj+z/r7ZDJg9CKxJefsVuDwN5KVKRWGHEeKTgXUfYHaKiDzM6mrISKijkxfDnw5CagsAEL7AmPfB1yu+FoP7AYMf1a8vXF2p/sDmyHHkWQyYMgj4u29HwJG63e+JSIisjCZgO8eAfIPA17B4kbQima2X0h9EgjuLYah9BcdW6fEGHIcre89gEcAUJoDnPpJ6mqIiKgj2jpf3DJIrgAmfCGO+2yOqwK4Y7F4O/MzIGuXQ0psDxhyHM3NA0iaJt7ewwHIRETUQkfWAtsXiLfvWAxENZpM3lh0iriUCQD8OBOo1durunaFIUcKg6YDMjmQtQPIOyJ1NURE1FHkZgL/e1S8nfI4MOB+6x97y8ti11bBKWDXO/apr51hyJGCKgKIu1O8zenkRERkjbI8caBxbRXQ/Rbglnkte7yHP3DbG+Lt7W8BBX/avsZ2hiFHKubp5Ie+7nSj3YmIqIVqqoGv7gfKcoGgnsA9H7VuH8SEvwDdbqpbO+cph66dIwWGHKlEXQeE9gNqq4HfV0pdDRERtVeCAPzwJHBxP+DuB0z8CnBXte65zGvnuLoD57YDf3xl01LbG4Ycqchk9a05e1dwOjkRETVt1zvAoa/EsZzjV7Z9e4aA2PpFA39+HqgobHuN7RRDjpQS/gJ4BgK6C8DJ9VJXQ0RE7c3JjcAvc8Xbt70hbtNgC6l/B0L6AFVFTr12DkOOlNzcgaQHxNucTk5ERJfTHgfWPARAEL8rBj9su+eWuwF31s2wOvg5cG6H7Z67HWHIkZp5Ovn5XYDmkNTVEBFRe1BZBHx5H2AoA6KvB25/UxzmYEuRg4HkB8XbTrp2DkOO1HzDgT53i7c5nZyIqHMTBLGL6uNbgeIswC8aGL9KbHmxh5EvAd5qoPBPYOfbbX8+fTlw+Fvg91Vtfy4bYMhpDyzTyb9x6gFgRER0FRf2A5+OBr6cIC7Y5xkozqTyCrTfa3r41a+ds2MhUHC65c9RUwUcWwd8PRV4szuwZjqw+VXAZLRpqa3hKnUBBLHJMKw/oDkIrLwDGDoTSBhnv+RORETtR+EZ4NeXgWPfiz+7uot//F7/lBhC7C1+LHDwv8Cf6cCPTwFTf7h211itATizGTi6FjixHjCU19/nHytOrKmtbn7TUAeRCYKTrwR0BZ1OB5VKhdLSUvj6+kpdTr1z24EvJ9Z/UHwjgJTHgIFTAKW3tLUREZHtlWuBbf8GDnwKmGoByID+k4AbnxdXxnek4izgvevE1ZTvXtr0dhHGWiBru7h31vEfgOqS+vtUkUD8GDHchPW3/fghtO77myGnPakqBvZ/DPz2PlChFY+5+4mDk4fMALxDJC2PnEhxltiU3Nb1Noio5fTlwO4lQMa79X/Y9kgDbp4LqOOlq2vXO+J0co8A4PH9YjeZyQRk7waOrBFbmioL6s/3VoutQPHjgIhBgIt9R8Aw5FihXYccs5pqceGnjHfFwWAAIFcC/ScCKX8HgrpLWx91bCd/Ar6eAhgN4j+sqX8HYobZ5S8vIrqMsUZc4X7rv+v/kA0fKO5BFTtM2toAsb7lI4D8I0Cv0YB/NHD0O6BMU3+OZyAQd5fYYhOd2rqtJVqJIccKHSLkmJmMwMkNYrq+sK/uoAyIuwNIfRKIHCRpedQBHVkDrP1bXdP4ZcISgdQngD5jADmH6hHZlCCI3Tu/vlz/h6t/LDDyRbElpD39gXFhP7DiZgCXRQOlStxUOmEsEDtcsvGiDDlW6FAhx0wQgOzfxLBz6qf641GpwNAnxb/G7dxMSE7g91XAuicACEC/CcCwZ8RlCzK/EPvhAbFf/bpHgYGTAaWPpOWSDV3YD5z4EQjoJk50COzBfzMc5XyG2AVk/kPVM0jcUiFpGuCqkLS0Zm15Hdj3EdDtxvoNPV2VUlfFkGONDhlyLqc9Aex+F/hjNWCqEY8F9xa7HPre2y4+iNQO/bYM2PiceDvpAXGDPvOXXEUhsP8jcdVtc3+7UgUkPyCOBfMNk6ZmajuTEdi5CNgyHxAum87r7gdEJAMRg8UW4S7JgHsH/PewPTLWAtpjYqg5tRE4vUk87uYJpDwu/lvN97pVOmTIWbp0Kd58801oNBrEx8dj8eLFGDas6b5JjUaDp59+GgcOHMDp06fxxBNPYPHixS16vQ4fcsx0ucCe94H9nwB6nXjMK1j8Cz1xIhCaIG191H5sfwvY/Ip4O+VxIO3VppvHa6qAQ6uBjCVAYd1aGS5uQL/x4uPUfRxXM7WdTgN89zdx5iYAdL8FMFQAuZn1LXcWMiAkThw8GjlYDD+B3dnaY42yPLGl7MI+8Tr3d6Cmsv5+mVycJTviOcAnVLo6nUCHCzmrV6/G5MmTsXTpUgwdOhQffPABVqxYgWPHjiEqKqrR+VlZWXj77beRlJSEt99+G8OHD++8IcesulScfvjbsoaDw0L7Av3vF1t3vIIkK48kJAjiGADzKqYjngeG/+Pa/f8mk/gXaMa7QHZG/fHuN4t/hcYOb19jCKix0+nAdzPEljk3L2D0QnHiAiAOLs07LH4p5+wVr0vON34Odz8x9EQMElt7IodIvuaJ5GqqgbxDdYGmLtSU5jQ+T+kLdEkSW8v6jgeCezq+VifU4ULOkCFDMHDgQCxbtsxyLC4uDmPGjMH8+fOv+tgRI0agf//+DDlmxhrxH7Y//isuCW7uynJxFcfs9J8E9Li1/fYBt1ZZPnDphDiV0TeczcBmJhOw8Z/A3uXiz2mvigGlpS7sF8PO8XWAYBKPhfatH6TsbJ+njq7WAGyeJ/43AwB1X+DeT4CgHld/XFl+3Zf2XiBnn9gaUVvd8ByFt7hI6cCp4he4swZdk1H847G6BKgqAYrO1ocazaH6f1vNZC7ibt4RyWK3X8QgIKgnW8HsoEOFHIPBAE9PT3zzzTcYO3as5fiTTz6JgwcPYtu2bVd9vLUhR6/XQ6+v33RMp9MhMjLS+ULO5SqLxFk0B78Qm6bNPAKAvveIgcdOizU5hCCI6zbsXS7OWLh8ppDSVww7vl3Ea1XEZT93AVRdbDeg1lgLQGh/K1Mba4EfnhD/+0MG3LGofhO+1io6J7YWZn5W3xTvGQj0u08cpBwS1+ayqY2KzgHfPigGFAAY/Ig4NdnNveXPdWVrT/ZvgO5C/f0h8WIXTL/xgGeAbeq3pVpDfUixXJc2PHb58cvvN3f/N8cruK6Fqy7QhA/gIH0H6VAhJzc3F126dMGuXbuQmppqOf76669j5cqVOHny5FUfb23ImTt3Ll5++eVGx5065FxOe1xcrvvQ10B5Xv3x4Dix+brfhI7TT2yoEH+PvR8C2qP1x/1j6v/hsobStz4E+YaLIaVWX3epbnxtNDRxXC8O5JTJxenXMUPFnYKjrnPMMuzNqTUAax8Gjv1PrG3MMiBxgu2ev7JIHKS8d0XDz1PEIGDAZPEvff6D73hH1gDrnhR3rHb3A8YsBXqPtt3zC4I4S+j3VeJny9zKI1cCfe4SA0/09bZvvRAEsRu+JKeJ0HKV68vHxLSWwlt8L33D6rqe6oKNX3TH/QOxg+uQIScjIwMpKSmW46+99ho+++wznDhx4qqPZ0tOCxlrgbNbxb/uT6wHjHXvicwF6DYSSLwPCO0nrqrsrmpf/xMXngH2rRCnOutLxWNunuJ4o8EPi90ngLiKaJkGKL0A6C6Kg7NLL4jXuovipbrUvrXKXMR6oq8Xg09UiuP+0q2pEjfIO/0zIFcA93wsrm1hD8Za4M9fxJadkz/Vz9xx8xLX/Rg4WRzD0Z4+R87IUCl2S5p3fI5KAcZ9CPhF2u81q4rFXaYPrATyD9cf948Vw07/Sa3/w6ksT2x9zs0Ecg+K1+ZF81pMJnZfu/uJf3i4qy673dQx/8uOqdpfCy11rJDjqO6qKzntmJyWqCoRV7E8+F+xD/5KcqUYdrxDAK+Q+tve6sbHFN72+SIzGcWpl3uXi5vAmQV0BQY9JP5D6uHf8ufVl9eFHnP4yRXHmrgqxd/bVSlujme5Nt9u6j4loC8Dzu8GsnYA53eJ/fcNyMRl2mOuB6KHihd77CisLxP3PsvaAbh6APd9Lg4UdoSyfOCPL8XAY17oDBDHJQyYLM728w52TC2dSf5R4JsHgIKTAGTADc8Aw59z3GKOgiBuKnxgpRh6DGXicZkc6DVKDDzdb25+RdxybcMwk5vZsHXQTCYXu509/K8IKNe4Vvo6dDVesr8OFXIAceBxUlISli5dajnWp08f3H333Rx47CgFf4pfUCfWi1/4+ha2dLh5in3UvuFiM65/dN11jHjbJ6xl/9BUFAKZq8Q9vEqy6w7KgJ63AoMeFhelas8D+nQaMexk7RSvC041Pic4rq57a6jYBK6KaFtQrCoGvrhXHD+h8AEmrRaf39HMi1ZmfiaGaHOXgYsr0PM28Uuv20iuqNxWggAc+ATYOFvsNvJWi603XYdLV5OhQvxv/vsqIGdP/XGfcGDAX8WNG3UaMchoDorXuouNn0fmIq77FdZfHOsSPkBcDsPNw0G/CLVnHS7kmKeQv//++0hJScHy5cvx4Ycf4ujRo4iOjsbs2bNx8eJFrFq1yvKYgwcPAgAeeugh9OrVC88++ywUCgX69LFuDQ+GnGuoqQIqLol/ZZXn111rxSbjy38u1wI1Fdd+PrlCXEXXvy74mIOQ+baHv/gFf/F3cazNkTX1XWke/mJLwKDp4vkdUVm+GHbO7wKydgGXjjc+R+krDtwNiRMHdIbEibM1rGnxKb8EfD5WHCTq4Q/8dY04fkBq1Trg6Frg98+Ai/vrj/uEia1wcXeJnwvPAMd0aQmCOJ6oLFccT+UVLLZEdrQvz6oSYN3fxdlugLj2zZhl7aulTHtCDDt/fAlUFV3lRJnY2mcOM+H9xa7ezj5NnZrV4UIOIC4GuGDBAmg0GiQkJODtt9/GDTfcAACYNm0asrKysHXrVsv5sib+QYyOjkZWVpZVr8eQY0P68rrwc0ns/ik+L663UZwl3i7NabxH0pWUvuKX8+XrdIT1Bwb/TRzE2tG+hK6lokAcwGkOPtrjzb9H3ur6wGO+BPcClN7i/bpcYNXdYmuRVwgw5X/S7mDcnPxjQObnTX/pyZXi+A2fMHGAp0/dxTe8/rhPGKDwbP75aw3iWKwyjfieNLjWiMFGp6kPz5dTqhp2xfqENvzZWy1ePAOl6/qoqRYH02qPiYOLS7PF1rGb5wLXPdZ+WzZr9WIL8e+rxG5U/9j6MBM+QAw0HKROLdAhQ46jMeQ4kLFW/IIprgs+Jecb3i7Prz9XrhAHrA7+m3OvwXGlWoM4jkV7rO5yXLwuzmr+Mf4xYuDJPyJ26flGAFO+b/+709fqxQ1nMz8XuysqC61/rLtK7Powh5CqkvrwYt6KwhqeQWJwLtc2HXqaI3Opb/1RqsTQ5eYhDrRWeIrdtgqvuuu6nxsd8xKDUnWJ2MVYddn1lccu//nK1Yn9Y4C/fAxEtIMWO2sJQuf5f5rshiHHCgw57YihUvySLssVFy1rT03uUtOXA5dOXhZ8jorXlwdDQPzreOo6wK/xCuHtXq1enE1jaYWpa3Upy2vYAtNoC4ImyBViAPLtcllLkLl1KLy+lci8t5sgiOuhlOXXdcPmX9FFe9l1xSU02JFZCjIXMej1uh24bb54m6iTYcixAkMOdWgVhfXBp6ZC3LrDO0TqquxHEMRp/2V59aGnQlu3fkl4faDxDLRfS4GxVmwtMgcffZk4qNpQKf43MFTW/VwhXlvuu+yY+VyTsW72T91MIctsIPPP/k3/rPRtv91SRA7CkGMFhhwiIqKOpzXf3/zTgIiIiJwSQw4RERE5JYYcIiIickoMOUREROSUGHKIiIjIKTHkEBERkVNiyCEiIiKnxJBDRERETokhh4iIiJwSQw4RERE5JYYcIiIickoMOUREROSUGHKIiIjIKTHkEBERkVNylboARxMEAYC4ZTsRERF1DObvbfP3uDU6XcgpKysDAERGRkpcCREREbVUWVkZVCqVVefKhJZEIidgMpmQm5sLHx8fyGQymz63TqdDZGQkcnJy4Ovra9PndmZ831qO71nr8H1rHb5vrcP3reWu9p4JgoCysjKEh4fDxcW60TadriXHxcUFERERdn0NX19ffqBbge9by/E9ax2+b63D9611+L61XHPvmbUtOGYceExEREROiSGHiIiInBJDjg0plUq89NJLUCqVUpfSofB9azm+Z63D9611+L61Dt+3lrP1e9bpBh4TERFR58CWHCIiInJKDDlERETklBhyiIiIyCkx5BAREZFTYsixkaVLlyI2Nhbu7u5ISkrCjh07pC6pXZs7dy5kMlmDS2hoqNRltTvbt2/HnXfeifDwcMhkMvzvf/9rcL8gCJg7dy7Cw8Ph4eGBESNG4OjRo9IU245c632bNm1ao8/fddddJ02x7cT8+fMxaNAg+Pj4ICQkBGPGjMHJkycbnMPPW2PWvG/8vDW2bNky9OvXz7LoX0pKCn766SfL/bb6rDHk2MDq1asxc+ZMzJkzB5mZmRg2bBhGjRqF7OxsqUtr1+Lj46HRaCyXw4cPS11Su1NRUYHExEQsWbKkyfsXLFiARYsWYcmSJdi3bx9CQ0Nxyy23WPZo66yu9b4BwG233dbg87dhwwYHVtj+bNu2DY899hh+++03pKeno7a2FmlpaaioqLCcw89bY9a8bwA/b1eKiIjAG2+8gf3792P//v246aabcPfdd1uCjM0+awK12eDBg4UZM2Y0ONa7d2/hueeek6ii9u+ll14SEhMTpS6jQwEgfPfdd5afTSaTEBoaKrzxxhuWY9XV1YJKpRLef/99CSpsn6583wRBEKZOnSrcfffdktTTUWi1WgGAsG3bNkEQ+Hmz1pXvmyDw82Ytf39/YcWKFTb9rLElp40MBgMOHDiAtLS0BsfT0tKQkZEhUVUdw+nTpxEeHo7Y2Fjcd999OHv2rNQldSjnzp1DXl5eg8+eUqnE8OHD+dmzwtatWxESEoKePXvi4YcfhlarlbqkdqW0tBQAEBAQAICfN2td+b6Z8fPWPKPRiK+++goVFRVISUmx6WeNIaeNCgoKYDQaoVarGxxXq9XIy8uTqKr2b8iQIVi1ahV+/vlnfPjhh8jLy0NqaioKCwulLq3DMH+++NlruVGjRuGLL77A5s2bsXDhQuzbtw833XQT9Hq91KW1C4IgYNasWbj++uuRkJAAgJ83azT1vgH8vDXn8OHD8Pb2hlKpxIwZM/Ddd9+hT58+Nv2sdbpdyO1FJpM1+FkQhEbHqN6oUaMst/v27YuUlBR069YNK1euxKxZsySsrOPhZ6/lJkyYYLmdkJCA5ORkREdHY/369Rg3bpyElbUPjz/+OA4dOoSdO3c2uo+ft+Y1977x89a0Xr164eDBgygpKcGaNWswdepUbNu2zXK/LT5rbMlpo6CgIMjl8kbpUqvVNkqh1DwvLy/07dsXp0+flrqUDsM8G42fvbYLCwtDdHQ0P38A/v73v2PdunXYsmULIiIiLMf5ebu65t63pvDzJlIoFOjevTuSk5Mxf/58JCYm4p133rHpZ40hp40UCgWSkpKQnp7e4Hh6ejpSU1Mlqqrj0ev1OH78OMLCwqQupcOIjY1FaGhog8+ewWDAtm3b+NlrocLCQuTk5HTqz58gCHj88cexdu1abN68GbGxsQ3u5+etadd635rCz1vTBEGAXq+37WfNRoOiO7WvvvpKcHNzEz766CPh2LFjwsyZMwUvLy8hKytL6tLaraefflrYunWrcPbsWeG3334T7rjjDsHHx4fv2RXKysqEzMxMITMzUwAgLFq0SMjMzBTOnz8vCIIgvPHGG4JKpRLWrl0rHD58WJg4caIQFhYm6HQ6iSuX1tXet7KyMuHpp58WMjIyhHPnzglbtmwRUlJShC5dunTq9+3//u//BJVKJWzdulXQaDSWS2VlpeUcft4au9b7xs9b02bPni1s375dOHfunHDo0CHh+eefF1xcXIRNmzYJgmC7zxpDjo289957QnR0tKBQKISBAwc2mD5IjU2YMEEICwsT3NzchPDwcGHcuHHC0aNHpS6r3dmyZYsAoNFl6tSpgiCI03pfeuklITQ0VFAqlcINN9wgHD58WNqi24GrvW+VlZVCWlqaEBwcLLi5uQlRUVHC1KlThezsbKnLllRT7xcA4ZNPPrGcw89bY9d63/h5a9qDDz5o+c4MDg4WRo4caQk4gmC7z5pMEAShlS1LRERERO0Wx+QQERGRU2LIISIiIqfEkENEREROiSGHiIiInBJDDhERETklhhwiIiJySgw5RERE5JQYcoiIiMgpMeQQUae3detWyGQylJSUSF0KEdkQQw4RERE5JYYcIiIickoMOUQkOUEQsGDBAnTt2hUeHh5ITEzEt99+C6C+K2n9+vVITEyEu7s7hgwZgsOHDzd4jjVr1iA+Ph5KpRIxMTFYuHBhg/v1ej3+8Y9/IDIyEkqlEj169MBHH33U4JwDBw4gOTkZnp6eSE1NxcmTJ+37ixORXTHkEJHkXnjhBXzyySdYtmwZjh49iqeeegp//etfsW3bNss5zz77LN566y3s27cPISEhuOuuu1BTUwNADCfjx4/Hfffdh8OHD2Pu3Ln417/+hU8//dTy+ClTpuCrr77Cf/7zHxw/fhzvv/8+vL29G9QxZ84cLFy4EPv374erqysefPBBh/z+RGQf3IWciCRVUVGBoKAgbN68GSkpKZbjDz30ECorK/G3v/0NN954I7766itMmDABAFBUVISIiAh8+umnGD9+PO6//35cunQJmzZtsjz+H//4B9avX4+jR4/i1KlT6NWrF9LT03HzzTc3qmHr1q248cYb8csvv2DkyJEAgA0bNmD06NGoqqqCu7u7nd8FIrIHtuQQkaSOHTuG6upq3HLLLfD29rZcVq1ahTNnzljOuzwABQQEoFevXjh+/DgA4Pjx4xg6dGiD5x06dChOnz4No9GIgwcPQi6XY/jw4VetpV+/fpbbYWFhAACtVtvm35GIpOEqdQFE1LmZTCYAwPr169GlS5cG9ymVygZB50oymQyAOKbHfNvs8kZqDw8Pq2pxc3Nr9Nzm+oio42FLDhFJqk+fPlAqlcjOzkb37t0bXCIjIy3n/fbbb5bbxcXFOHXqFHr37m15jp07dzZ43oyMDPTs2RNyuRx9+/aFyWRqMMaHiJwfW3KISFI+Pj545pln8NRTT8FkMuH666+HTqdDRkYGvL29ER0dDQCYN28eAgMDoVarMWfOHAQFBWHMmDEAgKeffhqDBg3CK6+8ggkTJmD37t1YsmQJli5dCgCIiYnB1KlT8eCDD+I///kPEhMTcf78eWi1WowfP16qX52I7Iwhh4gk98orryAkJATz58/H2bNn4efnh4EDB+L555+3dBe98cYbePLJJ3H69GkkJiZi3bp1UCgUAICBAwfi66+/xosvvohXXnkFYWFhmDdvHqZNm2Z5jWXLluH555/Ho48+isLCQkRFReH555+X4tclIgfh7CoiatfMM5+Ki4vh5+cndTlE1IFwTA4RERE5JYYcIiIickrsriIiIiKnxJYcIiIickoMOUREROSUGHKIiIjIKTHkEBERkVNiyCEiIiKnxJBDRERETokhh4iIiJwSQw4RERE5pf8HoxIoEluRpdgAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# evaluate model\n",
    "score = best_model.evaluate(X_test, Y_test, verbose=1)\n",
    "\n",
    "# print performance\n",
    "print()\n",
    "print('Test loss:', score[0])\n",
    "print('Test accuracy:', score[1])\n",
    "\n",
    "# look into training history\n",
    "\n",
    "# summarize history for accuracy\n",
    "plt.plot(history_.history['accuracy'])\n",
    "plt.plot(history_.history['val_accuracy'])\n",
    "plt.ylabel('model accuracy')\n",
    "plt.xlabel('epoch')\n",
    "plt.legend(['train', 'test'], loc='best')\n",
    "plt.show()\n",
    "\n",
    "# summarize history for loss\n",
    "plt.plot(history_.history['loss'])\n",
    "plt.plot(history_.history['val_loss'])\n",
    "plt.ylabel('model loss')\n",
    "plt.xlabel('epoch')\n",
    "plt.legend(['train', 'test'], loc='best')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Step 6: Modify the Hyperparameters to Optimize Performance of the Model\n",
    "\n",
    "Last, we show how to use the grid search option of scikit-learn to optimize the \n",
    "hyperparameters of our model. An excellent blog on this by Jason Brownlee can be found on [https://machinelearningmastery.com/grid-search-hyperparameters-deep-learning-models-python-keras/](https://machinelearningmastery.com/grid-search-hyperparameters-deep-learning-models-python-keras/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import GridSearchCV\n",
    "from keras.wrappers.scikit_learn import KerasClassifier\n",
    "\n",
    "# call Keras scikit wrapper\n",
    "model_gridsearch = KerasClassifier(build_fn=compile_model_(0.25), \n",
    "                        epochs=15, \n",
    "                        batch_size=64, \n",
    "                        verbose=1)\n",
    "\n",
    "# list of allowed optional arguments for the optimizer, see `compile_model()`\n",
    "optimizer = ['SGD', 'RMSprop', 'Adagrad', 'Adadelta', 'Adam', 'Adamax', 'Nadam']\n",
    "ly1 = range(200, 500, 10)\n",
    "ly2 = range(230, 340, 10)\n",
    "ly3 = range(10, 100, 10)\n",
    "\n",
    "# define parameter dictionary\n",
    "param_grid = dict(ly3=ly3) # param_grid = dict(optimizer=optimizer)\n",
    "# call scikit grid search module\n",
    "grid = GridSearchCV(estimator=model_gridsearch, param_grid=param_grid, n_jobs=10, cv=4)\n",
    "grid_result = grid.fit(X_train,Y_train)\n",
    "\n",
    "# summarize results\n",
    "print(\"Best: %f using %s\" % (grid_result.best_score_, grid_result.best_params_))\n",
    "means = grid_result.cv_results_['mean_test_score']\n",
    "stds = grid_result.cv_results_['std_test_score']\n",
    "params = grid_result.cv_results_['params']\n",
    "for mean, stdev, param in zip(means, stds, params):\n",
    "    print(\"%f (%f) with: %r\" % (mean, stdev, param))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating Convolutional Neural Nets with Keras\n",
    "\n",
    "We have so far considered each MNIST data sample as a $(28\\times 28,)$-long 1d vector. This approach neglects any spatial structure in the image. On the other hand, we do know that in every one of the hand-written digits there are *local* spatial correlations between the pixels, which we would like to take advantage of to improve the accuracy of our classification model. To this end, we first need to reshape the training and test input data as follows"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X_train shape: (60000, 28, 28, 1)\n",
      "Y_train shape: (60000, 10)\n",
      "\n",
      "60000 train samples\n",
      "10000 test samples\n"
     ]
    }
   ],
   "source": [
    "# reshape data, depending on Keras backend\n",
    "if keras.backend.image_data_format() == 'channels_first':\n",
    "    X_train = X_train.reshape(X_train.shape[0], 1, img_rows, img_cols)\n",
    "    X_test = X_test.reshape(X_test.shape[0], 1, img_rows, img_cols)\n",
    "    input_shape = (1, img_rows, img_cols)\n",
    "else:\n",
    "    X_train = X_train.reshape(X_train.shape[0], img_rows, img_cols, 1)\n",
    "    X_test = X_test.reshape(X_test.shape[0], img_rows, img_cols, 1)\n",
    "    input_shape = (img_rows, img_cols, 1)\n",
    "    \n",
    "print('X_train shape:', X_train.shape)\n",
    "print('Y_train shape:', Y_train.shape)\n",
    "print()\n",
    "print(X_train.shape[0], 'train samples')\n",
    "print(X_test.shape[0], 'test samples')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One can ask the question of whether a neural net can learn to recognize such local patterns. As we saw in Sec. X of the review, this can be achieved by using convolutional layers. Luckily, all we need to do is change the architecture of our DNN, i.e. introduce small changes to the function `create_model()`. We can also merge **Step 2** and **Step 3** for convenience: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_CNN():\n",
    "    # instantiate model\n",
    "    model = Sequential()\n",
    "    # add first convolutional layer with 10 filters (dimensionality of output space)\n",
    "    model.add(Conv2D(10, kernel_size=(5, 5),\n",
    "                     activation='relu',\n",
    "                     input_shape=input_shape))\n",
    "    # add 2D pooling layer\n",
    "    model.add(MaxPooling2D(pool_size=(2, 2)))\n",
    "    # add second convolutional layer with 20 filters\n",
    "    model.add(Conv2D(20, (5, 5), activation='relu'))\n",
    "    # apply dropout with rate 0.5\n",
    "    model.add(Dropout(0.5))\n",
    "    # add 2D pooling layer\n",
    "    model.add(MaxPooling2D(pool_size=(2, 2)))\n",
    "    # flatten data\n",
    "    model.add(Flatten())\n",
    "    # add a dense all-to-all relu layer\n",
    "    model.add(Dense(20*4*4, activation='relu'))\n",
    "    # apply dropout with rate 0.5\n",
    "    model.add(Dropout(0.5))\n",
    "    # soft-max layer\n",
    "    model.add(Dense(num_classes, activation='softmax'))\n",
    "    \n",
    "    # compile the model\n",
    "    model.compile(loss=keras.losses.categorical_crossentropy,\n",
    "                  optimizer='Adam',\n",
    "                  metrics=['accuracy'])\n",
    "    \n",
    "    return model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Training the deep conv net (**Step 4**) and evaluating its performance (**Step 6**) proceeds exactly as before:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "938/938 [==============================] - 9s 10ms/step - loss: 0.2432 - accuracy: 0.9233 - val_loss: 0.0775 - val_accuracy: 0.9831\n",
      "Epoch 2/10\n",
      "938/938 [==============================] - 9s 10ms/step - loss: 0.0947 - accuracy: 0.9717 - val_loss: 0.0604 - val_accuracy: 0.9849\n",
      "Epoch 3/10\n",
      "938/938 [==============================] - 9s 10ms/step - loss: 0.0730 - accuracy: 0.9772 - val_loss: 0.0486 - val_accuracy: 0.9883\n",
      "Epoch 4/10\n",
      "938/938 [==============================] - 9s 10ms/step - loss: 0.0602 - accuracy: 0.9813 - val_loss: 0.0375 - val_accuracy: 0.9898\n",
      "Epoch 5/10\n",
      "938/938 [==============================] - 9s 10ms/step - loss: 0.0562 - accuracy: 0.9826 - val_loss: 0.0336 - val_accuracy: 0.9922\n",
      "Epoch 6/10\n",
      "938/938 [==============================] - 9s 10ms/step - loss: 0.0514 - accuracy: 0.9844 - val_loss: 0.0311 - val_accuracy: 0.9917\n",
      "Epoch 7/10\n",
      "938/938 [==============================] - 9s 10ms/step - loss: 0.0466 - accuracy: 0.9851 - val_loss: 0.0320 - val_accuracy: 0.9911\n",
      "Epoch 8/10\n",
      "938/938 [==============================] - 9s 10ms/step - loss: 0.0434 - accuracy: 0.9861 - val_loss: 0.0296 - val_accuracy: 0.9915\n",
      "Epoch 9/10\n",
      "938/938 [==============================] - 9s 10ms/step - loss: 0.0412 - accuracy: 0.9875 - val_loss: 0.0296 - val_accuracy: 0.9915\n",
      "Epoch 10/10\n",
      "938/938 [==============================] - 10s 10ms/step - loss: 0.0404 - accuracy: 0.9875 - val_loss: 0.0274 - val_accuracy: 0.9919\n",
      "313/313 [==============================] - 1s 2ms/step - loss: 0.0274 - accuracy: 0.9919\n",
      "\n",
      "Test loss: 0.027357378974556923\n",
      "Test accuracy: 0.9919000267982483\n"
     ]
    }
   ],
   "source": [
    "# training parameters\n",
    "batch_size = 64\n",
    "epochs = 10\n",
    "\n",
    "# create the deep conv net\n",
    "model_CNN=create_CNN()\n",
    "\n",
    "# train CNN\n",
    "model_CNN.fit(X_train, Y_train,\n",
    "          batch_size=batch_size,\n",
    "          epochs=epochs,\n",
    "          verbose=1,\n",
    "          validation_data=(X_test, Y_test))\n",
    "\n",
    "# evaliate model\n",
    "score = model_CNN.evaluate(X_test, Y_test, verbose=1)\n",
    "\n",
    "# print performance\n",
    "print()\n",
    "print('Test loss:', score[0])\n",
    "print('Test accuracy:', score[1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}