intro_to_deep_learning/04-overfitting-and-underfitting.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "ad442dfc",
   "metadata": {
    "papermill": {
     "duration": 0.014978,
     "end_time": "2022-02-05T23:15:40.277539",
     "exception": false,
     "start_time": "2022-02-05T23:15:40.262561",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "**This notebook is an exercise in the [Intro to Deep Learning](https://www.kaggle.com/learn/intro-to-deep-learning) course.  You can reference the tutorial at [this link](https://www.kaggle.com/ryanholbrook/overfitting-and-underfitting).**\n",
    "\n",
    "---\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3c4a33f3",
   "metadata": {
    "papermill": {
     "duration": 0.013099,
     "end_time": "2022-02-05T23:15:40.305142",
     "exception": false,
     "start_time": "2022-02-05T23:15:40.292043",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Introduction #\n",
    "\n",
    "In this exercise, you’ll learn how to improve training outcomes by including an early stopping callback to prevent overfitting.\n",
    "\n",
    "When you're ready, run this next cell to set everything up!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "65200ec1",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-05T23:15:40.337169Z",
     "iopub.status.busy": "2022-02-05T23:15:40.335484Z",
     "iopub.status.idle": "2022-02-05T23:15:40.366083Z",
     "shell.execute_reply": "2022-02-05T23:15:40.365589Z",
     "shell.execute_reply.started": "2022-02-05T23:06:54.528922Z"
    },
    "papermill": {
     "duration": 0.047657,
     "end_time": "2022-02-05T23:15:40.366253",
     "exception": false,
     "start_time": "2022-02-05T23:15:40.318596",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Setup plotting\n",
    "import matplotlib.pyplot as plt\n",
    "plt.style.use('seaborn-whitegrid')\n",
    "# Set Matplotlib defaults\n",
    "plt.rc('figure', autolayout=True)\n",
    "plt.rc('axes', labelweight='bold', labelsize='large',\n",
    "       titleweight='bold', titlesize=18, titlepad=10)\n",
    "plt.rc('animation', html='html5')\n",
    "\n",
    "# Setup feedback system\n",
    "from learntools.core import binder\n",
    "binder.bind(globals())\n",
    "from learntools.deep_learning_intro.ex4 import *"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c1d590e4",
   "metadata": {
    "papermill": {
     "duration": 0.020957,
     "end_time": "2022-02-05T23:15:40.400648",
     "exception": false,
     "start_time": "2022-02-05T23:15:40.379691",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "First load the *Spotify* dataset. Your task will be to predict the popularity of a song based on various audio features, like `'tempo'`, `'danceability'`, and `'mode'`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "392d608e",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-05T23:15:40.455715Z",
     "iopub.status.busy": "2022-02-05T23:15:40.454834Z",
     "iopub.status.idle": "2022-02-05T23:15:46.600026Z",
     "shell.execute_reply": "2022-02-05T23:15:46.600666Z",
     "shell.execute_reply.started": "2022-02-05T23:09:31.944941Z"
    },
    "papermill": {
     "duration": 6.176258,
     "end_time": "2022-02-05T23:15:46.600917",
     "exception": false,
     "start_time": "2022-02-05T23:15:40.424659",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input shape: [18]\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "from sklearn.preprocessing import StandardScaler, OneHotEncoder\n",
    "from sklearn.compose import make_column_transformer\n",
    "from sklearn.model_selection import GroupShuffleSplit\n",
    "\n",
    "from tensorflow import keras\n",
    "from tensorflow.keras import layers\n",
    "from tensorflow.keras import callbacks\n",
    "\n",
    "spotify = pd.read_csv('../input/dl-course-data/spotify.csv')\n",
    "\n",
    "X = spotify.copy().dropna()\n",
    "y = X.pop('track_popularity')\n",
    "artists = X['track_artist']\n",
    "\n",
    "features_num = ['danceability', 'energy', 'key', 'loudness', 'mode',\n",
    "                'speechiness', 'acousticness', 'instrumentalness',\n",
    "                'liveness', 'valence', 'tempo', 'duration_ms']\n",
    "features_cat = ['playlist_genre']\n",
    "\n",
    "preprocessor = make_column_transformer(\n",
    "    (StandardScaler(), features_num),\n",
    "    (OneHotEncoder(), features_cat),\n",
    ")\n",
    "\n",
    "# We'll do a \"grouped\" split to keep all of an artist's songs in one\n",
    "# split or the other. This is to help prevent signal leakage.\n",
    "def group_split(X, y, group, train_size=0.75):\n",
    "    splitter = GroupShuffleSplit(train_size=train_size)\n",
    "    train, test = next(splitter.split(X, y, groups=group))\n",
    "    return (X.iloc[train], X.iloc[test], y.iloc[train], y.iloc[test])\n",
    "\n",
    "X_train, X_valid, y_train, y_valid = group_split(X, y, artists)\n",
    "\n",
    "X_train = preprocessor.fit_transform(X_train)\n",
    "X_valid = preprocessor.transform(X_valid)\n",
    "y_train = y_train / 100 # popularity is on a scale 0-100, so this rescales to 0-1.\n",
    "y_valid = y_valid / 100\n",
    "\n",
    "input_shape = [X_train.shape[1]]\n",
    "print(\"Input shape: {}\".format(input_shape))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cc4b1d1d",
   "metadata": {
    "papermill": {
     "duration": 0.013766,
     "end_time": "2022-02-05T23:15:46.629929",
     "exception": false,
     "start_time": "2022-02-05T23:15:46.616163",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Let's start with the simplest network, a linear model. This model has low capacity.\n",
    "\n",
    "Run this next cell without any changes to train a linear model on the *Spotify* dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "fca3ede8",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-05T23:15:46.663906Z",
     "iopub.status.busy": "2022-02-05T23:15:46.663180Z",
     "iopub.status.idle": "2022-02-05T23:15:59.823947Z",
     "shell.execute_reply": "2022-02-05T23:15:59.824376Z",
     "shell.execute_reply.started": "2022-02-05T23:09:45.490981Z"
    },
    "lines_to_next_cell": 0,
    "papermill": {
     "duration": 13.181016,
     "end_time": "2022-02-05T23:15:59.824522",
     "exception": false,
     "start_time": "2022-02-05T23:15:46.643506",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2022-02-05 23:15:46.740470: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2022-02-05 23:15:46.843453: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2022-02-05 23:15:46.844168: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2022-02-05 23:15:46.845851: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA\n",
      "To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.\n",
      "2022-02-05 23:15:46.848658: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2022-02-05 23:15:46.849388: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2022-02-05 23:15:46.850000: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2022-02-05 23:15:48.552266: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2022-02-05 23:15:48.553071: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2022-02-05 23:15:48.553724: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2022-02-05 23:15:48.554321: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 15403 MB memory:  -> device: 0, name: Tesla P100-PCIE-16GB, pci bus id: 0000:00:04.0, compute capability: 6.0\n",
      "2022-02-05 23:15:48.983770: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Minimum Validation Loss: 0.1999\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAyp0lEQVR4nO3deXxU9b3/8deZmSxAFkhCJgFCWBIwQhAUBK2QGkQgkQsK1Fpaq1fKLdcFq7+qtL3Ui3XXVq73Xi0XS92qVotSCBZqUOLKJhjACLIEQyDDkpAQIAmZmd8fkwyELBPIcmaS9/Ohj5lz5uScz3zJzDvnfL/nHMPtdrsRERHxMxazCxAREWmIAkpERPySAkpERPySAkpERPySAkpERPySrT03tnnz5vbcnIiIBIgrrrii3rx2DajGirgQeXl5pKSktFI1HZPaqGlqH9/URr6pjZp2Ie3T2M6LDvGJiIhfUkCJiIhfUkCJiIhf8hlQ8+fP56qrruKGG25o8HW3283vfvc7JkyYwJQpU9ixY0erFykiIp2Pz4C66aabWLJkSaOv5+TkkJ+fz5o1a3jkkUd4+OGHW7M+ERHppHwG1KhRo4iMjGz09ezsbKZNm4ZhGAwfPpyysjIOHz7cqkWKiEjn0+Jh5g6Hg7i4OO90XFwcDoeD2NjYBpfPy8tr0fYqKipavI6OTm3UNLWPb2oj39RGTWuN9mn386Baet6Azj3wTW3UNLWPb2oj39RGTfOL86DsdjtFRUXe6aKiIux2e0tXKyIizTRixAizS2gTLQ6o9PR03nvvPdxuN1u3biU8PLzRw3st9U1RGQvXFlFxxtkm6xcREf/h8xDffffdx4YNGygpKWHcuHHcfffdVFdXA3DLLbeQlpbGunXrmDBhAl26dOGxxx5rs2ILS07zecEptheWMrJfVJttR0QkELndbp566ik+/vhjDMNg7ty5ZGRkcPjwYX7xi19QXl6O0+nk4YcfZsSIEfz6179m+/btGIbB9OnTue2228x+C3X4DKjf//73Tb5uGAa//e1vW62gpgzt7RlNqIASEX/0t80H+OumglZd5w9GJjD9ij7NWnbNmjV88803LF++nJKSEmbMmMHIkSNZuXIl11xzDXPnzsXpdHL69Gny8vJwOBysXLkSgLKyslatuzUE1JUkYsND6BFqZVuh/zWkiIjZNm/eTGZmJlarlZiYGEaNGsW2bdtITU1l2bJlPP/88+zatYuwsDASEhIoKCjgkUceIScnh7CwMLPLr6fdR/G1hGEYDIwOZsfBUrNLERGpZ/oVfZq9t9OeRo0axWuvvca6det46KGHuP3225k2bRrLly/nk08+4c033+T999/n8ccfN7vUOgJqDwogKSqEbw+Xa6CEiMh5Ro4cyfvvv4/T6aS4uJhNmzYxbNgwCgsLiYmJ4Qc/+AEzZ85kx44dFBcX43a7mThxIvfeey9ff/212eXXE1B7UABJ0SE4XW6+KTrB8ITuZpcjIuI3JkyYwJYtW5g6dSqGYfDLX/6Snj178u677/LSSy9hs9no2rUrTz75JIcPH2b+/Pm4XC7AMyDO3wRgQAUDnoESCigREdiyZQvg6QZ58MEHefDBB+u8fuONN3LjjTfW+7l33323Xeq7WAF3iC+2m43uXYPYXqh+KBGRjizgAsowDIb2imS7BkqIiHRoARdQAEN6R7Cz6ARV1S6zSxERkTYSkAE1tFckZ5xudjlOmF2KiIi0kYAMqNSaK0rofCgRkY4rIAOqb1RXwkNsbNNACRGRDisgA8piMbi0VwTbdckjEZEOKyADCjwXjs07VEa1UwMlREQuRFP3jzpw4AA33HBDO1bTuAAOqAgqq13sOXLS7FJERKQNBNyVJGqlnnPrjcFx4SZXIyICbH0DtrzWuusc8WMYfkuTizzzzDPEx8cza9YsAJ5//nmsVivr16+nrKyM6upq5s2bx3XXXXdBm66srOThhx9m+/btWK1WHnroIcaMGcO3337L/PnzOXPmDC6Xi+eff57Y2FjuvfdeioqKcLlcTJ06tdm3fG9MwAZU/5gwugRZ2VZY6pdXDxYRaS8ZGRk89thj3oB6//33eemll7j11lsJCwujuLiYm2++mfHjx2MYRrPX+/rrrwOwYsUK9uzZwx133MHq1at58803ufXWW/mXf/kXqqqqcLlcrFu3jtjYWBYvXgzApk2bWvy+AjagrDUDJTTUXET8xvBbfO7ttIVLL72UY8eO4XA4KCkpISIigpiYGB5//HE2btyIxWLB4XBw9OhRevbs2ez1bt68mR//+McADBw4kF69erFv3z6GDx/Oiy++SFFREddffz39+vVj0KBBPPnkkzz99NNce+21dOvWrcXvK2D7oACG9opgx8EyXC632aWIiJhq0qRJrF69mlWrVpGRkcGKFSsoLi5m2bJlLF++nJiYGCorK1tlW1OmTOGFF14gNDSUOXPm8Pnnn9O/f3+WLVvGoEGDeO6553jrrbdavJ2ADqghvSM5VeVk3zENlBCRzi0jI4NVq1axevVqJk2axIkTJ4iOjiYoKIgvvviCwsLCC17nyJEjWbFiBQD79u3j0KFDDBgwgIKCAhISErj11lsZP348O3fuxOFw0KVLF6ZOncodd9zBnj17WvyeAusQX/4nDFz5M+j/BYRG1hkoMbCn/92uWESkvSQnJ3Py5EliY2OJjY1lypQpzJ07lylTpjB06FAGDBhwwev80Y9+xMMPP8yUKVOwWq08/vjjBAcH8/7777N8+XJsNhsxMTH827/9G9u2beOpp57CYrFgs9m47bbbWvyemhVQOTk5PProo7hcLmbOnMmcOXPqvF5YWMivfvUriouL6d69O08//TRxcXEtLq4eVzXBJw/CgU2QNJ6k2DCCbRa2F5YydXjv1t+eiEgAqd3bAYiKimr0MFvt/aMa0qdPH1auXAlASEhIg7eBnzNnTr0cGDt2LGPHjvVO5+XlXVDtDfF5iM/pdLJw4UKWLFlCVlYWK1euZPfu3XWWefLJJ5k2bRorVqzg3//933n22WdbXFiD4od7Hg96GjfIaiElLlxXlBAR6YB87kHl5uaSmJhIQkICAJmZmWRnZ5OUlORdZs+ePcyfPx+AMWPGcOedd7ZNtV26UxmWQMjBs+k/pHckK746iNvtvqDhkyIindnOnTt54IEH6swLDg7m7bffNqmi+nwGlMPhqHO4zm63k5ubW2eZSy65hDVr1vDTn/6Uf/7zn5w8eZKSkhJ69OhRb30t3e2zdx+MZf8GdtesJ9pyihMV1Xy4cRvx4UEtWndHUVFR0Sq71x2V2sc3tZFvHaGNnnjiiXrzWus9tUb7tMogiQceeIBHHnmEd999l5EjR2K327FarQ0u29Izix07hxB04ANS+kRBuJ0z4cd5/vOjVHTpSUpKfIvW3VHk5eW1uJ07MrWPb2oj39RGTbuQ9tm8eXOD830GlN1up6ioyDvtcDiw2+31lvnv//5vAE6ePMmaNWuIiIhoVmEX6nRUzRs+uAUGT2JwXDg2i8G2wlIyUhVQIiIdhc9BEqmpqeTn51NQUEBVVRVZWVmkp6fXWaa4uBiXy3NV8cWLFzN9+vS2qRao6DEYDIt3oESIzcogezjbdW8oEZEOxWdA2Ww2FixYwOzZs8nIyGDy5MkkJyezaNEisrOzAdiwYQOTJk1i4sSJHD16lLlz57ZZwW5bF+h5CRz80jtvaG/PFSXcbl1RQkSko2hWH1RaWhppaWl15s2bN8/7fNKkSUyaNKl1K2tKrxHw7Rpwu8EwGNo7kr9uOsCh0gp6de/SfnWIiEibCcxLHfUaASePQJnn0h1Dep29ooSIiHQMARpQl3seCz2H+S6Nj8BiKKBERDqSwAwo+xCw2LwDJboEW0mKDWP7QV1RQkSkowjMgAoK9YTUuQMlekVqD0pEpAMJzIACTz/UwS2egRJ4Lnl0+EQlh8sqTC5MRERaQwAH1OVQUQrFewHPzQsBtusOuyIiHUIAB9QIz2NNP9SQ3pEYBuQeUECJiHQEgRtQsSlgC/UGVFiIjaSeYQooEZEOInADyhoEcanegAIY1qc7uQeO64oSIiIdQOAGFHgO8x36ClxOAC5LiORoeRWFx0+bXJiIiLRUgAfU5VBVDke/BTx7UKB+KBGRjiDAA6ruQImU+HCCrAZfHThuXk0iItIqAjugYpIhOMx7wm6IzUpKfAS5BdqDEhEJdIEdUBYrxF923kAJzxUlXC4NlBARCWSBHVDgOcxXtA2cZwBPP9SJymr2Hj1pcmEiItISHSOgqivgcB4Al3kHShw3ryYREWmxwA+o3jW33qg5zJcUG0bXYCtfFRw3ryYREWmxwA+oHv0hNNI7UMJqMRjaK5KvNNRcRCSgBX5AGcbZK5vXuCwhkq8PlVFV7TKxMBERaYnADyjwnLDr2AFnPLfaGNanO1XVLnY5TphcmIiIXKxmBVROTg4TJ05kwoQJLF68uN7rBw8e5Cc/+QnTpk1jypQprFu3rtULbVKvEeCq9oQUZwdK6IRdEZHA5TOgnE4nCxcuZMmSJWRlZbFy5Up2795dZ5kXXniByZMn89577/GHP/yB//zP/2yzghvkHSjh6YdKiOpCj65BOmFXRCSA+Qyo3NxcEhMTSUhIIDg4mMzMTLKzs+ssYxgG5eXlAJw4cYLY2Ni2qbYxEb2hW09vP5RhGKT26a49KBGRAGbztYDD4SAuLs47bbfbyc3NrbPMXXfdxR133MFrr73G6dOnWbp0aaPry8vLa0G5UFFR0eA6+kQMImjfF+yrea136Bk++fYEW3J3EBrUMbramquxNhIPtY9vaiPf1EZNa4328RlQzZGVlcWNN97Iv/7rv7JlyxYeeOABVq5cicVSPxhSUlJatK28vLyG11E0FnKeImVgXwjuxni3gzdyN1EdEU9Kv6gWbTPQNNpGAqh9mkNt5JvaqGkX0j6bN29ucL7PXQu73U5RUZF32uFwYLfb6yzzzjvvMHnyZABGjBhBZWUlJSUlzSqs1fQaAW6X5/5QwLCESACdsCsiEqB8BlRqair5+fkUFBRQVVVFVlYW6enpdZaJj4/n888/B2DPnj1UVlYSFdXOey29r/A8FnoGSsSGhxIfGap7Q4mIBCifh/hsNhsLFixg9uzZOJ1Opk+fTnJyMosWLWLo0KGMHz+ehx56iN/85jf8+c9/xjAMnnjiCQzDaI/6zwrrCd37woGN3lnD+kTqmnwiIgGqWX1QaWlppKWl1Zk3b9487/OkpCTefPPN1q3sYvQZBQUbvJPD+nRn9Q4HpafOENk1yMTCRETkQnWs4W29R0JpAZQdAs65snnhcfNqEhGRi9KxAqrPKM9j4SYAUvtooISISKDqWAEVPwyswXDAE1CRXYIYENNNVzYXEQlAHSugbCEQl+oNKNBACRGRQNWxAgo8h/kOfgnOasAzUMJRVomjrMLkwkRE5EJ0vIDqPRLOnIIjNbeA1wm7IiIBqeMFVJ+Rnsea86EujY/EajF0wq6ISIDpeAHVox90jYYDnms7dQm2Mtgeriubi4gEmI4XUIbh6Yc654oSlyVEknugFLfbbWJhIiJyITpeQIHnMN/RnXD6OOAZKFF6+gz7j50yty4REWm2jhlQvWv6oWrusDus9oRdHeYTEQkYHTSgLgcM7/lQg+zhhAZZNFBCRCSAdMyACo2EnoO9/VBBVgtDe0Wy5bt2vkeViIhctI4ZUODphzqwCWoGRlye2IPthWVUVjtNLkxERJqjAwfUKDhdDMV7Abi8b3eqnC52HCwzuTAREWmOjhtQtQMlCj3nQ13etwcAX+7XYT4RkUDQcQMqNgWCunn7oWIjQundvQtbdMkjEZGA0HEDymL1jOY754TdEX27s0V7UCIiAaHjBhR4BkoUbYMzpwHPYb6DpRUUlerK5iIi/q6DB9QocFXDoVzAM5IP4EsNNxcR8Xu25iyUk5PDo48+isvlYubMmcyZM6fO64899hjr168HoKKigmPHjrFp06aGVtW+ep9zZfO+o7k0PoIQm4Uv95eQkRpvbm0iItIknwHldDpZuHAhS5cuxW63M2PGDNLT00lKSvIu86tf/cr7/NVXX+Xrr79um2ovVLgdIvtCoScsg20WUntHag9KRCQA+DzEl5ubS2JiIgkJCQQHB5OZmUl2dnajy2dlZXHDDTe0apEt0ueKOreAvzyxB9sP6oRdERF/53MPyuFwEBcX55222+3k5uY2uGxhYSEHDhxgzJgxja4vLy/vIso8q6Ki4oLWERXUF3vpu3z75cdUd4mhp6WcqmoXqz7L5ZKeoS2qxV9daBt1Nmof39RGvqmNmtYa7dOsPqjmysrKYuLEiVit1kaXSUlJadE28vLyLmwd3TJh6yKSu5RCyliielfw6EfZFFu6k5LSv0W1+KsLbqNORu3jm9rIN7VR0y6kfTZv3tzgfJ+H+Ox2O0VFRd5ph8OB3W5vcNlVq1aRmZnZrILaTfwwsAR5z4ey15ywq34oERH/5jOgUlNTyc/Pp6CggKqqKrKyskhPT6+33J49eygrK2PEiBFtUuhFC+oCcaneSx6BTtgVEQkEPgPKZrOxYMECZs+eTUZGBpMnTyY5OZlFixbVGSyxatUqMjIyMAyjTQu+KH1GQuGX4KwGdMKuiEggaFYfVFpaGmlpaXXmzZs3r8703Xff3XpVtbY+o2DDYjiSB3GpjOjbHYAt35UwWedDiYj4pY59JYlava/wPNYMNx/SK5Jgm0X9UCIifqxzBFTUAOga7R0ocfaE3ePm1iUiIo3qHAFlGJAwGr77wjvr8r7d2VZYSlW1y8TCRESkMZ0joAD6joHiPVB+GPAMlKiqdrHjYKnJhYmISEM6UUBd5Xms2YsaUXuHXR3mExHxS50noOIvA1uoN6DiIkPpFRnKFg2UEBHxS50noGwhntF8333unTUisQdbtAclIuKXOk9Agacf6tBXUHUS8PRDFR4/jaNMJ+yKiPibThZQV4Hb6T0f6vKaE3a/1GWPRET8TucKqD6jAMPbD6UTdkVE/FfnCqgu3cE+xNsPFWyzMLRXhPqhRET8UOcKKPD0Qx3YWOfCsbk6YVdExO90woC6CqrKwbEd8NwCvqraxdeHykwuTEREztUJA6rmdvQ1/VCX156wq4ESIiJ+pfMFVGQfiEzw9kPVnrCrgRIiIv6l8wUUePaivvsC3G7Ac8Ku9qBERPxL5w2o8iIoyQfgyn5RHCyt4EDJKXPrEhERr04aUHUvHHtl/ygANuwrNqsiERE5T+cMqJ4pEBLp7YcabA8nItSmgBIR8SOdM6AsFuh79gaGFovBqH5RbMhXQImI+ItmBVROTg4TJ05kwoQJLF68uMFlVq1aRUZGBpmZmdx///2tWmSb6DsGju6Ek8cAz2G+vUdOcuREpcmFiYgIgM3XAk6nk4ULF7J06VLsdjszZswgPT2dpKQk7zL5+fksXryYN954g8jISI4dO9amRbeK2n6ogvVwSYa3H2pjfjEZqfEmFiYiItCMPajc3FwSExNJSEggODiYzMxMsrOz6yzz17/+lVmzZhEZGQlAdHR021TbmnpdDtZgbz/U0N6RdAmyqh9KRMRP+NyDcjgcxMXFeaftdju5ubl1lsnPzwfghz/8IS6Xi7vuuotx48Y1uL68vLwWlAsVFRUtXketxO6DYeeH7E/wrO+SmGBy8g6SlxzYXXOt2UYdkdrHN7WRb2qjprVG+/gMqOZwOp3s37+fV199laKiIn784x+zYsUKIiIi6i2bkpLSom3l5eW1eB1eB9Lh8/8lJakfBHXh2oM2/vDBLnolJhHZNah1tmGCVm2jDkjt45vayDe1UdMupH02b97c4Hyfuwp2u52ioiLvtMPhwG6311smPT2doKAgEhIS6Nevn3evyq/1vQpcZ6DwSwBG9YvC7YZN+3WYT0TEbD4DKjU1lfz8fAoKCqiqqiIrK4v09PQ6y1x33XVs2LABgOLiYvLz80lISGibiltTwmjPY4FnuPmIvt0Jshoabi4i4gd8HuKz2WwsWLCA2bNn43Q6mT59OsnJySxatIihQ4cyfvx4xo4dy6effkpGRgZWq5UHHniAHj16tEf9LdM1Cnpe4j0fKjTIymV9umughIiIH2hWH1RaWhppaWl15s2bN8/73DAM5s+fz/z581u3uvbQdwxsfxdcLrBYuLJ/FItz9nKqqpquwa3SRSciIhchsIertYa+V0FlKRzxjDa5sn8U1S63bgMvImIyBZT3Boae86GuSOyBxYD1OswnImIqBVT3RAiP9/ZDhYcGcWmvCDbsC4CrYYiIdGAKKMPw7EXlf+q9geGV/aLZ8t1xKqudJhcnItJ5KaAA+o2FEweheC/g6YeqrHaxvbDU5MJERDovBRRA/5oRins/AmBUP88QefVDiYiYRwEFED0QInrDvhzPZFgIybFhOh9KRMRECijw9EP1H+cJKJcL8Bzm25RfgtPlNrk4EZHOSQFVq38anC6GwzsAT0CVV1aTd6jM5MJERDonBVSt/jW3B9m7DsB7A0P1Q4mImEMBVSuyN0QnwT5PQMVHdiEhqovOhxIRMYkC6lz902D/Z+A8A3jOh9qYX4LbrX4oEZH2poA6V/9xUFXuvT/U6P5RFJ+sYs+RcpMLExHpfBRQ56rth6oZbq5+KBER8yigztU1CuJSvf1QidFdiQ0P0flQIiImUECdr38aFKyHM6cxDIMr+0exfm+x+qFERNqZAup8/dPAWeW9uvno/lEUlVXwXfEpkwsTEelcFFDnS7wKLDZvP9TVSTEAfPztUTOrEhHpdBRQ5wsJh95XePuhBsR0o3f3LnyigBIRaVcKqIb0T4ODW6CiFMMwuCYphk/3HKXa6TK7MhGRTqNZAZWTk8PEiROZMGECixcvrvf6smXLGDNmDFOnTmXq1Km8/fbbrV5ou+o/Dtwuz00MgbGDYjhRUU2u7g8lItJubL4WcDqdLFy4kKVLl2K325kxYwbp6ekkJSXVWS4jI4MFCxa0WaHtKuFKsHXxHOa7JIPvDYzBMODjXUe5vG8Ps6sTEekUfO5B5ebmkpiYSEJCAsHBwWRmZpKdnd0etZnHFuK5DXzNQIke3YJJ7R3JJ7uPmFyYiEjn4XMPyuFwEBcX55222+3k5ubWW27NmjVs3LiR/v37M3/+fOLj4xtcX15eXgvKhYqKihavozmiw1KI3fshu7Z8gjM0mpQeBm9vL2HTVzvoFuzfXXft1UaBSu3jm9rIN7VR01qjfXwGVHNce+213HDDDQQHB/Pmm2/y4IMP8sorrzS4bEpKSou2lZeX1+J1NEvEDMj9XwbZiiDlGqYFH+OtbV9QEhTDyBR722+/BdqtjQKU2sc3tZFvaqOmXUj7bN68ucH5PncF7HY7RUVF3mmHw4HdXvcLukePHgQHBwMwc+ZMduzY0ayi/Fr8cAiJ9A43vzyxO12CrHz8rQ7ziYi0B58BlZqaSn5+PgUFBVRVVZGVlUV6enqdZQ4fPux9vnbtWgYOHNj6lbY3ixX6XePthwqxWRkzIErnQ4mItBOfh/hsNhsLFixg9uzZOJ1Opk+fTnJyMosWLWLo0KGMHz+eV199lbVr12K1WomMjOTxxx9vj9rbXv9xsDMLSvZDj0SuSe7Jhzu/5kDJKfr06Gp2dSIiHVqz+qDS0tJIS0urM2/evHne5/fffz/3339/61bmDwbUvOd9OdDjJ4xL9lz26JNvj/LDK/uaWJiISMfn38PRzNbzEugW6+2HSooNwx4RouvyiYi0AwVUUwzDc5hvXw643RiGwdjknny65yhOl26/ISLSlhRQvgxIg3IHODwjE8cmx3D81Bm267JHIiJtSgHlS/L1nsdd7wPwvZrbb3yyW4f5RETakgLKl/A46HU57PwHADFhIVwaH0HOLp0PJSLSlhRQzTF4MhRuhhMOwHN18y+/K+FkZbXJhYmIdFwKqOYYPBlww7erARib1JMzTjfr9x0zty4RkQ5MAdUc9qEQ0cd7mG9kvx6E2Cwabi4i0oYUUM1hGJ69qD1r4cxpQoOsXNk/SgElItKGFFDNNXgSVJ/2XptvXHJPdh8u51DpaZMLExHpmBRQzdVvLASHwU7PcPNrai57pL0oEZG2oYBqLlsIDEyHXf8At5tL4sKJCQvR1c1FRNqIAupCDM6AE4fg0Naayx7F8Mnuo7h02SMRkVangLoQydeDYfEe5hubHEPxySq+PlRmcmEiIh2PAupCdIuGhNF1+qEMA9Z+c9jHD4qIyIVSQF2oQZOgKBdKDxAbHsrlfXuwekeR2VWJiHQ4CqgLNXiy53GX56TdiUPs7DhYRkHxKROLEhHpeBRQFypmEEQN8F5VYuKQOADtRYmItDIF1IUyDBg02XOX3cpyEqO7kRIfoYASEWllCqiLMXgyOKtg74cATBoSx6b9JRw5UWlyYSIiHUezAionJ4eJEycyYcIEFi9e3Ohyq1evZvDgwWzbtq3VCvRLfcdAaKR3NN/EoXbcbvjn1w6TCxMR6Th8BpTT6WThwoUsWbKErKwsVq5cye7du+stV15eziuvvMJll13WJoX6FWuQ55yoXavB5WSwPZx+0V35hw7ziYi0Gp8BlZubS2JiIgkJCQQHB5OZmUl2dna95RYtWsTPfvYzQkJC2qRQvzNoEpw6CoWbMQyDiUPj+Gz3UUpPnzG7MhGRDsHmawGHw0FcXJx32m63k5ubW2eZHTt2UFRUxPe//31eeumlJteXl5d3kaV6VFRUtHgdrcHi6ssgw8qxz17lyLBwLulWQbXLzWvZW0gfGG5qbf7SRv5K7eOb2sg3tVHTWqN9fAaULy6XiyeeeILHH3+8WcunpKS0aHt5eXktXker2fo9Yo5uJCYlhcGD3TzxyTG2lVi50+T6/KqN/JDaxze1kW9qo6ZdSPts3ry5wfk+D/HZ7XaKis72rTgcDux2u3f65MmT7Nq1i1tvvZX09HS2bt3K3LlzO/5ACfAMNz+SB8X7sFgMJg6J46Ndhzld5TS7MhGRgOczoFJTU8nPz6egoICqqiqysrJIT0/3vh4eHs769etZu3Yta9euZfjw4bzwwgukpqa2aeF+ofaqEnl/BzzDzSvOuFi364iJRYmIdAw+A8pms7FgwQJmz55NRkYGkydPJjk5mUWLFjU4WKJTieoPfUbBV28BcGX/KLp3DdJJuyIiraBZfVBpaWmkpaXVmTdv3rwGl3311VdbXlUgGXYzrPp/ULQNW1wq16XYWbOjiKpqF8E2nQctInKx9A3aUkOngyUIvnoT8BzmK6uo5ou9x0wuTEQksCmgWqprFAyaCNveBmc11yTH0DXYqpN2RURaSAHVGobdDOUO2PcRoUFWrr0kljU7HDh1K3gRkYumgGoNgyZCaHfvYImJQ+I4Wl7Jlu9KzK1LRCSAKaBagy0EhtwI36yEynKuHdyTYKuFf2zXYT4RkYulgGotl90CZ05B3grCQ4O4JjmGf+wowu3WYT4RkYuhgGotCVdCj/7w1RuAZzTfgZLT7DhYZnJhIiKBSQHVWgzDM1hiXw6UFjI+JRaLgQ7ziYhcJAVUaxr2A8AN294mOiyE7yXF8LcvD1DtdJldmYhIwFFAtabogdDnSsh9C9xuZo3uy6HSCj7aqWvziYhcKAVUa7vsh3D4ayjaxvgUO7HhIfxlw3dmVyUiEnAUUK1tyI1gDYav3iTIauHmUQl8uPMwB0pOmV2ZiEhAUUC1tq5RkHy999JHP7yyLwbw1sYCsysTEQkoCqi2cNkP4eRh2PsRvbt34fuDY3lzYwFnNFhCRKTZFFBtIfl66NIDcj1XOJ81ui9HTlSSnecwuTARkcChgGoLthAYchPkrYTKE3x/cCy9IkN5fb0GS4iINJcCqq1c9kOoPg1f/x2rxeDmUX35+Nuj7D920uzKREQCggKqrfQZBdFJsGExuN3cPCoBq8XgjQ0aLCEi0hwKqLZiGPC9e+HQVtidTVxkKOMvieXtTQVUVWuwhIiILwqotjTsZohMgJynPFeWGJPIsZNVrNbddkVEfGpWQOXk5DBx4kQmTJjA4sWL673+xhtvMGXKFKZOncott9zC7t27W73QgGQLhu/Ng4L1kP8JY5NiSIjqwuvr95tdmYiI3/MZUE6nk4ULF7JkyRKysrJYuXJlvQCaMmUKK1asYPny5cyePZvHH3+8zQoOOCN+AmF2yHkai8Xgliv78sXeYnYfLje7MhERv+YzoHJzc0lMTCQhIYHg4GAyMzPJzs6us0xYWJj3+enTpzEMo/UrDVRBoXD1PbBvHRRsYOYVCdgsBm/o+nwiIk2y+VrA4XAQFxfnnbbb7eTm5tZb7vXXX2fp0qWcOXOGl19+udH15eXlXWSpHhUVFS1eR3szwq8mKTiSilW/5ei433N13678deN+piS6CbG1fjdgILZRe1L7+KY28k1t1LTWaB+fAdVcs2bNYtasWaxYsYIXXniBJ598ssHlUlJSWrSdvLy8Fq/DFCX3ELb2EVIiK/n5dUP50ZL17D0TyU2pfVp9UwHbRu1E7eOb2sg3tVHTLqR9Nm/e3OB8n3++2+12iorOjjpzOBzY7fZGl8/MzOSDDz5oVlGdypU/g9BI+PgZrhoYzYCYbrzy+X7cbrfZlYmI+CWfAZWamkp+fj4FBQVUVVWRlZVFenp6nWXy8/O9zz/66CMSExNbvdCAFxoJo38OeSswjnzDHWP7s7XgOO/rlvAiIg3yeYjPZrOxYMECZs+ejdPpZPr06SQnJ7No0SKGDh3K+PHjee211/j888+x2WxEREQ0eniv0xv9c/j8fyDnGW6+8f949fP9PJqVR/olsYQGWc2uTkTErzSrDyotLY20tLQ68+bNm+d9/pvf/KZ1q+qoukbBqDvgs+exfX8+C264lB8tWc//5ezl7vHJZlcnIuJXdCWJ9nbVXZ477n7yB65OimHSkDj+96M9HCo9bXZlIiJ+RQHV3sJi4YrbPPeKKtnPrzNTcLrdPPn+N2ZXJiLiVxRQZrj6HsCAT58jIaorc8YO4L2tB9m8v9jsykRE/IYCygyRveGKn8LmP0P+p8z9/kDsESH854qvcbk07FxEBBRQ5rnuYejRD/42m27OMh6afAm5B0r525cHzK5MRMQvKKDMEhIOM5bCySPw3r8z7bJeXN63O0/+YycnKs6YXZ2IiOkUUGbqNRyufwR2vY+xYTG/nTKEo+WV/M+He8yuTETEdAoos43+OQyaBP/8Dy6z5jPjij786ZN95B89aXZlIiKmUkCZzTBg6v9C1xh451958NreBFkNfpelqySLSOemgPIH3aJh+v9ByT565vyau8cn80GegyUf7zW7MhER0yig/EW/ayDtQch9kzkR68lMjed3WXn8dWOB2ZWJiJhCAeVPxv0SEq/Bsur/8YfrujFuUE8eWpbLqm2HzK5MRKTdKaD8icUKNy0GWwjBy+7gxen9ubxvD+a9uYV1u46YXZ2ISLtSQPmbyN5w4x/hyDd0/dO1/Pl6C8mx4fzbq5vYlK9LIYlI56GA8keDroc7VgMGYa/fwF9HfEWviFBu//NGdhwsNbs6EZF2oYDyV72vgJ/nQNJ1hK39NVnx/0dccCW3vrSBvUfKza5ORKTNKaD8WZcecMsbcP3v6LLnH6wK/Q8Gu/fy4yXr2Vl0wuzqRETalALK3xkGXH033L6KIHcVrxn/QUbVP8j4rxx+u3w7x09VmV2hiEibUEAFir5j4OcfY+l3Db9xL2Z9+INEbvwDP376TV75PJ9qp8vsCkVEWpUCKpB0i4FZ78C0F4iJ78d9tndY6b6bIe/P4IWnH+KLbTvNrlBEpNXYmrNQTk4Ojz76KC6Xi5kzZzJnzpw6ry9dupS3334bq9VKVFQUjz32GL17926Tgjs9iwWG/8jzf+kB3NveYdDG17mi9I+ceWcJue+PxN17NM6gdKyxgyE83nOYUEQkwPgMKKfTycKFC1m6dCl2u50ZM2aQnp5OUlKSd5mUlBT+9re/0aVLF/7yl7/w9NNP89xzz7Vl3QIQ2QfjmnsJv+ZeKg/ksmP1EuK/W0n8t+vh2/8CoMoWhjt6EMHxl2D0vAR69Ieu0TX/R3kGYliDTH4jIiL1+Qyo3NxcEhMTSUhIACAzM5Ps7Ow6ATVmzBjv8+HDh/P3v/+9DUqVpoT0Gcbld/wXJeVP8/IHH1J5/BAnDuygZ0U+SQcLGeR4nxj+0sgPR54Nq6AuYAsBW+1jKASFeh4tVjCsYLGd89zimTasYFjO/m+xevbcDAtQ82gYNc/PfzzvtTo/d+7eX81z7zxfe4ZucLvrPYYXFoIzr4HtNrTNxrbjrnlwN7zpJutuar3nrLvB9Z837X3dXff5udtosJ2bOrrvptuh78BaQINteH79tes8/3lj7827Ptd5626mRtvWqPu74a2l4d+Deu+pwfZrXHhhIbh21t3++c99vofGfjca4Ot3of5GGtlmE8s21La12zq/rQwL9B8HwV191HHxfAaUw+EgLi7OO22328nNzW10+XfeeYdx48Y1+npeXstuI1FRUdHidXR0wxLthA5OxH3laA6eqObTg6f5n0On2Fd0lB5nDtPdOEEPyulhnMBuPUmv6nJ6njxJj9PlhFJGCFUEc4YgdxVB7ips7ipsrioMtxPD7cLAhcXtNPttXrQ+ZhcQAPqaXUAA0O8ROIbfQ/HgHzX4Wmt8VzerD6q5li9fzvbt23nttdcaXSYlJaVF28jLy2vxOjq6c9voUuC6mvlOl5sDJac4VFpBUWkFB0tPc+h4BVtLKzhUeprDJyopr6jm9JnmhI8bC26suLDixIIbC67zHt2eMMON4Z121/yRVjsPz7KG59FqgGHUrNtwU/tXnFHzl9vZx6YZuGu2ULslg9qtOt1urIZRM/fsq4b3J85dT+OjI921Y4zO/0OVs3/snl/3ucucX6t32vD8nVo759zX3OdN1847u1ajzjzDMLDgqvP+DNwYbhd19ofcddflctfWUbd1auvx1Ox5o7XPa5c6n2GAUfMzGJ4lPL8VcG6re9fvraWBdZ3XbmfXUDt99t/VYpxtW5f7bNu53LX/qg1s+/z2O+991OF2YzHq/36e/Xdz13k/5/xgnXWf22YGbtxuT9Xn/pi7ZiVuDAzDOO83wKj9r047nVtvQ/8u576P+mpb6NzfL0vd3z/Dym2JmUxNSWhwtRfyXb158+YG5/sMKLvdTlFRkXfa4XBgt9vrLffZZ5/x4osv8tprrxEcHNysoqR9WS0GidHdSIzu1uRyTpebU1XVnKx0Ul5ZzamqasorqznjdHOm2kW1y0WV002100W1002V04XL7cblcuN0ez5MLrcbpwtcbjdutxu3m5ovCc/z2i83t+ehznxXzZMLOOhTj9vtxmjkkMaxY8eIjo5u9s81cYDP+8XheX7el8JFDE7xfhGdExj1Dgw2strzv2eaewCodn2G98u2bhvVO7B0ThsZ56zo3K9N79eru+G2aqz+hmpprga/0M9bp3d9Rt16G17fuYda62+rsd+jeu1zzpPmbNPAqAn1c2qvmVf7Wal9b+d+fuoVeN5kk1s+bzt1aufcf8O6dQ5LiGry/bSUz4BKTU0lPz+fgoIC7HY7WVlZPPvss3WW+frrr1mwYAFLlixp9IMvgcNqMQgPDSI8tGMOntBeuG9qI9/URm3PZ0DZbDYWLFjA7NmzcTqdTJ8+neTkZBYtWsTQoUMZP348Tz31FKdOnWLevHkAxMfH8+KLL7Z58SIi0nE1qw8qLS2NtLS0OvNqwwjgz3/+c6sWJSIioitJiIiIX1JAiYiIX1JAiYiIX1JAiYiIX1JAiYiIX1JAiYiIX1JAiYiIXzLcvq4/0ooau96SiIh0bldccUW9ee0aUCIiIs2lQ3wiIuKXFFAiIuKXFFAiIuKXWvWGhW0tJyeHRx99FJfLxcyZM5kzZ47ZJZlu/vz5fPTRR0RHR7Ny5UoAjh8/zi9+8QsKCwvp3bs3zz33HJGRkSZXao5Dhw7xwAMPcOzYMQzD4Ac/+AE//elP1UY1KisrmTVrFlVVVTidTiZOnMg999xDQUEB9913H8ePH2fIkCE89dRTnf4+b7V3c7Db7fzxj39UG50nPT2dbt26YbFYsFqtLFu2rMWfs4DZg3I6nSxcuJAlS5aQlZXFypUr2b17t9llme6mm25iyZIldeYtXryYq666ijVr1nDVVVexePFik6ozn9Vq5aGHHmLVqlW89dZb/OUvf2H37t1qoxrBwcG8/PLL/P3vf+e9997j448/ZuvWrTzzzDPcdttt/POf/yQiIoJ33nnH7FJN98orrzBw4EDvtNqovpdffpnly5ezbNkyoOXfRQETULm5uSQmJpKQkEBwcDCZmZlkZ2ebXZbpRo0aVe8vkuzsbKZNmwbAtGnT+OCDD0yozD/ExsYyZMgQAMLCwhgwYAAOh0NtVMMwDLp189xhubq6murqagzD4IsvvmDixIkA3HjjjZ3+s1ZUVMRHH33EjBkzAM9dc9VGvrX0cxYwAeVwOIiLi/NO2+12HA6HiRX5r2PHjhEbGwtAz549OXbsmMkV+YcDBw6Ql5fHZZddpjY6h9PpZOrUqVx99dVcffXVJCQkEBERgc3m6QGIi4vr9J+1xx57jF/+8pdYLJ6vzJKSErVRA+644w5uuukm3nrrLaDl30UB1QclF84wDAzDMLsM0508eZJ77rmHX/3qV4SFhdV5rbO3kdVqZfny5ZSVlXHnnXeyd+9es0vyKx9++CFRUVEMHTqU9evXm12O33rjjTew2+0cO3aM22+/nQEDBtR5/WI+ZwETUHa7naKiIu+0w+HAbrebWJH/io6O5vDhw8TGxnL48GGioqLMLslUZ86c4Z577mHKlClcf/31gNqoIREREYwePZqtW7dSVlZGdXU1NpuNoqKiTv1Z+/LLL1m7di05OTlUVlZSXl7Oo48+qjY6T+37j46OZsKECeTm5rb4cxYwh/hSU1PJz8+noKCAqqoqsrKySE9PN7ssv5Sens57770HwHvvvcf48ePNLchEbrebX//61wwYMIDbb7/dO19t5FFcXExZWRkAFRUVfPbZZwwcOJDRo0ezevVqAN59991O/Vm7//77ycnJYe3atfz+979nzJgxPPvss2qjc5w6dYry8nLv808//ZTk5OQWf84C6lJH69at47HHHvMO95w7d67ZJZnuvvvuY8OGDZSUlBAdHc3dd9/Nddddx7333suhQ4fo1asXzz33HN27dze7VFNs2rSJWbNmMWjQIG//wX333cewYcPURsA333zDQw89hNPpxO12M2nSJO666y4KCgr4xS9+QWlpKSkpKTzzzDOdegh1rfXr1/OnP/3JO8xcbeRRUFDAnXfeCXj6NG+44Qbmzp1LSUlJiz5nARVQIiLSeQTMIT4REelcFFAiIuKXFFAiIuKXFFAiIuKXFFAiIuKXFFAiIuKXFFAiIuKX/j9cVQruEVO9ywAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model = keras.Sequential([\n",
    "    layers.Dense(1, input_shape=input_shape),\n",
    "])\n",
    "model.compile(\n",
    "    optimizer='adam',\n",
    "    loss='mae',\n",
    ")\n",
    "history = model.fit(\n",
    "    X_train, y_train,\n",
    "    validation_data=(X_valid, y_valid),\n",
    "    batch_size=512,\n",
    "    epochs=50,\n",
    "    verbose=0, # suppress output since we'll plot the curves\n",
    ")\n",
    "history_df = pd.DataFrame(history.history)\n",
    "history_df.loc[0:, ['loss', 'val_loss']].plot()\n",
    "print(\"Minimum Validation Loss: {:0.4f}\".format(history_df['val_loss'].min()));"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "37bcf3ac",
   "metadata": {
    "papermill": {
     "duration": 0.01521,
     "end_time": "2022-02-05T23:15:59.856229",
     "exception": false,
     "start_time": "2022-02-05T23:15:59.841019",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "It's not uncommon for the curves to follow a \"hockey stick\" pattern like you see here. This makes the final part of training hard to see, so let's start at epoch 10 instead:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "96e80b7d",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-05T23:15:59.916936Z",
     "iopub.status.busy": "2022-02-05T23:15:59.907285Z",
     "iopub.status.idle": "2022-02-05T23:16:00.170947Z",
     "shell.execute_reply": "2022-02-05T23:16:00.170530Z",
     "shell.execute_reply.started": "2022-02-05T23:10:20.556773Z"
    },
    "papermill": {
     "duration": 0.299265,
     "end_time": "2022-02-05T23:16:00.171093",
     "exception": false,
     "start_time": "2022-02-05T23:15:59.871828",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Minimum Validation Loss: 0.1999\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAA+lElEQVR4nO3de3wU9b3/8dfes8km2VzIBpLINUAUBBWVn0HyMxRQLgeRYKtUbSsHD9AjLbWt2lOOh1atYKscOT+QYqE9qK2AmJKI2OIFIgoasYESUFA0XLIBkmyum73N749JloQkJJBsdpP9PB+Pfczu7OzsZyabfe935jszGkVRFIQQQogQow12AUIIIURbJKCEEEKEJAkoIYQQIUkCSgghREiSgBJCCBGS9MEu4FIKCwuDXYIQQogecMMNN7Qa16mA2r17N08++SQ+n4+5c+eyYMGCFs9v2LCBzZs3o9PpiI+P56mnniIlJQWAbdu2sWbNGgAWLlzI7NmzAcjLy+PFF18EICkpiZUrVxIfH9+poi9XcXExGRkZXZ5PT5F6A0vqDSypN7D6Yr3tNUY63MTn9XpZvnw569evJz8/n7y8PI4dO9ZimoyMDLZu3cr27duZOnUqK1euBKCyspLVq1fz2muvsXnzZlavXo3D4cDj8fDkk0/yxz/+ke3btzNixAhefvnlzi6vEEKIMNBhQBUVFTFw4EDS0tIwGo1Mnz6dXbt2tZhm/PjxmM1mAMaOHUtpaSkABQUFZGZmYrVaiY2NJTMzkz179qAoCoqiUF9fj6Io1NTUkJSUFIDFE0II0Vt1GFB2u53k5GT/Y5vNht1ub3f6LVu2MHHixEu+1mAw8MQTTzBz5kxuvfVWjh8/Tk5OTleWQwghRB/TrZ0kcnNzOXToEJs2bbrkdG63m1dffZU33niDtLQ0fvWrX/Hiiy+yaNGiVtMWFxd3uS6n09kt8+kpUm9gSb2BJfUGVjjV22FA2Ww2/yY7UFtFNput1XR79+5l7dq1bNq0CaPR6H/t/v37W7z2pptu8hd71VVXAXDHHXewbt26Nt+/O3YG9sWdiqFE6g0sqTewpN7ACmgnidGjR3PixAlKSkpwuVzk5+eTnZ3dYprDhw+zbNky1qxZQ0JCgn/8hAkTKCgowOFw4HA4KCgoYMKECdhsNo4fP055eTkAH3zwAUOHDu1wQYUQQoSPDltQer2eZcuWMX/+fLxeL3PmzCE9PZ1Vq1YxatQoJk2axIoVK6irq2PJkiUA9O/fn7Vr12K1Wlm0aJF//9LixYuxWq3++/PmzUOv15OSksLTTz8duKUUQgjR63RqH1RWVhZZWVktxjWFEcDGjRvbfW1OTk6bHSDuuece7rnnnk6WKYQQItzIqY6EEKKXu+6664JdQkD07YCqr4A/zUJffzbYlQghhLhMfTugqkvhy/eIPPtZsCsRQoiAUxSFZ555hhkzZjBz5kzefPNNAMrKypg3bx6zZs1ixowZfPLJJ3i9Xh599FH/tJfaVRMsIX2y2C6LUc8HqK8t7WBCIYTouq2FJ3ntk5Junefd49KYc0Nqp6Z9++23OXLkCLm5uVRUVJCTk8O4cePIy8tjwoQJLFy4EK/XS319PcXFxdjtdvLy8gCoqqrq1rq7Q99uQUXEQEQshrr2z3whhBB9RWFhIdOnT0en05GYmMiNN97IwYMHGT16NK+//jovvPACn3/+ORaLhbS0NEpKSvjVr37F7t27sVgswS6/lb7dggKITcNQJy0oIUTgzbkhtdOtnZ504403smnTJt5//30effRRvv/973PnnXeSm5tLQUEBf/7zn9mxY0fIHe7Tt1tQALGp0oISQoSFcePGsWPHDrxeL+Xl5XzyySdce+21nDp1isTERO6++27mzp3LP//5T8rLy1EUhalTp/KjH/2Iw4cPB7v8Vvp+CyomBcOJvcGuQgghAm7y5MkcOHCAWbNmodFo+OlPf0q/fv3Ytm0bL730Enq9nsjISJ555hnKysp47LHH8Pl8ACxdujTI1bfW9wMqNhWdqwoaasAUettYhRCiqw4cOACARqPh5z//OT//+c9bPD979mz/xWKb27ZtW4/Ud6XCYBNfmjqsOhXcOoQQQlyWMAioxh2Wju7t+imEECKwwiigTga3DiGEEJelTwdURa2Lb796AgWtBJQQQvQyfTqgTlXWs+/rKmqNiRJQQgjRy/TpgLJGGgBwGPpJQAkhRC/TxwNKvfR8hb6fdJIQQohepk8HVJRRh16r4Zw2ERynoPGANCGECGeXun7UyZMnmTFjRg9W074+HVAajQZrpJFSEsHnhlq5LpQQQvQWff5MEtZIA6eUBPWB4yRE24JbkBCi7/rsVTiwqXvned13Yew9l5zk2WefpX///sybNw+AF154AZ1Ox759+6iqqsLj8bBkyRK+9a1vXdZbNzQ08MQTT3Do0CF0Oh2PPvoo48eP54svvuCxxx7D7Xbj8/l44YUXSEpK4kc/+hGlpaX4fD4WLVrEtGnTrnixIRwCymzgm7p49YGjBFJvCG5BQgjRzaZNm8ZTTz3lD6gdO3bw0ksvcf/992OxWCgvL+fb3/42kyZNQqPRdHq+L7/8MgDbt2/n+PHjPPjgg+zcuZM///nP3H///fzLv/wLLpcLn8/H+++/T1JSEuvWrQOgurq6y8vV9wMq0sBXVc1aUEIIEShj7+mwtRMIV199NefPn8dut1NRUUFMTAyJiYk8/fTTfPzxx2i1Wux2O+fOnaNfv36dnm9hYSHf/e53ARg6dCgDBgzgq6++YuzYsaxdu5bS0lKmTJnCoEGDGD58OM888wwrV67ktttuY9y4cV1erj69Dwog1myktMEMRosElBCiz7r99tvZuXMnb775JtOmTWP79u2Ul5fz+uuvk5ubS2JiIg0NDd3yXjNnzmTNmjVERESwYMECPvzwQwYPHszrr7/O8OHDef7551m9enWX36fPB5Q10kC1y6ee8ki6mgsh+qhp06bx5ptvsnPnTm6//Xaqq6tJSEjAYDDw0UcfcerU5Z8we9y4cWzfvh2Ar776ijNnzjBkyBBKSkpIS0vj/vvvZ9KkSRw9ehS73Y7ZbGbWrFk8+OCD3XJ9qb6/ic9soN6j4ItJQSstKCFEH5Wenk5tbS1JSUkkJSUxc+ZMFi5cyMyZMxk1ahRDhgy57Hnee++9PPHEE8ycOROdTsfTTz+N0Whkx44d5ObmotfrSUxM5KGHHuLgwYOsWLECrVaLXq/niSee6PIy9f2AajybREPkAMxn/hHkaoQQInCaWjsA8fHx/OUvf2lzuqbrR7UlNTWVvLw8AEwmU5uXgV+wYAELFixoMe7WW2/l1ltvvZKy2xUGm/jUs0nUmpOh7hy464NckRBCiM4ImxZUldFGIkDVaUgYGtSahBAi2I4ePcrPfvazFuOMRiObN28OUkWt9f2AMqstqHK9jSGgdpSQgBJChLkRI0aQm5sb7DIuKQw28aktqDJNojpCOkoIIUSv0KmA2r17N1OnTmXy5Mn+o4Sb27BhA9OmTWPmzJk88MADLbozbtu2jSlTpjBlyhS2bdvmH+9yufjlL3/J1KlT/f33AyG2MaBKiQc0ElBCCNFLdLiJz+v1snz5cjZs2IDNZiMnJ4fs7GyGDRvmnyYjI4OtW7diNpt55ZVXWLlyJc8//zyVlZWsXr2arVu3otFouOuuu8jOziY2Npa1a9cSHx/Pzp078fl8VFZWBmQBo016tBood2rAYpNjoYQQopfosAVVVFTEwIEDSUtLw2g0Mn36dHbt2tVimvHjx2M2mwEYO3YspaWlABQUFJCZmYnVaiU2NpbMzEz27NkDwNatW3nooYfUIrRa4uPju3XBmmg0GixGLZX1rsaDdaUFJYQQvUGHLSi73U5ycrL/sc1mo6ioqN3pt2zZwsSJE9t9rd1up6qqCoBVq1axf/9+0tLSWLZsGYmJia3mV1xc3PmlaUe0Ucs3peepMsRiOvsFX3bDPAPJ6XR2y3L3FKk3sKTewJJ6A6sr9XZrL77c3FwOHTrEpk2XPt28x+OhtLSU6667jscee4wNGzb4TzJ4sYyMjC7XFRNxCp/eTExqBpzZS8bIkXAZZ/TtacXFxd2y3D1F6g0sqTewpN7A6ky9hYWFbY7vcBOfzWbzb7IDtVVks7W+ptLevXtZu3Yta9aswWg0XvK1cXFxmM1mpkyZAqgnOeyO8za1J9qku7CJz1MPdeUBey8hhBDdo8OAGj16NCdOnKCkpASXy0V+fj7Z2dktpjl8+DDLli1jzZo1JCQk+MdPmDCBgoICHA4HDoeDgoICJkyYgEaj4bbbbmPfvn0AfPjhhwwdGrhjk6KNWirr3GpAgXSUEEKIXqDDTXx6vZ5ly5Yxf/58vF4vc+bMIT09nVWrVjFq1CgmTZrEihUrqKurY8mSJQD079+ftWvXYrVaWbRoETk5OQAsXrwYq9UKwCOPPMLPfvYznnrqKeLj49s831N3sZi0OOqczQLqJAwYG7D3E0II0XWd2geVlZVFVlZWi3FNYQSwcePGdl+bk5PjD6jmUlJS/FdrDLRok47qBg9uSwoGkJ58QgjRC/T5M0mAuokPoEoTA/oI2cQnhBC9QHgElEkHQKXTI8dCCSFELxEWARVjUhezsk4O1hVCiN4iLALK4g8otwSUEEL0EmERUNHGxk18dW6ISYUaO3hcQa5KCCHEpYRHQDW1oOqbjoVSoPp0cIsSQghxSWERUFFGLRoNOJr2QYFs5hNCiBAXFgGl1WiINRsaW1Bp6kgJKCGECGlhEVAAVrOhsZNEijpCjoUSQoiQFj4BFWlUW1AGM0QmSgtKCCFCXBgFlEE9Dgqkq7kQQvQC4RNQTZv4QAJKCCF6gfAJqEhjsxZUmhpQihLcooQQQrQrbAIq1mygyunB61PUjhKuGnA6gl2WEEKIdoRNQFkjDQBU1bvlWCghhOgFwi6g5FgoIYToHcInoMxGoNkZzUGOhRJCiBAWPgHV1IKqc0NUEmgN0oISQogQFkYB1diCqneBVqt2lJCAEkKIkBU+AWVu1oKCC13NhRBChKSwCaiYVgGVClWngliREEKISwmbgNJpNcRE6HHUNw+o0+D1BLcwIYQQbQqbgIKLziYRkwKKF2pKg1uUEEKINoVZQDVeEwrkWCghhAhxYRZQxpb7oEACSgghQlR4BZS5+SU35MKFQggRysIroJpv4jNFQ4RVWlBCCBGiwiugzAYc9W58vsbLbMixUEIIEbI6FVC7d+9m6tSpTJ48mXXr1rV6fsOGDUybNo2ZM2fywAMPcOrUheOLtm3bxpQpU5gyZQrbtm1r9dp/+7d/Y8aMGV1YhM6LjTSiKFDtbOxaHpsKDjkWSgghQlGHAeX1elm+fDnr168nPz+fvLw8jh071mKajIwMtm7dyvbt25k6dSorV64EoLKyktWrV/Paa6+xefNmVq9ejcNx4RpMb7/9NlFRUd28SO3zn02ivvml32UflBBChKIOA6qoqIiBAweSlpaG0Whk+vTp7Nq1q8U048ePx2w2AzB27FhKS9VjiwoKCsjMzMRqtRIbG0tmZiZ79uwBoLa2lg0bNrBw4cLuXqZ2tThhLKgB5ayEhuoeq0EIIUTn6DuawG63k5yc7H9ss9koKipqd/otW7YwceLEdl9rt9sBWLVqFT/4wQ+IiIi45PsXFxd3VGKHnE4nxcXFVJY5ATh49DjGmkhiajSkAMcP7MEVO7jL79NdmurtLaTewJJ6A0vqDayu1NthQF2O3NxcDh06xKZNmy45XXFxMd988w2PP/44J09eupNCRkZGl+sqLi4mIyMDY2IN7DhNdGIyGRkpEOWAj2BoghHSu/4+3aWp3t5C6g0sqTewpN7A6ky9hYWFbY7vcBOfzWbzb7IDtVVks9laTbd3717Wrl3LmjVrMBqNl3ztgQMHOHToENnZ2dx7772cOHGC++67r6NSuqz1Gc3lwoVCCBGqOgyo0aNHc+LECUpKSnC5XOTn55Odnd1imsOHD7Ns2TLWrFlDQkKCf/yECRMoKCjA4XDgcDgoKChgwoQJ3HvvvRQUFPDOO+/wyiuvMGjQIP73f/+3+5fuIrEXB5QlGTQ66WouhBAhqMNNfHq9nmXLljF//ny8Xi9z5swhPT2dVatWMWrUKCZNmsSKFSuoq6tjyZIlAPTv35+1a9ditVpZtGgROTk5ACxevBir1RrQBbrksui0RJv0F3rx6fQQM0ACSgghQlCn9kFlZWWRlZXVYlxTGAFs3Lix3dfm5OT4A6otqamp5OXldaaMbhEbacDR1IKCxq7mElBCCBFqwupMEnDR6Y6g8bpQElBCCBFqwi+gzM2uCQUXzibh8wWvKCGEEK2EX0Bd3IKyXgU+N1R+HbyihBBCtBKeAdV8H1Tazerwmw+DU5AQQog2hV9ANW7i85/RvF8GmOPgxAfBLUwIIUQL4RdQkQZ8CtS4Gs9ortXCwEw4sSe4hQkhhGgh7AKq6WDdFl3NB2aq+6Cku7kQQoSMsAsoa6R6GqYW+6EGTVCHsplPCCFCRhgG1EXXhAKwXQMRsfB1QZCqEkIIcbHwC6iLz8cHoNXBVbfACQkoIYQIFeEXUE2b+JofCwUwKBPKv4SqM0GoSgghxMXCLqD8ZzSvdbV8omk/1NeyH0oIIUJB2AWUUa8lyqhr3YJKvhZMMbKZTwghQkTYBRSom/la7IOCxv1Q4yWghBAiRIRlQMWaDTjqXa2fGDQBzn8B1faeL0oIIUQLYRlQrc7H12Sg7IcSQohQEb4BdfE+KID+Y8BokYASQogQEKYB1cY+KFAvAZ92s+yHEkKIEBCeAdW4D0pRlNZPDpoAZ49A7bmeL0wIIYRfeAZUpAG3V6HW5W39pBwPJYQQISE8A8rcdMLYNnryDbgODJGymU8IIYIsLAMqNrKN8/E10Rka90NJC0oIIYIpLAOq6YSxjrZ68oF6Xr6yf0JdeQ9WJYQQornwDKi2rgnVnP94qL09VJEQQoiLhWlAtXFNqOZSrgd9hOyHEkKIIArLgIpt65pQzelNkHaTXMBQCCGCKCwDKsKgw2zQtd2Lr8nACVB6COoreq4wIYQQfmEZUHCJ8/E1GZQJKPDNRz1WkxBCiAvCNqBize2cj69JyjjQmWQ/lBBCBEmnAmr37t1MnTqVyZMns27dulbPb9iwgWnTpjFz5kweeOABTp065X9u27ZtTJkyhSlTprBt2zYA6uvrWbBgAbfffjvTp0/n2Wef7abF6TxrpAHHpVpQhghIvVECSgghgqTDgPJ6vSxfvpz169eTn59PXl4ex44dazFNRkYGW7duZfv27UydOpWVK1cCUFlZyerVq3nttdfYvHkzq1evxuFwAPCDH/yAt956i23btvHpp5/y/vvvB2Dx2mc1G9vvxddkUCaUFoHT0TNFCSGE8OswoIqKihg4cCBpaWkYjUamT5/Orl27Wkwzfvx4zGYzAGPHjqW0tBSAgoICMjMzsVqtxMbGkpmZyZ49ezCbzYwfPx4Ao9HI1Vdfjd3esxcJ7HAfFMDATFB88M2+nilKCCGEn76jCex2O8nJyf7HNpuNoqKidqffsmULEydObPe1FwdRVVUV7777Lg888ECb8ysuLu6oxA45nc5W8/HWV1FR6+Lw4cNoNJo2X6fxxDBca6Di0zco86Z1uY7OaqveUCb1BpbUG1hSb2B1pd4OA+py5ObmcujQITZt2tSp6T0eD0uXLuW+++4jLa3tAMjIyOhyXcXFxa3mM7TsOO5DDgYPG4HZqGv/xR+PI6H6CAndUEdntVVvKJN6A0vqDSypN7A6U29hYWGb4zvcxGez2fyb7EBtFdlstlbT7d27l7Vr17JmzRqMRmOnXvvLX/6SQYMG8b3vfa+jMrpd0/n4Ki51LBSom/lOfwYN1YEvSgghhF+HATV69GhOnDhBSUkJLpeL/Px8srOzW0xz+PBhli1bxpo1a0hISPCPnzBhAgUFBTgcDhwOBwUFBUyYoJ7n7rnnnqOmpobHH3+8mxepc6yXOqN5c4MyQfHKfighhOhhHW7i0+v1LFu2jPnz5+P1epkzZw7p6emsWrWKUaNGMWnSJFasWEFdXR1LliwBoH///qxduxar1cqiRYvIyckBYPHixVitVkpLS1m7di1Dhgxh9uzZAHz3u99l7ty5AVzUlmKbrgnVUU++tJtBq1dPe5T+rR6oTAghBHRyH1RWVhZZWVktxjWFEcDGjRvbfW1OTo4/oJokJydz9OjRyyiz+zW1oC55LBSAMQoGXA9f7emBqoQQQjQJ2zNJXDijeQcBBTByOpz6REJKCCF6UPgGlLmDa0I1d/NDEHsVvPUo+LwBrkwIIQSEcUCZjTpMem3H+6AADGaY8iuwH4LCjQGvTQghRBgHFDSeTaK2Ey0ogKtnwaBb4Z1fyyU4hBCiB4R3QHXmfHxNNBq4/TfgrIT3fhPQuoQQQoR5QMV25nx8zSWPghu+D/t/D2W951QjQgjRG4V1QFnNBhyd6cXXXPZ/gCla7TChKIEpTAghRJgH1OW2oAAi4+G2x+HL9+DomwGpSwghRNgH1GXsg2pu3IPQLwN2Pg5uZ/cXJoQQItwDyoDT7cPpvsxjm3R6uP1pqDgBH/1PQGoTQohwF94B1Xiw7mXvhwIYehuMnAG7fwtVZ7q5MiGEEOEdUJGdvORGe6b8Gnxu+PsT3VeUEEIIINwDytzJS260J34w/J8fQtGfoeTjbqxMCCFEWAdUbGevCXUpt/4ELMmw42fg83VTZUIIIcI6oKyRTfugrnATH4DJApP/C05/Cp9u7J7ChBBChHlAdXUTX5PRd8NVt0DejyH/EXDVdkN1QggR3sI6oCKNOgw6TeeuCXUpWi18dyvc/G/w8e9hTSZ8/WH3FCmEEGEqrANKo9GoB+t2tQUFYIyEO56B7+WD4oMNd8DOX4C7vuvzFkKIMBTWAQVN5+Prwj6oiw2aAAv3wrjvw4erYe2tcPKT7pu/EEKECQmoSAMVnb0mVGeZLDDjObhvm9qCemmyeqyUp6F730cIIfqwsA+oWLOx6/ug2jM0GxbthbH3QsFz8GIWnCiQs6ALIUQnhH1AWSMNOK70TBKdERELs/4H7t2sXuxw43T4w+3w+dsSVEIIcQkSUGZD4FpQzQ2fAv/+KdyxEhwn4ZW56v6pQ6+D7zJPViuEEGFAAirSQJ3LS4OnB0LCGAk3L4CHD8Cs/wceJ2z5PvzPTXBgE3gC2JITQoheJuwDKsFiAuBsdQ92YNAb4bp5sHgfzP0jGCIhdzH893Xw0VoMtadl858QIuzpg11AsA3tZwHgi7IaUuMie/bNtTq45k64ehYc2wV7fgtv/ZxhAO8kwIDrIeX6C0NLUs/WJ4QQQRT2ATXc1hhQ9mpuGxGkANBoIP1b6q30IGc+/iv9fafh1AE4vks98BcgJhVSroOUG2DYt8A2Sn2tEEL0QWEfUNZII/2iTXxurwl2Kark0VQO09M/I0N97KqFM0VwqlA9Ie2pT6F4u3pcVUwKpE+G4bfD4Cx1H5cQQvQRnQqo3bt38+STT+Lz+Zg7dy4LFixo8fyGDRvYvHkzOp2O+Ph4nnrqKVJSUgDYtm0ba9asAWDhwoXMnj0bgEOHDvHYY4/hdDrJysriF7/4BZogtQaG2yx8Ya8Oynt3yBgFA/+PemtSbYcv3oYvdsLBLVC4EXQmGDwRhk+F9CkQNzBwNTVUq0F56hM4U0Q/JRbM34a0m0BnCNz7CtEZigL1FVB1CmrPgdetXljU6wafRx16XY3jPMSWV0N0HfQbDqboYFcvmukwoLxeL8uXL2fDhg3YbDZycnLIzs5m2LBh/mkyMjLYunUrZrOZV155hZUrV/L8889TWVnJ6tWr2bp1KxqNhrvuuovs7GxiY2N54okn+NWvfsWYMWP413/9V3bv3k1WVlZAF7Y96UnRvPZJCT6fglbbCzaZRdvg+vvUm6cBvt4Ln+9UA+vNR9RprAPVFla0DSw2df+VJfnC/ehkiLCqgXKpHwY+L5w9qobRyY/V0zaVFQONnThi00ioOg3FfwRjtBqSw7Jh6CT1go7hxuOC6jNQdVrdNJt6o9opprerKYMTe+D8l82+3N3Nvvxd4PWAz03/WheUXg3R/SFmwIVhZEL3bZKutsOZz8BRoq5rxyk1kKpOqzdP58+BOQBg/6/VBzGp0G8E9BsJSSPVYeJwMFu7p25xWToMqKKiIgYOHEhaWhoA06dPZ9euXS0Cavz48f77Y8eO5a9//SsABQUFZGZmYrVaAcjMzGTPnj3cdNNN1NTUMHbsWADuvPNOdu3aFbSAGm6Lps7l5VRlPWnxvWwzmd4EQ29Tb3f8Bs4dg8/fUgOlpgxKD0LNLmioan8eOqPaAtM3DnUGdb46I1R8Da7G1mWEFVLHQca/qF+8KddDZDyf/2M/Iwyl6v6yY+/A0Xx1+vihMGySuvkxOhmMFvUXqsmi3tfqWtfirofq0sbbmZZDb0PrsG0K4Kh+bc+vO7id6kHW9ZUXhvXlF74Mq05DdeOw9mzL1xqj1b9NU8u2Ozq6OKug7LD6t7UfgrOfq1+g8UMgYai63hOGQvQA9Uz7V6KuXD3ryYk98NUeOFvc8nmtofFzo1eHWoN6X2vAUlcJX+Xh/xHTRGe8EFYJQyHp6sYguFr9fLQXXooC54/BNx/CNx+pw/Ivm9Wib5xvCvQfAyPugNhU9X2i+jXWaWis0aBO3+zxsUMfMyzGA2ePQNkRdfj1B+phIP7aTer/hN4E+ojG/4+LHputEJkIUY23yIuGEVY1yN116s1VB+5a9TPvqms2vvbCzV0Hrhr1eVctuGu5qqYKPk0Eg1ntAdxqeNE4Y2Sz8ZGNj81gir38z4fPBzWlUHECyr9S/8Zj5wVsX3iHAWW320lOTvY/ttlsFBUVtTv9li1bmDhxYruvtdvtrcYnJydjt9vbnF9xcXGb4y+H0+m85HyMTvWDuOuTw9ycFtXl9+uqjurtUNwk9daMxuNE7zx/4VZ/Hq27Bo3PjcbrUoc+N9pm9zVeF+6rRlCfMApnwjW4LGkXPoge4Gs7YMfpM1CsSYdh6TD0IYzV3xBV+hGW0n1EFv4J7f51bZbp05vx6SPx6iNBq0fnPI/e1TpIfVojHnMiitaA3lmOzt16c6yi0eI1WfGY4vAZo/EaovEao/E1DpvfNzXUc/qrPLTuWrTuWnSeWrTuOrTuGnSN47SuanTuanSuKrTe9g9B8Bhj8UQm4TH3w23LxBOZhNvcD09kEhpvA5YzH2I5UYChWP3RVh9/NTUDJlDT/xaccSPa/sf2edF66tTa3DWYyr/k7KHfE1H5BabKYxhrT/sn9RpjaIgZjLayFOMXf0fru3AsnU9nwmVJxWVJxW1JxWeIxKczoWiNKDoTPp06VO+b0HrqiCz7lKiyTzFVfoEGBZ8ugrp+Y6i7dhG1STfgtA5Xv+Av8YXkdDqJMOrVz1ndWQz1Zejrz6KvP4uh/iz6ujJMxW+iP7Cp5XLEDqEhZggNsUNwWVIwOb4k8lwR5nP/QN9Qqa5vk5X6xDHUjZlOfcIo3JYUPKa49n+c1LX6izXeLoSP05BEsRIBiUMgcZr/b2CoK8VU9RVGx1foXA60XjcanwuNtwGN14XW51bvO51oPQ60ZcfQN1S2+fm8EgoaFH2E+n+iMzf+v5jVLZjnvkHrdaLxONF6G/z3NRf/KLjU/DVavMZYPBHxeE1xeCLiWgy9xhj09Wcx1pzCUHuqcXimxf+DxxTHMf01KPqIdt+nK99n3dpJIjc3l0OHDrFp06aOJ+6kjKbOAl1QXFx8yfkMGOSGHaepN8aRkTG0y+/XVR3VG2pa13s1cLt619OgdvKor1BbYg016j4sVw3ahmq0DdXoXTXqL0uLTf0lHd2/2bA/WnMcxuZfiG4n1NjVFmJNKdTY0dSUoa8uRV93HpwOtZVTcUK9f6nWI4De3Niya7xZYsB8lfqL12xtHMY1u9/4OLo/eoO5g3+ih9QWQOlB+Hwn5i92Yj70e/odWqcuX+Jw9RdyQ7V6c1apv6ovptFCwjAYdLPaezN5NNiuQReTQmTTuvH51M1c5cfh/HG05V8Scf44EeePQelHagu0IzqTui/xurkweCLaAddj0RuxdPxKv05/fmvPqZuLy4rRnS0msqyYyJO74PjrF6aJHwIZ0+Gq8XDV/0GfMIxojYbu3FPUfr2jrmyGXjfUnVeXr+5c4/C8+pnUG8EQdaEV47/fvHUTBcYoNAYzGo2m1cGq7darKOr/W1NLzF3frKXW7HHjOE19Ofras+hrz6kt/5rjYD/X+v/FEKVurh8wCuJmqPfjBkPcIPTWqxjZwX7nznweCgsL2xzfYUDZbDZKS0v9j+12OzabrdV0e/fuZe3atWzatAmj0eh/7f79+1u89qabbmo1z9LS0jbn2VNizQaSYyJCt6NEb6Y3QdqN3TtPQ4TaCaSzHUG8HvWfrr4CnA6+/PokQzKuBVOMuqkx0PuINBrof616y/op1JyFY39XN8VWnVZDz3pVY0DGtAxLUzRfVfgYfNMdHffS1GrBmqbehvzf1s/7vOpmK7dTHXqc6peWp0HdZ6M1qJvIDO3/Gu5WUYkw+Fb11kRR1E265V9CQrq6Cbe30Rkaf2Aldzxtd9Jo1L+dIQKIv/L5uJ1qsNZXXNh8HqQObB0G1OjRozlx4gQlJSXYbDby8/P57W9/22Kaw4cPs2zZMtavX09CQoJ//IQJE/jd736Hw+EA1H1SS5cuxWq1YrFY+OyzzxgzZgxvvPEG9913Xzcv2uVJt1n4vEwCqk/S6SEyXr0BDVVmiBsUvHos/WDsPeqtE5zFxd1zCIFWp/YKNQZ/M3a7NBp131HMgGBXEr4MEeo+vNjUYFfScUDp9XqWLVvG/Pnz8Xq9zJkzh/T0dFatWsWoUaOYNGkSK1asoK6ujiVLlgDQv39/1q5di9VqZdGiReTk5ACwePFif4eJ//zP//R3M584caJ/v1WwDLdF8/K+r3tPTz4hhOjjOrUPKisrq1UPu6YwAti4cWO7r83JyfEHVHOjR48mLy+vk2UG3nCbBafbR0lFHQMTQvgXphBChImwP1lsk3Sbuts1ZM4oIYQQYU4CqlF6ktpP6XPpKCGEECFBAqpRdISBAbHSk08IIUKFBFQz6bZojsomPiGECAkSUM2MSI7m+NkavD65WKAQQgSbBFQz6UkWXB4fX59v40h+IYQQPUoCqpnh0pNPCCFChgRUM8OSLlxdVwghRHBJQDUTZdKTGmfm8zJpQQkhRLBJQF1kuC1aWlBCCBECJKAukm6z8OXZWtxeX7BLEUKIsCYBdZHhSdG4vNKTTwghgk0C6iLSk08IIUKDBNRFhiVZ0GjknHxCCBFsElAXMRt1XBUfyRfSghJCiKCSgGpDelK0tKCEECLIJKDaMNxm4atztbg80pNPCCGCRQKqDcNt0Xh8CiekJ58QQgSNBFQb0m1y8UIhhAg2Cag2DO1nQauBz0sloIQQIlgkoNoQYdAxMCFKjoUSQoggkoBqR3qShc/LpAUlhBDBIgHVjuG2aL4+X0eDxxvsUoQQIixJQLVjeHI0Xp/Cl2elJ58QQgSDBFQ7hktPPiGECCoJqHYMToxCp9XIKY+EECJIJKDaYdLrGJQQKS0oIYQIEgmoSxhui+YLufy7EEIERacCavfu3UydOpXJkyezbt26Vs9//PHHzJ49m6uvvpq33nqrxXMrV65kxowZzJgxgzfffNM//sMPP2T27NnMmjWLe+65h6+//rqLi9L90m3RnDhfi9MtPfmEEKKndRhQXq+X5cuXs379evLz88nLy+PYsWMtpunfvz9PP/00M2bMaDH+vffe4/Dhw7zxxhu89tprvPTSS9TUqC2SJ554gmeffZbc3FxmzJjBmjVrunGxusdwmwVFgWPSihJCiB7XYUAVFRUxcOBA0tLSMBqNTJ8+nV27drWYJjU1lZEjR6LVtpzdsWPHGDduHHq9nsjISEaMGMHu3bv9zzeFVU1NDUlJSd2xPN2q6eq6X8gBu0II0eM6DCi73U5ycrL/sc1mw263d2rmI0eOZM+ePdTX11NeXs6+ffsoLS0F4Mknn2TBggVMnDiR3NxcFixYcIWLEDiDEqLQazVyyiMhhAgCfSBnPmHCBA4ePMh3vvMd4uPjGTt2rL+VtXHjRtatW8eYMWNYv349Tz/9NE8++WSreRQXF3e5DqfTecXzGRCt59NjZyguVrpcR2d1pd5gkHoDS+oNLKk3sLpSb4cBZbPZ/K0eUFtUNput02+wcOFCFi5cCMBPfvITBg8eTHl5OUeOHGHMmDEATJs2jfnz57f5+oyMjE6/V3uKi4uveD6jD9Rz8KSjW+rorK7UGwxSb2BJvYEl9QZWZ+otLCxsc3yHm/hGjx7NiRMnKCkpweVykZ+fT3Z2dqcK83q9VFRUAHDkyBGOHj1KZmYmMTExVFdX89VXXwHwwQcfMHTo0E7Ns6cNT4qmpKKOepf05BNCiJ7UYQtKr9ezbNky5s+fj9frZc6cOaSnp7Nq1SpGjRrFpEmTKCoq4oc//CFVVVW8++67vPDCC+Tn5+PxeJg3bx4AFouFlStXoterb/nrX/+ahx9+GI1GQ2xsLE899VRgl/QKNe/JNzo1NtjlCCFE2OjUPqisrCyysrJajFuyZIn//rXXXtuid14Tk8nU4tin5iZPnszkyZMvp9agSG/syXfUXi0BJYQQPUjOJNGBQQmRGHVavpBTHgkhRI+SgOqAXqdlSL8oOSefEEL0MAmoThhui+bwmSoUpee6mgshRLiTgOqErOH9sFc1sP+r8mCXIoQQYUMCqhPuGJ2MxaRnc+HJYJcihBBhQwKqEyKNeqaP7s+bB89Q2+AJdjlCCBEWJKA6ae64VOpcXvIPngl2KUIIERYkoDrphoFxDEmMYssnsplPCCF6ggRUJ2k0GubckMr+E+WcOFcb7HKEEKLPk4C6DHOuT0WrgS3SWUIIIQJOAuoyJMdGcGt6P7Z+ehKvT46JEkKIQJKAukxzx6VyxuHkg2Pngl2KEEL0aRJQl+lbGTZizQY5JkoIIQJMAuoyRRh0zBo7gJ3/LMVR5w52OUII0WdJQF2BuTek4fL4+GvR6WCXIoQQfZYE1BUYlRLDyORotnxSEuxShBCiz5KAugIajYacG1L5x0mHXIZDCCECRALqCt15XQp6rYbN0ooSQoiAkIC6QokWE7eNTGLbgVO4vb5glyOEEH2OBFQXzL0hlXM1Lt4/ejbYpQghRJ8jAdUFt41MItFiZHOhbOYTQojuJgHVBQadljvHprCruIzzNQ3BLkcIIfoUCagumjsuDY9P4Y3P5JgoIYToThJQXTQiOZprU2PZ/EkJiiInkBVCiO4iAdUN5t6QypHSav55uirYpQghRJ8hAdUN/mVMCka9lv/98OtglyKEEH2GBFQ3iI00cO9NV/GXT0r43w9PBLscIYToE/TBLqCv+I/pGZSU17Hsr/8kwWJi2uj+wS5JCCF6NWlBdRO9Tsvqe6/n+qvi+NGfP2PvcbmgoRBCdEWnAmr37t1MnTqVyZMns27dulbPf/zxx8yePZurr76at956q8VzK1euZMaMGcyYMYM333zTP15RFJ577jmmTp3KHXfcwZ/+9KcuLkrwmY06XnpgHAMTInnoT4X887Qj2CUJIUSv1WFAeb1eli9fzvr168nPzycvL49jx461mKZ///48/fTTzJgxo8X49957j8OHD/PGG2/w2muv8dJLL1FTUwPA66+/zpkzZ9ixYwc7duxg+vTp3bhYwWONNPKnB28iOkLP9zZ8zDfn64JdkhBC9EodBlRRUREDBw4kLS0No9HI9OnT2bVrV4tpUlNTGTlyJFpty9kdO3aMcePGodfriYyMZMSIEezevRuAV199lcWLF/tfk5CQ0F3LFHT9Y8386cGbcHt93P+HfZyTs0wIIcRl67CThN1uJzk52f/YZrNRVFTUqZmPHDmS1atX84Mf/ID6+nr27dvHsGHDACgpKeHNN9/kb3/7G/Hx8fzHf/wHgwYNajWP4uLiTi5K+5xOZ7fM53Ity+rHY2+f4Z41u/nN1AFEGjq3yy9Y9V4pqTewpN7AknoDqyv1BrQX34QJEzh48CDf+c53iI+PZ+zYsf4Wk8vlwmQy8frrr/P222/z+OOP88orr7SaR0ZGRpfrKC4u7pb5XK6MDLDaBvCvfyrkuf01/OF7N2LUdxxSwar3Skm9gSX1BpbUG1idqbewsLDN8R1+W9psNkpLS/2P7XY7Nput08UtXLiQ3NxcNmzYAMDgwYP98508eTIAkydP5ujRo52eZ2+SPdLGM3OupeDYOX6y+R/4fHI6JCGE6IwOA2r06NGcOHGCkpISXC4X+fn5ZGdnd2rmXq+XiooKAI4cOcLRo0fJzMwE4Fvf+hb79u0DYP/+/W1u3usrcm5I5dE7RrL9H6eZt34f+748H+yShBAi5HW4iU+v17Ns2TLmz5+P1+tlzpw5pKens2rVKkaNGsWkSZMoKirihz/8IVVVVbz77ru88MIL5Ofn4/F4mDdvHgAWi4WVK1ei16tvuWDBAh555BH++Mc/EhkZyZNPPhnYJQ2yhyYOwWzQ8cI7X/DtdR9x0+B4Hs5OJ3NYAhqNJtjlCSFEyOnUPqisrCyysrJajFuyZIn//rXXXuvvndecyWRqcexTczExMW0eU9VXaTQaHrhlEHePS+PPH3/Di+9/yXdf2sfYNCsPTxrGbSOSJKiEEKIZOZNEDzMbdXw/czDv/+z/8uTsUZytbuAHGz9hxgsFvHXojOyjEkKIRnIuviAx6XXMu3kgd49LY9uBU/y/d4/xb5s+ZbjNwvVJeiZjZ2yalQSLKdilCiFEUEhABZlBp+XucWncdV0K+QfP8IcPTvDaoUr+fPATAAYlRHLdVXFcd5WV66+KY0RyNAadNHyFEH2fBFSI0Ou0zBqbwqyxKRw4+E9cUckcKKnkwDcVFBw7x7YDpwCIMGi5NsXK9QPjGDcwjusHxhEfZQxy9UKIK1Hb4OGovZqjpdUcOVPFuRoXQ/tFMSI5hpH9oxmUEIVOG777piWgQlCEXst1QxK4eYh6+idFUThVWc+Bbyr59JsKPv2mkvV7vmTt++r+qiGJUf7AumFgHEP7WdCG8YdagMfr43yti7KqBsqqnVTUuUmIMpIUYyI5JoK4SGOv/Iz4fApnaxo4WVHP2eoGtBrQ6zTotFr0Wg06rabZUItRryXGrCc6wkCUUddtHZEURaGizk1ZtZOz1Q2cq2lAgwaDTn1Po16LsfG+qfGxT1E4XlbL0dIqikvVUPqm/MK5OqOMOhKjTew4dIamXdEmvZZ0m4WRyTGMTI5mRHI03ho3KU430SZ9l5ZHURTO1bg4VVnP6cp6TlXUc6qy8VZRT1l1A3GRBgZYzQywmkmxRpASZ2ZArPo4OTYi4FtzJKB6AY1GQ2pcJKlxkcwcMwAAp9tL0UkHhV9XUPh1Be8cKWNL4UkAYiL0jEyOwajXom32D6vTaNDpGh9rNGj9Q/U9dBoNWg1otRq0jfe9PnB7fbi9PlweH65m991eRX3srCfx41oijToijToiDLrG+3oiDDpMei21DR6qnG6q6puGbqqcnsahmwaPT/0CjY4gKcZEUrTJf79f4/0Ig5bKOjfltS4q6lzqsNZFeZ2Lilp1fIPH619nABr/OmwcokHjcTL4sJtEi4lEi5HEaFPjfRP9LCZizHoUBercXupcHuoavNS51Pu1Li/1Lg8NHh8RBh1RRj2RpsahUUeUSR2a9No2vzwURUFRwKso+BSFaqeHUoeTUoeTM1VO7A4nZxxOSqvqKXU4sVc14PF6iTGfwmLSY4nQE2VUhxaTnqjG93bUuymrbsBe5aSsuoHzNQ1cqr+NQachKTqC5NgIbDEmbDERJFpMmA06TAYtJr26DCa9FpPhwn2PT+FcdQPnal3qsKbp5lKH1Q043V502hNoNPg/R1qNRn3c+NmKMumIizQSazZgjTQSF2nA2njfGmkgOsLAuZoG9Yuz8QvztENdJ27vlXUk0mk1REfoiY7QExNhICbCQHSEHrezlriDDf7/E61G/R9p+t/RajVUOz2UVTVwtlpdv+dqGq64Dq0GBidGMTollrk3pDIiOZqM/jGkWM1otRqcbi/Hymo40tiqOmqv5v3Pz/r/vwHYWoJWAzFmdTlizQZizHp1GGFAq9XQ4Pbh9HhpcHtxun043V4aPOqw3u3lbHUDDR5fi9osJj0pVjMpcWauTY2los7F6Uonh045OF/rajGtRgMjk2PYtugWIgy6K1oXHZGA6qUiDDpuGhzPTYPjAfWL76tztRR+XcGn31Rw/Gwt9W4vHp+C1+fD66NxqOD1KXh8Cj6fgq/xy1JRGu/71C9OpfG+TqvBqNdi0DX7dajT+u/rtRrqPT5Kyuuod6tf5E6Xlzq3F+9F35A6rYaYCL3/nyrGrCcp2kJMhAGDXsP5Ghdl1Q18+k0FZVWt/3naotFAXOMXXFykkSiTGi4ACkrjusE/zqconK318OWRMsprXa1qBNBrNXi62JtSp9VgavzV7FNoXNfKJUOj6XVJ0WpgDLdFc2t6PxyVFZiiYqhu8FDbeCspr6PW5aG2wUtNg4eYCEPj60yMGhCLLcZEv5gI/7ysZgMVdS7sVWoYllY1UFblpLTKyZHSanZ/fo6aBs9lLWPTuk+0GEm0mBiTaiXRYqK2qoL4hAT/56jpc+Zr/Jx5FYUap4fKejcVdW5KyuuoqFN/qCgXrR+tBpJj1F/u118V1/hLXr31i1Y7EDV9ntWh+hn3eNVxDR4v1Y0/hKqdF34cNd3/+nwdjlonunIvPuXC/4VXUfB61aHHpxBt0tMvWv2xNCwpWv3hZDH5h4nRJrQajfojzuPD5VXD4MJjHz5F3doxLMlyyS/0CIOOUSmxjEqJbTH+fE0DR0ur2X/4OJa4fjjq3Tjq1eVpum+vasBR70ZRFPVHhkFLRLNhdMSFH439ok2kNLWO4sykWiOJMbffKnO6vZyurOd0pdP/w0Gr0WAMYCtKAqqP0Gg0DOlnYUg/C3PHpfXoe7d1ri1FUVtXTpePBo/X37Lo7CYJRVGocnrUX6xVDZRVN9Dg8RIXaSQ+ykhclJH4SCMxZsNlb6NvqtfnU6ioc1349V/TwNnqBsprXRh0WqJMaiuwqTWotpDU+0a9lnqXGsi1ja0sdai2supcHhrcvhatUZ1W06qlGmXUkRyrbi7pH6u2Yi5enu4699ogoi75fNMv7AaPlwa3z3/f6W4c5/Gh1WhItBjpZzERH2VE38aXk1rvyMuuz+tTqKp3U1HnotrpIcFiJDkmos336E695dx2CRYTtwwzEec+S0bGkB5//wiDzv8d01MkoERAaDSaxs1EOsBwRa+PNaubLoYlRXd/gagBkWAxkWAxMYLAvEdvEmHQNf6yv/y/V3fQaTXENf74EALkQF0hhBAhSgJKCCFESJKAEkIIEZIkoIQQQoQkCSghhBAhSQJKCCFESJKAEkIIEZIkoIQQQoQkCSghhBAhSaMoF5/9KnQUFhYGuwQhhBA94IYbbmg1LqQDSgghRPiSTXxCCCFCkgSUEEKIkCQBJYQQIiT1qcttPPbYY7z33nskJCSQl5cHQGVlJT/+8Y85deoUKSkpPP/888TGxnYwp57RVr0vvPACr732GvHx6oUIly5dSlZWVjDL9Dtz5gw/+9nPOH/+PBqNhrvvvpsHHnggZNdxe/WG6jpuaGhg3rx5uFwuvF4vU6dO5eGHH6akpISlS5dSWVnJNddcw4oVKzAag39JivbqffTRR9m/fz/R0eolTH7zm9+E1PWWvF4vc+bMwWaz8eKLL4bs+m1ycb2hvn6zs7OJiopCq9Wi0+l4/fXXr/w7QulD9u/frxw6dEiZPn26f9wzzzyjvPjii4qiKMqLL76orFixIljltdJWvf/93/+trF+/PohVtc9utyuHDh1SFEVRqqurlSlTpihffPFFyK7j9uoN1XXs8/mUmpoaRVEUxeVyKTk5OcqBAweUhx9+WMnLy1MURVF++ctfKi+//HIwy/Rrr96f//znyo4dO4JcXfv+8Ic/KEuXLlUWLFigKIoSsuu3ycX1hvr6ve2225Tz58+3GHel3xF9ahPfjTfe2CqVd+3axZ133gnAnXfeyd///vcgVNa2tuoNZUlJSVxzzTUAWCwWhgwZgt1uD9l13F69oUqj0RAVpV711uPx4PF40Gg0fPTRR0ydOhWA2bNns2vXrmCW6ddevaGstLSU9957j5ycHEC9cnOorl9oXW9vdaXfEX0qoNpy/vx5kpKSAOjXrx/nz58PckUde/nll5k5cyaPPfYYDocj2OW06eTJkxQXFzNmzJhesY6b1wuhu469Xi+zZs3illtu4ZZbbiEtLY2YmBj0enVrfHJyckiF7MX1Nq3f5557jpkzZ/LUU0/hcrmCXOUFTz31FD/96U/RatWvvoqKipBevxfX2yRU12+TBx98kLvuuou//OUvwJV/D/f5gGpOo9GE/C+8e+65h7/97W/k5uaSlJTEb37zm2CX1EptbS0PP/wwjz/+OBaLpcVzobiOL643lNexTqcjNzeX999/n6KiIr788stgl3RJF9f7+eefs3TpUt566y22bt2Kw+Fg3bp1wS4TgHfffZf4+HhGjRoV7FI6pb16Q3X9Nnn11VfZtm0bv//973n55Zf5+OOPWzx/Od8RfT6gEhISKCsrA6CsrMy/YzxUJSYmotPp0Gq1zJ07l4MHDwa7pBbcbjcPP/wwM2fOZMqUKUBor+O26g31dQwQExPDzTffzGeffUZVVRUejwdQN/nYbLYgV9daU7179uwhKSkJjUaD0WjkrrvuCpn1++mnn/LOO++QnZ3N0qVL+eijj3jyySdDdv22Ve8jjzwSsuu3SdP6S0hIYPLkyRQVFV3xd0SfD6js7GzeeOMNAN544w0mTZoU3II60PRHBPj73/9Oenp6EKtpSVEUfvGLXzBkyBC+//3v+8eH6jpur95QXcfl5eVUVVUB4HQ62bt3L0OHDuXmm29m586dAGzbto3s7OxglunXVr1Dhgzxr19FUUJq/f7kJz9h9+7dvPPOO/zud79j/Pjx/Pa3vw3Z9dtWvc8++2zIrl+Auro6ampq/Pc/+OAD0tPTr/g7ok91M1+6dCn79++noqKCiRMn8u///u8sWLCAH/3oR2zZsoUBAwbw/PPPB7tMv7bq3b9/P0eOHAEgJSWF5cuXB7nKCwoLC8nNzWX48OHMmjULUJchVNdxe/Xm5eWF5DouKyvj0Ucfxev1oigKt99+O7fddhvDhg3jxz/+Mc8//zwZGRnMnTs32KUC7dd7//33U1FRgaIojBw5kv/6r/8KdqmX9NOf/jQk1297HnnkkZBdv+fPn2fx4sWAun9yxowZTJw4kdGjR1/Rd4Sci08IIURI6vOb+IQQQvROElBCCCFCkgSUEEKIkCQBJYQQIiRJQAkhhAhJElBCCCFCkgSUEEKIkPT/AZUYDKp8EQYOAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Start the plot at epoch 10\n",
    "history_df.loc[10:, ['loss', 'val_loss']].plot()\n",
    "print(\"Minimum Validation Loss: {:0.4f}\".format(history_df['val_loss'].min()));"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9e0ec65d",
   "metadata": {
    "papermill": {
     "duration": 0.018133,
     "end_time": "2022-02-05T23:16:00.206225",
     "exception": false,
     "start_time": "2022-02-05T23:16:00.188092",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# 1) Evaluate Baseline\n",
    "\n",
    "What do you think? Would you say this model is underfitting, overfitting, just right?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "bb7a5a84",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-05T23:16:00.257196Z",
     "iopub.status.busy": "2022-02-05T23:16:00.255190Z",
     "iopub.status.idle": "2022-02-05T23:16:00.260581Z",
     "shell.execute_reply": "2022-02-05T23:16:00.260950Z",
     "shell.execute_reply.started": "2022-02-05T23:10:44.327332Z"
    },
    "papermill": {
     "duration": 0.029241,
     "end_time": "2022-02-05T23:16:00.261101",
     "exception": false,
     "start_time": "2022-02-05T23:16:00.231860",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"outcomeType\": 1, \"valueTowardsCompletion\": 0.25, \"interactionType\": 1, \"questionType\": 4, \"questionId\": \"1_Q1\", \"learnToolsVersion\": \"0.3.4\", \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\"}}, \"*\")"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<span style=\"color:#33cc33\">Correct:</span> \n",
       "\n",
       "The gap between these curves is quite small and the validation loss never increases, so it's more likely that the network is underfitting than overfitting. It would be worth experimenting with more capacity to see if that's the case."
      ],
      "text/plain": [
       "Correct: \n",
       "\n",
       "The gap between these curves is quite small and the validation loss never increases, so it's more likely that the network is underfitting than overfitting. It would be worth experimenting with more capacity to see if that's the case."
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# View the solution (Run this cell to receive credit!)\n",
    "q_1.check()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "382193b6",
   "metadata": {
    "papermill": {
     "duration": 0.017009,
     "end_time": "2022-02-05T23:16:00.295538",
     "exception": false,
     "start_time": "2022-02-05T23:16:00.278529",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Now let's add some capacity to our network. We'll add three hidden layers with 128 units each. Run the next cell to train the network and see the learning curves."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "26599d00",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-05T23:16:00.340960Z",
     "iopub.status.busy": "2022-02-05T23:16:00.340180Z",
     "iopub.status.idle": "2022-02-05T23:16:11.138130Z",
     "shell.execute_reply": "2022-02-05T23:16:11.137674Z",
     "shell.execute_reply.started": "2022-02-05T23:11:31.347674Z"
    },
    "papermill": {
     "duration": 10.825134,
     "end_time": "2022-02-05T23:16:11.138260",
     "exception": false,
     "start_time": "2022-02-05T23:16:00.313126",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/50\n",
      "49/49 [==============================] - 0s 4ms/step - loss: 0.2342 - val_loss: 0.2091\n",
      "Epoch 2/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.2017 - val_loss: 0.2053\n",
      "Epoch 3/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1967 - val_loss: 0.2063\n",
      "Epoch 4/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1932 - val_loss: 0.2026\n",
      "Epoch 5/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1903 - val_loss: 0.1997\n",
      "Epoch 6/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1886 - val_loss: 0.1999\n",
      "Epoch 7/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1869 - val_loss: 0.1992\n",
      "Epoch 8/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1860 - val_loss: 0.1997\n",
      "Epoch 9/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1853 - val_loss: 0.1995\n",
      "Epoch 10/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1845 - val_loss: 0.1994\n",
      "Epoch 11/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1840 - val_loss: 0.1997\n",
      "Epoch 12/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1821 - val_loss: 0.2000\n",
      "Epoch 13/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1823 - val_loss: 0.2002\n",
      "Epoch 14/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1810 - val_loss: 0.1987\n",
      "Epoch 15/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1807 - val_loss: 0.1991\n",
      "Epoch 16/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1796 - val_loss: 0.1999\n",
      "Epoch 17/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1790 - val_loss: 0.1997\n",
      "Epoch 18/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1787 - val_loss: 0.2000\n",
      "Epoch 19/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1780 - val_loss: 0.2002\n",
      "Epoch 20/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1773 - val_loss: 0.2002\n",
      "Epoch 21/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1765 - val_loss: 0.2006\n",
      "Epoch 22/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1761 - val_loss: 0.2001\n",
      "Epoch 23/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1750 - val_loss: 0.2009\n",
      "Epoch 24/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1747 - val_loss: 0.2006\n",
      "Epoch 25/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1744 - val_loss: 0.2001\n",
      "Epoch 26/50\n",
      "49/49 [==============================] - 0s 4ms/step - loss: 0.1740 - val_loss: 0.2015\n",
      "Epoch 27/50\n",
      "49/49 [==============================] - 0s 5ms/step - loss: 0.1741 - val_loss: 0.2010\n",
      "Epoch 28/50\n",
      "49/49 [==============================] - 0s 4ms/step - loss: 0.1723 - val_loss: 0.2029\n",
      "Epoch 29/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1729 - val_loss: 0.2027\n",
      "Epoch 30/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1718 - val_loss: 0.2028\n",
      "Epoch 31/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1717 - val_loss: 0.2015\n",
      "Epoch 32/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1710 - val_loss: 0.2025\n",
      "Epoch 33/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1700 - val_loss: 0.2029\n",
      "Epoch 34/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1694 - val_loss: 0.2038\n",
      "Epoch 35/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1696 - val_loss: 0.2029\n",
      "Epoch 36/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1687 - val_loss: 0.2036\n",
      "Epoch 37/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1684 - val_loss: 0.2035\n",
      "Epoch 38/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1696 - val_loss: 0.2041\n",
      "Epoch 39/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1678 - val_loss: 0.2036\n",
      "Epoch 40/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1671 - val_loss: 0.2045\n",
      "Epoch 41/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1668 - val_loss: 0.2038\n",
      "Epoch 42/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1673 - val_loss: 0.2044\n",
      "Epoch 43/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1661 - val_loss: 0.2043\n",
      "Epoch 44/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1658 - val_loss: 0.2042\n",
      "Epoch 45/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1650 - val_loss: 0.2040\n",
      "Epoch 46/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1646 - val_loss: 0.2047\n",
      "Epoch 47/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1648 - val_loss: 0.2057\n",
      "Epoch 48/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1646 - val_loss: 0.2062\n",
      "Epoch 49/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1636 - val_loss: 0.2054\n",
      "Epoch 50/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1634 - val_loss: 0.2065\n",
      "Minimum Validation Loss: 0.1987\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABBOUlEQVR4nO3deVyU5f7/8dcsDAz7PiAgbqgoKOYuJgaZJpKZ2qmszLJVT548nZaz+O14fmZpnZPHzsljpnWO2eZGiktlJi5pigsquIsiwqCyys4wvz9uRUkQkGVg+Dwfj3kAM/fc85lLhzf3fV33danMZrMZIYQQooVRW7oAIYQQojoSUEIIIVokCSghhBAtkgSUEEKIFkkCSgghRIuktXQBAAkJCZYuQQghhAX17dv3lvtaREBB9cXVR3JyMsHBwY1UjfWR9qmZtM3tSfvUTNqmZvVpm5oOUuQUnxBCiBZJAkoIIUSLJAElhBCiRZKAEkII0SJJQAkhhGiRJKCEEEK0SBJQQgghWiQJKCGEEC2SBJQQQrRyffr0sXQJTcIqAiotp4g/f59OfnGZpUsRQgjRSKwioE5lXiXhYhEnjPmWLkUIISzGbDbz7rvvMmbMGGJiYtiwYQMAmZmZTJo0ibFjxzJmzBj27duHyWTijTfeqNz2008/tWzx1Wgxc/E1hIveBoDcIjmCEkJYzqqEC3y9L7VR9/lwvwDG9/Wv07bfffcdx44dIzY2luzsbCZMmEC/fv1Yv349Q4cO5cUXX8RkMlFUVERycjJGo5H169cDkJeX16h1NwarOIK6HlA5hRJQQoi2KyEhgejoaDQaDZ6envTv35/Dhw8TGhrK6tWrWbhwISdOnMDR0ZGAgABSU1P529/+Rnx8PI6OjpYu/xZWcQTlKkdQQogWYHxf/zof7TSn/v37s3z5crZt28Ybb7zBlClTePDBB4mNjWXHjh18+eWXbNy4kblz51q61Cqs4gjKWQJKCCHo168fGzduxGQykZWVxb59++jVqxdpaWl4enry8MMPM3HiRI4ePUpWVhZms5mRI0fyu9/9jqSkJEuXfwurOILSqFXY26jkFJ8Qok0bMWIEBw4cYOzYsahUKv7whz/g5eXFmjVr+OSTT9Bqtdjb2/Puu++SmZnJm2++SUVFBQAzZ860cPW3soqAAnDUaciTIyghRBt04MABAFQqFa+//jqvv/56lcfHjRvHuHHjbnnemjVrmqW+O2UVp/gAnGzVcopPCCGsiNUElKNOAkoIIayJ9QSUrYYcCSghhLAadQqo+Ph4Ro4cyYgRI1i8ePEtjy9btozRo0cTExPD5MmTSUtLAyAtLY1x48YxduxYoqOj+eKLLxq3+ps4yRGUEEJYlVoHSZhMJmbPns2yZcswGAxMmDCByMhIunTpUrlNcHAwq1atQq/Xs2LFCubPn88HH3yAl5cXX331FTqdjoKCAmJiYoiMjMRgMDT6G3GUPighhLAqtR5BJSYmEhgYSEBAADqdjujoaLZs2VJlm0GDBqHX6wEICwsjIyMDAJ1Oh06nA6C0tLRyOGNTcNJpKC2voLjM1GSvIYQQovnUGlBGoxEfH5/Knw0GA0ajscbtV65cybBhwyp/Tk9PJyYmhuHDh/Pss882ydETKIMkQKY7EkIIa9Go10HFxsZy5MgRli9fXnmfr68v69atw2g0Mm3aNEaOHImnp+ctz01OTm7Qa9uqlSOnA0eP08FN16B9WaPi4uIGt7G1kra5PWmfmrXWtnnkkUf48ssvq33MaDQyZ84c/vnPfzboNRqjbWoNKIPBUHnKDpTiqzsK2rVrF4sWLWL58uWVp/V+vZ+goCD27dvHqFGjbnk8ODi4vrVXceBiAgAevgEEd3Rv0L6sUXJycoPb2FpJ29yetE/NWmvbqNXqGut2cnLC1ta2we+rPm2TkJBQ7f21BlRoaCgpKSmkpqZiMBiIi4vj/fffr7JNUlISs2bNYsmSJXh4eFTen5GRgaurK3Z2duTm5rJ//36eeuqpOhVcX462GgByCkubZP9CCFGrg1/AgeW1b1cffR6HsEdvu8l7772Hr68vkyZNAmDhwoVoNBr27NlDXl4e5eXlzJgxg3vvvbdeL11SUsJbb73FkSNH0Gg0vPHGGwwaNIiTJ0/y5ptvUlZWRkVFBQsXLsTb25vf/e53ZGRkUFFRwdixYxsccrUGlFarZdasWUydOhWTycT48eMJCgpiwYIFhISEEBUVxbx58ygsLGTGjBmAclpv0aJFnD59mnfeeQeVSoXZbObpp5+mW7duDSq4Jtf7oGQknxCirRk9ejRvv/12ZUBt3LiRTz75hCeffBJHR0eysrL4zW9+Q1RUFCqVqs77/fzzzwFYt24dp0+f5plnnmHz5s18+eWXPPnkkzzwwAOVA+C2bduGt7d35aVI+/bta/D7qlMfVEREBBEREVXuux5GQI0rMYaHh7Nu3bo7r64enGwloIQQFhb2aK1HO02hR48eXLlyBaPRSHZ2Ns7Oznh6ejJ37lz27t2LWq3GaDRy+fJlvLy86rzfhIQEHn/8cQA6d+5Mu3btOHv2LGFhYSxatIiMjAzuu+8+OnToQNeuXXn33XeZP38+99xzDw4ODg1+X1Yzk4S9jRqVCpkwVgjRJo0aNYrNmzezYcMGRo8ezbp168jKymL16tXExsbi6elJSUlJo7xWTEwMH330EXZ2djz33HP8/PPPdOzYkdWrV9O1a1c++OADvvrqqwa/jtUElFqlwtnORqY7EkK0SaNHj2bDhg1s3ryZUaNGkZ+fj4eHBzY2Nuzevbtyhp/66NevX+VZsLNnz5Kenk6nTp1ITU0lICCAJ598kqioKI4fP47RaESv1zN27FieeeYZTp8+3eD3ZDXLbYCy9Luc4hNCtEVBQUEUFBTg7e2Nt7c3MTExvPjii8TExBASEkKnTp3qvc/HHnuMt956i5iYGDQaDXPnzkWn07Fx40ZiY2PRarV4enry/PPPc/jwYebNm4darUar1TbKgDiV2Ww2N3gvDZSQkEDfvn0btI/k5GRe33IFdwcdn04Z0EiVWY/WOhy2OUjb3J60T82kbWpW32Hm1WWA1ZziAzmCEkIIa2JVp/ic9TakZRdZugwhhGjxjh8/zmuvvVblPp1OxzfffGOhim5lVQElR1BCCFE33bp1IzY21tJl3JZVneJzvRZQLaBbTQghRANZVUC56G0orzBTUCpLbgghRGtndQEFMpuEEEJYA6sKKFf7awEla0IJIUSrZ1UB5SxHUEIIYTWsKqBunOKTJTeEEKK1s9KAkiMoIYRo7awqoFztlZV8JaCEEKL1s6qActBp0KhVElBCCGEFrCqgVCoVLnobcmQUnxBCtHpWFVAg0x0JIYS1kIASQgjRIklACSGEaJEkoIQQQrRIVhdQrvYSUEIIYQ2sLqBc9DbkFZVRUSFLbgghRGtmlQFVYYb8knJLlyKEEKIB6hRQ8fHxjBw5khEjRrB48eJbHl+2bBmjR48mJiaGyZMnk5aWBkBycjK/+c1viI6OJiYmhg0bNjRu9dW4PmFsnpzmE0KIVq3WJd9NJhOzZ89m2bJlGAwGJkyYQGRkJF26dKncJjg4mFWrVqHX61mxYgXz58/ngw8+wM7OjnfffZcOHTpgNBoZP348Q4cOxdnZucnekOtN8/EFNNmrCCGEaGq1HkElJiYSGBhIQEAAOp2O6OhotmzZUmWbQYMGodfrAQgLCyMjIwOAjh070qFDBwAMBgPu7u5kZWU18luoSiaMFUII61DrEZTRaMTHx6fyZ4PBQGJiYo3br1y5kmHDht1yf2JiImVlZbRv377a5yUnJ9el3hoVFxeTnJzMlWxlqY2jJ8/iXnapQfu0JtfbR9xK2ub2pH1qJm1Ts8Zom1oDqj5iY2M5cuQIy5cvr3J/ZmYmf/jDH3j33XdRq6s/aAsODm7QaycnJxMcHIxrbhF8ewFHdwPBwdWHYVt0vX3EraRtbk/ap2bSNjWrT9skJCRUe3+tAWUwGCpP2YFyRGUwGG7ZbteuXSxatIjly5ej0+kq77969SrPP/88r7zyCmFhYXUqtiFc9bLkhhBCWINa+6BCQ0NJSUkhNTWV0tJS4uLiiIyMrLJNUlISs2bN4qOPPsLDw6Py/tLSUqZNm8bYsWMZNWpU41dfDTsbNTqNmhxZVVcIIVq1Wo+gtFots2bNYurUqZhMJsaPH09QUBALFiwgJCSEqKgo5s2bR2FhITNmzADA19eXRYsWsXHjRvbt20dOTg5r1qwB4J133mnSQ2KVSoXztYt1hRBCtF516oOKiIggIiKiyn3Xwwjg008/rfZ5Y8eOZezYsXde3R2S6Y6EEKL1s7qZJEAmjBVCCGtgtQElq+oKIUTrZrUBJUdQQgjRuklACSGEaJGsNqDyi8sxyZIbQgjRalltQIHMaC6EEK2ZVQeUnOYTQojWyyoDytVeAkoIIVo7qwyo60dQORJQQgjRall1QMkRlBBCtF7WGVByik8IIVo96wwoGcUnhBCtnlUGlK1Wg52NmpxCWXJDCCFaK6sMKJDZJIQQorWz2oBy1eskoIQQohWz2oCSIyghhGjdrDagnGXJDSGEaNWsNqBcZNl3IYRo1aw2oGTZdyGEaN2sNqBc9DYUlJooM1VYuhQhhBB3wKoDCmQ2CSGEaK0koIQQQrRI1htQMh+fEEK0anUKqPj4eEaOHMmIESNYvHjxLY8vW7aM0aNHExMTw+TJk0lLS6t87JlnnqFfv348//zzjVd1HVQeQclQcyGEaJVqDSiTycTs2bNZsmQJcXFxrF+/nlOnTlXZJjg4mFWrVrFu3TpGjhzJ/PnzKx+bOnUq8+bNa/zKayGn+IQQonWrNaASExMJDAwkICAAnU5HdHQ0W7ZsqbLNoEGD0Ov1AISFhZGRkVH52ODBg3FwcGjksmvnKgElhBCtWq0BZTQa8fHxqfzZYDBgNBpr3H7lypUMGzascaprAOfrq+rKKT4hhGiVtI25s9jYWI4cOcLy5cvr/dzk5OQGvXZxcfEt+9BrVZxNyyA5ubxB+7YG1bWPUEjb3J60T82kbWrWGG1Ta0AZDIYqp+yMRiMGg+GW7Xbt2sWiRYtYvnw5Op2u3oUEBwfX+zk3O3b4AN1/tQ83h4to9M4N3rc1SE5OlnaogbTN7Un71Ezapmb1aZuEhIRq76/1FF9oaCgpKSmkpqZSWlpKXFwckZGRVbZJSkpi1qxZfPTRR3h4eNSpoEaVcZiua0dCyo4qd7vYy5IbQgjRWtV6BKXVapk1axZTp07FZDIxfvx4goKCWLBgASEhIURFRTFv3jwKCwuZMWMGAL6+vixatAiAxx57jDNnzlBYWMiwYcOYM2cOd999d+O+C48ulNu6odvwGjwfDxrlbbnoteQWyaq6QgjRGtWpDyoiIoKIiIgq910PI4BPP/20xueuWLHiziqrDxs9mX1m4L/zTdj3CQxUrrly0dtw9nJB07++EEKIRmc1M0nk+w2HTsNh6xwouAzIooVCCNGaWU1AoVLB/fOgtAC2zAbAVfqghBCi1bKegALw6gYDX4D9/4W0/bjobSguq6C4zGTpyoQQQtSTdQUUQMTr4OAFG1/D2U4DICvrCiFEK2R9AWXnDPe+BRf2EnJ5IyDTHQkhRGtkfQEF0PtR8O9Pz6Pv40ghOTUFVEUFnN8NZUXNW58QQrRmeRfhu7/Arg+b9GUadaqjFkOthvvnYfNxJC9r15Bb+Ku5ActLIPEr2LUQLp+AsEnw4L8tU6sQQjQnUxkYj8LF/ZC2H9IPgYs/hIyHbveD7jaTe185DTs/gINfgLlC6VJpQtYZUAB+d3G1x6NMOfoVWzNfgB4GKMqBhGWwexFczQCfXtB9DBz8HPpOgYD+lq5aCCEal6kMjsXB+Z8hLQEyDkN5sfKY3h18e8PFg3B8A9g4QPfREDIBOkeC9tq0demHYMc/ICkW1DbQdzIM+S24dWjS0q03oICKyL9QeHQtIYdmQ+kg2PcplOZDp3tg3CLluqnSAviwH2x4FZ79EdQaS5cthBANV16i/PG94x+Qcx5s7ME3DPpPBb+7wK8vuAYql+hUVMD5XXD4GyWEDn8DejcIfgDy0uDUD2DrDOEzYNBL4OjdLG/BqgPK0d2X2eUT+Wv2Z/Dzfug5DsJfVv5iuM7WEe77f7DqGTjwP+j7lMXqFUKIBisthP2fwc5/Qv5F8OunXCPaZUTlNHC3UKuhw1Dldv98OP0jHFkJh1eCjR4i/6IEm961Wd+KVQeURq0i1mYUYYHtGffghJoPR0PGw76l8MNflb8Y7N2btU4hhGiwknzYu0QZuFB4GQKHKn3rnYYrR0l1pdVBt1HKrbwEVGrQ2DRZ2bctxSKv2owc9XbE29/LuNudK70+C8V/7lamSop+v9nqE0K0YFlnwN5TuXylPopyoOAS2DopNxv7+oVEXZWXwOmtkPwtJK+HklzoHAXDXoXAIQ3fv9a24ftoyMtb9NWbgat9Hefj8wmB/s/C3o/hrsng26vpixNCtDwFl5U+mIOfKwMKNDql37rHA9BtdM1nWHLTlMEIx9YrS/+Yb5rBRqVRgsrOWenLsXNR+niu3+zdlQELejew91D6eBy8lO1+HWylhUqfUPK3cGIzlOSBrYsyAm/gc0rfkpWw+oBy0duQU1jHJTfueVM577rxNZiysWn+4hFCND2zGS4dhzNblf6U9ERw76T0P1+/eXa90SdTXgonv4NDX8CJTVBRDu36wMi3leBJXgcnNytB0yFc6QroHo0u9yzEXwuliweUfXl2UwYTePdQwqMkX/lafPP3ucrRWWEWFGWBqYbfURpbJawcvcHBW/mddOYnKCtUAq3HWOjxIHQcdmPEnRVpEwGVkVtct431bsosFN/+VvkLqtfDTVqbEG1KbhokfgmHVykd72GPKsOZ69LxfvUSHFunhI7eHRw8lFNvDp7KkYa9p3LEcjZeCaTTW5UBAgDunaFTBGSnKIMHygqV+7V2YOgJbh2VICu8ooTAoBeh92Ng6HHj9UfOUYZaJ38LSd8qo343vErn64/79YOo/4PgGPAMql+7mM1KTYVZUJSt1FFwCa4a4WqmcivIhNwLUFagTETQ4wGlj6mmQQ9WwrrfHeCi15FbVF73J4Q9DvuWwXd/hq6j6n/uWQhxQ2mB0jdyaAWc2QaYof1g5Ugi7vew6Y8QPEa5WL7T8KqXeeRnKEcuSbFwbqdyYaiNg/JL+nbsXJVA6hypnJpzC7zxWIUJrpxSwub6LWW7MnotbJLSf1PdL32VCtqFKbeoWUpQHt9ARtZVfIY/A87t7ryNVCrl4lidA7gG3Pl+rFAbCCgbcotKMZvNqOpyyk6thtHvwZJIiJ+nDEEXoqUzm5UwsLFX/g9bUmGWMkvBkTWQtBZKryrX20S8Dr1/o5xqM5uVcDj4OSR+DUdWgbOfcnTg4KWE0vmfAbNyKu7uV5XTWYaeyoWnhVeUkWoFl68dcVxWTpMFhishUtP1jGqNsuqBV7eGnSG5to/s5GR8GhJO4rbaRECVmcwUlZmw19Xx7fr3hT5PwO6PoM+T4NW1aYsUoiGuXoJvJitHGaiujRxzvtYhf+17ra0y4qu8WJl7srz42vfFYGMHQ2cq4VDfcCu4rMxCkH7wxhFJzjnlMZ0T9HxQOV3WfnDVfd98RDLib8osBgc/hx1/V46UvHvA8DeU/hXv7lVfU6sDZ1/lJqxamwgoUGY0r3NAgdIXlfwtxL6kDJiw0HUAQtxWxhH44lGlj2LYHwDVTR3y126Fl28EkVavBJfWW+mDsdFDZrLy/zzhUxg9XwmN2ynOU/py9n4C2Wdv3O/WURlY0G+KMgghYBDo7Gt/DzZ2EPKQcsvPUI4EPTrX/jxh9aw+oFztbwSUr4u+7k908IQxH8DKKbDlr3KqT7Q8x+Jg1bNK4EzZqExfcycqKpTRaz/8HywergRM5F9uHU6dewH2LIKEz5TgCxwK/Z9RwsinV+PMMuDk0/B9CKth9QF1/Qgqp/AO1oQKeUg5bbJroXJuu9v9jVydEHfAbFbmV9syWzlieWRFw053qdXQZxJ0j4af3oFfFsPRNcpggLsmY5t9Alb/Q+knMpuV03aDp995IApRR1YfUG72yrUBF7LvcM2n++ZA6i+w5gV4YYeMshFNp7RA6fcsL1F++be7C5wMVbcpK4Z1LyvLxYSMh7H/Uk7TNQa9K9z/Dtz1BGx4Dda/Atvm0Sk/HXSOMOB5GPQCuLZvnNcTohZWH1DdfJwI9LBnxZ5zTOjrX/8d2NjBxE/hPxHK6T7pjxJN4fRWWDdDGWCgUisDBUAZ2daujxJYhlBlZOmFvRD5Z2VkW1NcTG7oCU+tV46Y9v8XY8eHMNz/WrNPFCqE1QeURq1iypAOvLUuiQPns+nT3q3+O/HoDA/8U/qjhCL7nHIh97mdykJvnl2v3YKU4dT1WbKlMEu55u7g5+DRBZ7aoAxSSE+8saDcxf3KTAWgDCN/+H/KhZpNSaWC0AkQOoGs5GQMEk7CAuoUUPHx8cyZM4eKigomTpzIc889V+XxZcuW8c0336DRaHB3d+ftt9/Gz88PgDVr1vDRRx8B8OKLLzJu3LhGfgu1m9AvgPe/O8EnO87y4WN3EFDwq/6oocpMv6J1Ks6FSyeU1ZQvH8eQeRGKhytr5Xh1r/5CzaJsOLpWuWbn/C7lPu8eylxt+/97YzuNrRI0Xl2VU3QBA5T92thV3Z/ZrPTzbHxNCamhM5XrhK5vFzhYud38+umHlBn5m3iROCFailoDymQyMXv2bJYtW4bBYGDChAlERkbSpUuXym2Cg4NZtWoVer2eFStWMH/+fD744ANycnL48MMPWbVqFSqVioceeojIyEhcXFya9E39mqOtlkcGBLB0ZwoXc4po53qH5+zvmwOpe2DtC/D8duvsjyotgLx0yL92c+ug/JJtacqKlVWR8zOu1XpteDJmMHPtq/nG18LLytX/l08oU8hcp9HhqtLAyW+Un7V6ZeJg3zDlSMZGD0dWK/O0mUqVedYi/wKhE2/MUFCYBZdPXgu8E8r3afuVAAJlBVLf3ko7BgxQpt7Z+jac2Ki8zhNrwCf09u9X76bMtCBEG1JrQCUmJhIYGEhAgPLLODo6mi1btlQJqEGDBlV+HxYWxrfffgvAjh07CA8Px9XVFYDw8HC2b9/OmDFjGvM91MnkIR34ZMdZPvs5hTfvD76zndjYwcTPrvVHPQ1TNtzojyovVSZ9LMxSrmy3dQSf3pa/qv/XSvKVSSqzzsCV05B1FnJTb/ySL8m79Tn9n4URs+t2TcvtmM1K/8npH5WLR50M4OijDC12NChtdn27gsvKKqA5KcrX7HPK1+vBWZRdv9e2dVZOw3W5V/nq1U356hrI8ePHCfayUS42vX7R6aEvlJntQaltwHPKzAM+vW7t97F3h/YDldvNrl6CC78og2xSf1HWHNv9b+UxrV45VTzwRaufT02IO1XrJ8NoNOLjc+PaBIPBQGJiYo3br1y5kmHDhtX4XKPRWO3zkpOT61x0dYqLi2vdR3h7Bz7/OYWRfhXobe48OJz6vob/z3+h5B99UJnL0ZTkoikvvGW7Mr0XV9sNJd/vbgq9+2HWVD/bsKY4C/vLiegvH0aXn4LJxhmTnRvldu6YbN0pr/zejQqNLWaNDrNad2tfh7kCTXE2NkWX0BZlYlOYibboEtqiSwTknac89iLa4qyqNdp5UubgQ7nej3K3MMr1npTpvSjXe1Fu54HrmVg89n5MybHvuDjoLYrd6x/uNlfTcEnZiMu5TeiuXqhxuwqtnnJbV7QlOajLq466LNe5UObgS7m9gTK/YMr1npTbeSlfr90qtPaYUd0UIDV9D1QAmaWQeZLiklKSL6vBJhQCQyHwCTBXoMs/j6Y0jyL3nkpb5wA5x+r57juBfyfwfwRMZdjlnMQu5yQFhn6UOfrBiZP13F/zq8tnq62StqlZY7RNo/7pFhsby5EjR1i+fHm9nxscfIdHNdckJyfXuo/f2RsY/9HPHClw4MnBHe78xYKDwUmN7ZmflL+e7T2UGZbt3W58n5+OzbE43E59h9vpNcow3S5R0C0avIMhLeHaX9a7laMZUNad8ewKBanKzADltczCrtIoU9hobZXTSEVZyjIBN1NrwdGHAlsvtMHRyjxoHp2Vr24dsbF15LZjEgeNhjOPYrv2JTpueVbpJxk6s/a/+q/32Rz6UnmPqJQJOaP+qMz4XFGuHLFdzVBma87PQH3ViK7gsjIXm2t75RSaa3twbY/W1qnJRvTU/H+nZxO8WutbZ6wun622StqmZvVpm4SEhGrvr/UzbzAYyMjIqPzZaDRiMBhu2W7Xrl0sWrSI5cuXo9PpKp/7yy+/VHnugAGW68+4q70bvQNcWbYzhccHBqJWN2CI7pDpyu12ej+i9JWcjYfjcXB8ozIJ5nX2ntB+EPSdAgEDlT6P6ytYms3KJJtXM5XTXQXXvl6fT81Uem0+tZJr35co4ejcTrk5+SpDlB28QK3mfEM+SJ2Gw4s7Ie5VZcXhk9/BuP/cmI7GbFZC9vqIs7QEZW2c6302Uf+nnB5z+dUwf3v3qksaCCHETWoNqNDQUFJSUkhNTcVgMBAXF8f771ddEj0pKYlZs2axZMkSPDw8Ku8fOnQof//738nNzQWUPqmZM2c28luoO5VKxTNDO/LyFwfYejyTqOBbg7bR2dhB1/uUW/Q/lF/gWWeV61rcO9V8HYtKdWO56JYwL5neDSZ8osymETcTFt2tzD5w5ZQSTMU5ynZavTIgYODzyoWkvmGy8KMQ4o7UGlBarZZZs2YxdepUTCYT48ePJygoiAULFhASEkJUVBTz5s2jsLCQGTNmAODr68uiRYtwdXXlpZdeYsKECQBMmzatcsCEpdwf4oOvix2f7DjbPAF1M7Ua/Pspt9YqdIJy1Bc7TZks1LuHsgyC313KUtNewdLpL4RoFHX6TRIREUFERESV+66HEcCnn35a43MnTJhQGVAtgY1GzZODO/DupmMkXcyjRztZkLDeXPzhyVgwlUsYCSGaTAsbA908HhvQHr2NhqU7z9a+saiZhJMQogm1yYBysbdhQl9/vj14kUv5JZYuRwghRDXaZEABPBXegVJTBct3n7N0KUIIIarRZgOqs5cjkd29Wb77HMVlJkuXI4QQ4lfabEABPDO0I1cKSvkmoebZDYQQQlhGmw6oIZ09GNDRnfmbjpGZX8usDUIIIZpVmw4olUrF3IdCKS6v4K/fJlm6HCGEEDdp0wEFSl/UjKgg4g6n893RjNqfIIQQolm0+YACeG5YJ7r7OPGX2CPkFZdZuhwhhBBIQAHK7BLvjO9FZn4J8zbVdzkFIYQQTUEC6pqwAFemDOnI8t3n2ZuSVfsThBBCNCkJqJv8/r6u+LnqeWNVolwbJYQQFiYBdRMHWy1zxoVw+lIB/956ytLlCCFEmyYB9SvDu3kzro8fH207zfGMfEuXI4QQbZYEVDX+HB2Mo62W11clYqowW7ocIYRokySgquHhaMusmB4cTM1hmSzJIYQQFiEBVYMHw/yI6u7Nu5uOkXghx9LlCCFEmyMBVQOVSsV7E3vj5WjLtBX7yS2SC3iFEKI5SUDdhpuDjoWP3UV6TjGvrTyE2Sz9UUII0VwkoGrRN9CN10d1Z/NRI0t3pli6HCGEaDMkoOpg6t0duTfYwNwNyRw4n23pcoQQok2QgKoDlUrF+xN7Y3C2Y/qKA+QUllq6JCGEsHoSUHXkYm/DvybdRWZ+Ma9+I/1RQgjR1OoUUPHx8YwcOZIRI0awePHiWx7fu3cv48aNo0ePHmzatKnKY/Pnz2fMmDGMGTOGDRs2NE7VFhIW4Mqb9wfzQ3ImH28/Y+lyhBDCqtUaUCaTidmzZ7NkyRLi4uJYv349p05VnafO19eXuXPnMmbMmCr3//TTTyQlJbF27Vq+/vprPvnkE65evdq476CZTQnvwKiePry76Tj7ZNZzIYRoMrUGVGJiIoGBgQQEBKDT6YiOjmbLli1VtvH396d79+6o1VV3d+rUKfr164dWq8Xe3p5u3boRHx/fuO+gmalUKt6d0As/Vz2Tl/5CXGK6pUsSQgirpK1tA6PRiI+PT+XPBoOBxMTEOu28e/fufPjhhzz99NMUFRWxZ88eunTpUu22ycnJdSy5esXFxQ3eR338v0hP3t5mZNqK/fxw0IWn+7qjVaua7fXrq7nbpzWRtrk9aZ+aSdvUrDHaptaAaoihQ4dy+PBhHnnkEdzd3QkLC7vlKOu64ODgBr1WcnJyg/dRH8HAoLCevL0hmU93pZBaoObDx+7Cx8Wu2Wqoj+Zun9ZE2ub2pH1qJm1Ts/q0TUJCQrX313qKz2AwkJGRUfmz0WjEYDDUsUR48cUXiY2NZdmyZQB07Nixzs9t6XRaNW890JN/PtqHpPQ8xizczq7Tly1dlhBCWIVaAyo0NJSUlBRSU1MpLS0lLi6OyMjIOu3cZDKRna1c2Hrs2DGOHz9OeHh4wypugR7o3Y7YaeG46G14fMkePvrptAxDF0KIBqr1FJ9Wq2XWrFlMnToVk8nE+PHjCQoKYsGCBYSEhBAVFUViYiLTp08nLy+PrVu3snDhQuLi4igvL2fSpEkAODo6Mn/+fLTaJj2raDFBBidipw/l9VWJvLvpGPvPZ/PexN646G0sXZoQQrRKdUqLiIgIIiIiqtw3Y8aMyu979epV7eg8W1vbVn/tU3042mr58NE+9G3vxtsbkhn74Q4+erwvwb7Oli5NCCFaHZlJopGpVCqeHtqRL58bRGGpiXH/3smaAxcsXZYQQrQ6ElBNpF8Hd9a/PJTe/q688tUh/rL2CKXlFZYuSwghWg0JqCbk7WTH51MH8tywTvxv9zke/s/PpOcWWbosIYRoFSSgmphWo+aPo4P596S7OGnMZ8w/d7D95CVLlyWEEC2eBFQzGR3qS+z0obg56Hjik18Y+6+drEy4QHGZydKlCSFEiyQB1Yy6eDvy7fRw3orpwdXiMl795hCD5m5hTlwSKZcLLF2eEEK0KNZ5UVILZq/T8lR4RyYP6cDuM1ks332OZTtT+Hj7We4O8mTSwECGd/PCzkZj6VKFEMKiJKAsRKVSMbizB4M7e5CZV8yXe1NZsec8LyxPQG+j4e4gT+7tYSCquzcejraWLlcIIZqdBFQL4O1sx8tRQbw0vDO7Tl/hh2QjPyQZ+S7JiEoFfdu7cW8PAyN6GOjs5WjpcoUQollIQLUgWo2aYV29GNbVi78+0JOjF/OUsEo28s7GY7yz8Ri9/V2Y2C+AB8La4Wwn0ygJIayXBFQLpVKpCPFzIcTPhd/d25WLOUVsOpLB1/tS+fPaI/y/uCRGh/gysV8Agzq5o1K13LWohBDiTkhAtRLtXPU8PbQjU8I7cDgtl6/3pRJ78CKrD6QR6GHPxL7+DOvqRXcfZ3RaGZwphGj9JKBaGZVKRS9/V3r5u/Kn0T3YdDSdr/de4L3vTvDedyfQadQE+zrRy9+V3gGu9PZ3oZP0WwkhWiEJqFZMr9Mwro8/4/r4k55bxIHzORy6kMOh1BzWHEjjf7vPAcos62O7O/K3bmbULXhZeiGEuJkElJXwddHjG6pndKgvABUVZs5cvsrB1Fx+SDLy+aEMcs0HeG9ib7nGSgjRKkhAWSm1WkUXbye6eDsx/i4//rZyN0sT0rmYU8THT/aTa6uEEC2e9Ka3ASqViokhrnw06S6OXszjwX/v5FRmvqXLEkKI25KAakPuD/Xlq+cHU1Rawbh/72LnqcuWLkkIIWokAdXGhAW4snbaEHxd7Ji89Be+2nu+yuPFZSYy84o5Ycxnb0oWqVmFFqpUCNHWSR9UG+TvZs/KF4cw7fP9vL7qMB9vP8vV4nJyikopLqu66q9WreLJwR2YcW8QLnqZuUII0XwkoNooZzsblj3VnwVbTnI8Ix9Xextc7XW46G1w0dvgam+Ds50Nm45msGzXWdYeTOP393Xlkf7t0chQdSFEM5CAasO0GjW/v6/bbbcZ1tWLSQPb89d1SfxpzRGW7z7P/8X0YFAnj2aqUgjRVkkflKhVz3YufPXcIP712F3kFZXxyOLdvPR5AqcvXcVsNlu6PCGElZIjKFEnKpWK6F6+RAV7szj+DP/+6RQbDmdgr9PQwcOBTl4OdPJ0oKOXA508Hens7Yijrfz3EkLcuTr9BomPj2fOnDlUVFQwceJEnnvuuSqP7927l7fffpvjx4/z97//nVGjRlU+Nm/ePLZt20ZFRQXh4eH86U9/kpm3WzE7Gw0vRwUxsZ8/PyQZOXO5gLOXC0i8kMuGw+lUXDug0mnUPDIggJeGd8HHxc6yRQshWqVaA8pkMjF79myWLVuGwWBgwoQJREZG0qVLl8ptfH19mTt3LkuXLq3y3P3797N//36+/fZbAB577DF++eUXBg4c2MhvQzQ3Xxc9TwzuUOW+knITqVmFnLlUwNbjmazYc54v96by2ID2vDi8MwZnCSohRN3VGlCJiYkEBgYSEBAAQHR0NFu2bKkSUP7+/gCo1VW7tFQqFaWlpZSVlWE2mykrK8PT07Mx6xctiK1WUzm90n09fXhpeBf+tfUU/9t9ji9+Oc+kgYG8MLwT3k4SVEKI2tUaUEajER8fn8qfDQYDiYmJddp5nz59GDhwIEOHDsVsNvP444/TuXPnardNTk6uY8nVKy4ubvA+rJml2mdyDxvuC/Dni8RsPt11ls93p3B/VyeGtHegm5ctOo3lx+nI/53bk/apmbRNzRqjbZq0F/vcuXOcPn2abdu2AfD000+zb98++vXrd8u2wcHBDXqt5OTkBu/DmlmyfYKByAGQcrmAhT+eYu3BNNYm56HTqAkLcGVgJ3cGdHTnrvZuOFhgYIX837k9aZ+aSdvUrD5tk5CQUO39tf42MBgMZGRkVP5sNBoxGAx1etHvv/+e3r174+DgAMDdd9/NgQMHqg0oYf06eDrw/sO9+cuYYPalZLPn7BV+OZvFv386zcIfT6FVq+jp50LPds509Xakq8GJIIMTno46GVgjRBtUa0CFhoaSkpJCamoqBoOBuLg43n///TrtvF27dnz99deUl5djNpvZu3cvkydPbnDRonVztddxbw8D9/ZQ/tC5WlLO/nNKYO1NySYuMZ0VRWWV27vZ2xBkcKKrwZH+HdyJ6OqFq73OUuULIZpJrQGl1WqZNWsWU6dOxWQyMX78eIKCgliwYAEhISFERUWRmJjI9OnTycvLY+vWrSxcuJC4uDhGjhzJ7t27iYmJQaVScffddxMZGdkc70u0Io62WoZ19WJYVy8AzGYzl66WcNJ4lRPGfE4Yr3LSmE/swYss330etQr6BrpxT3dvIrt7083gJEdYQlihOp3wj4iIICIiosp9M2bMqPy+V69exMfH3/I8jUbD7NmzG1iiaGtUKhXeTnZ4O9kR3uXGqE9ThZlDF3LYeiyTH49lMm/TceZtOo6fq57h3by4r6cP4Z090LaAgRdCiIaTS/1Fq6FRq7irvRt3tXfj9/d1w5hXXBlWaw6k8fme83g46Bgd6ktM73b0C3RDLRPbCtFqSUCJVsvgbMcjA9rzyID2FJeZ2HbiEt8eusg3Can8b/c52rnYMaZ3O2J6tSPEz1lOAwrRykhACatgZ6NhZE8fRvb04WpJOT8kGVl36CJLd5xlcfwZOnjYMyrEl9GhPoT6uUhYCdEKSEAJq+Noq+XBPn482MePnMJSNh7JYMPhdD7efoZF207j56pnVIgPo0N96BPgZulyhRA1kIASVs3VXsejA9rz6ID25BSW8n2SkU1HMvjfz+f4ZMdZDM629DHoGJbnQKifC119HLHVaixdthACCSjRhrja65jYL4CJ/QLILy7jx2OZbDycwY6TmWw6eRgAG42KrgYnQtq5EOLvQr9AN4J9nS1cuRBtkwSUaJOc7GwYG+bH2DA/kpKScDR04HBaLkcu5nIkLZfNSRl8tS8VUFYVfuXeIPq0l9OBQjQnCSjR5qlUKtp72NPew57oXr6AcrFwWk4R6xPT+c+204z79y7u6ebFKyO60svf1bIFC9FGyBWNQlRDpVLh72bPCxGd2f56JH8Y2Y3953N44MOdTP1sH0fSci1dohBWT46ghKiFo62Wafd04cnBgXy6M4WPt59hzEIjUd29uSvQjUAPezp4OBDoYY+TnY2lyxXCakhACVFHTnY2/DYqiCeHdGDpjrN8vS+VLccyq2zj4aCrDKx2rnp8Xe3wdbHD10WPr4sdLnobuQZLiDqSgBKinlz0NrwyoiuvjOhKYWk557MKSblcQMqVQs5dKSDlciG7z1zBmF+CqcJc5bl6Gw2+LnbKSEE/Z3r6uRDq54Kno62F3o0QLZcElBANYK/T0t3Hme4+tw5FLzdVcPlqKRdzi0jPKSY9t4j03GLSsos4bsxn09Eb66z5ONspgdXOBXudhvzicvKLy8gvLifv2vdXS8rp38GdN0d3l2u1RJsgASVEE9Fq1Pi42OHjYgftb308r7iMpIt5HEnL5ei1rz8ey6TCDGqVckrR0VaLk50WZzsbXO1t+HRXCofTcln0eF+8nOSoS1g3CSghLMTZzoZBnTwY1Mmj8r6iUhMVZjP2Ok21fVVxien8/puDjP1wBx9P7kfPdi7NWbIQzUqGmQvRguh1GhxstTUOpIju5cvKF4ZgBiZ89DObjqQ3b4FCNCMJKCFamRA/F2Knh9Pd14kXlu/nn1tOYjabb9kuNauQL385z/QV+3nikz18n2SsdjshWio5xSdEK+TtZMcXzw7ij6sP8/fvT3DcmM+fo4M5eD6H7acus/PUZc5dKby2rS02GjXP/ncf3X2c+G1kEKNCfNDIYo6ihZOAEqKVsrPR8P7Dvenm48Q7m44Rl6ic7nO01TKokztPDenA0C6edPF2xFRh5ttDF/lw6ymmrdhPZy8Hpt3ThQd6t0OrkRMpomWSgBKiFVOpVDwf0ZlQPxcOpOYwqJM7vfxdsflV6Gg1Kh66y5+xYX5sPJLOhz+eYubXh/jgh5M8Hd6BQE8H3Ox1uNnb4KrX4WSnRS1HWMLCJKCEsAJDungypItnrdtp1CrG9GrH6BBfthzLZOGPJ3lrXdIt26lVyvIkHg46glxVPGR2Z0gXD+x18itDNB/53yZEG6RWqxjRw8C9wd6cu1LIlYJScgpLyS4sI6ewlJzCMnKKSrmYU8zWU5fYcGIfOo2agZ3cGd7Nm3u6edHR00GmbRJNSgJKiDZMpVLRwdOBDp4ONW6TeCSJq3bebD2eydbjl/jb+iT+th4CPez5Tf8AHh8UiLNMkiuaQJ16R+Pj4xk5ciQjRoxg8eLFtzy+d+9exo0bR48ePdi0aVPl/bt372bs2LGVt9DQUH744YfGq14I0eRsNCqGdPHkT9E9+GFmBNtfu4e/PRiCv5ueeZuOEz73R97ZeIzM/GJLlyqsTK1HUCaTidmzZ7Ns2TIMBgMTJkwgMjKSLl26VG7j6+vL3LlzWbp0aZXnDho0iNjYWABycnK47777CA8Pb+S3IIRoTgHu9jwxKJAnBgVyJC2XRdtOszj+NEt3nmVCX3+eu7vTbY/IhKirWgMqMTGRwMBAAgICAIiOjmbLli1VAsrf3x8AtbrmA7LNmzdz9913o9frG1qzEKKFCPFz4cPH7iLlcgGLt59h5b4LfPnLee4P9eU3/QIY0NEdOxuZ2FbcmVoDymg04uPjU/mzwWAgMTGx3i8UFxfHlClTanw8OTm53vu8WXFxcYP3Yc2kfWombXN7dW2fJ7priW7vz9rkXNYnZxCXmI6tRkUvHzv6+tnTz8+edk7VT+NkqjCTVWTCeLUcZ1s17V11jVa/2Wwmu9iEu77xu9zl/07NGqNtmmWQRGZmJidOnGDo0KE1bhMcHNyg10hOTm7wPqyZtE/NpG1ur77tE94XZpea2H3mCttOXGLbiUss+uUKcIX27vZEdPXCzd6GCzlFpGUXkZZTREZuMeU3rZ0V3sWDp8M7ck837wZdj5VwLpt5m46x52wWz97dkTfuD27UGTTk/07N6tM2CQkJ1d5fa0AZDAYyMm6sW2M0GjEYDHUsUbFx40ZGjBiBjY2M9BGiLdDrNNzT3Zt7unsDcO5KAfHXwmrV/gsUl5kwONvh56qnb6Abfq56/Nz0+LnqSU7P57NdKTzz2T46eTowJbwD4/v61+sarBPGfOZvPs73SUY8HW0Z2dPAx9vPcvpSAQseCcNJRh22CrX+i4eGhpKSkkJqaioGg4G4uDjef//9er1IXFwcM2fOvOMihRCtW6CHA08MduCJwR0oM1VgNoNOW32f9fBu3ky9uyMbDqezdMdZ/hJ7lPmbj/PowPY83C+AADf7Gp97IbuQf3x/ktUHLuCo0/LqfV2ZEt4RB1st//s5hbfWJTHho59ZMrkfAe72TfmWRSOoNaC0Wi2zZs1i6tSpmEwmxo8fT1BQEAsWLCAkJISoqCgSExOZPn06eXl5bN26lYULFxIXFwfAhQsXSE9PZ8CAAU3+ZoQQLd+vp2GqaZuxYX480Lsd+89ns3RHCh/Hn+E/284A4Oloi6+LHQZnO3yvLQp5Kb+EFXvOgwqmDu3IS8O74OZwoy/ricEd6OjpyEufJ/Dgv3bynyf60q+De5O9T9FwdTpmjoiIICIiosp9M2bMqPy+V69exMfHV/tcf39/tm/f3oAShRBtlUqlom+gO30D3bmQXciuU1dIzy0mI6+I9NxiLmQXsu9cFjmFZahVMLFvADPuDaKda/WjhYcGebJmWjhTP9vHYx/v4e2HQpnQ17+Z35WoK5lJQgjRKvi72fNw/+pPyxWVmigtr8DFvva+pc5ejqx5aQgvfb6fV785xKnMq8wc0bXG04bCciSghBCtnl6nQa+r+/VWrvY6Pnt6AP/37VEWbTvN53vOEdXdm1EhvkR09arXvkTTkYASQrRJNho1cx4MYWRPH9Yfusj3yUbWHryI3kbD8G5ejArx4Z7u3nWeZ9BsNnMhu4j957NJOJeNh4MtTw/tICMGG0ACSgjRZqlUKiK6ehHR1YtyUwV7zmax6UgGm49msPFIBjYaFR09HSqHwbdzVYbC+7vp8XXRc+xSMTsvn2H/+Wz2pWSTmV8CgL1OQ2Gpif/tTmHmiG483M9fFoa8AxJQQggBaDVqwrt4Et7Fk78+0JMDqdl8l2TkzKUCLuYUcSA1h5zCsmqeeRF/Nz1DOnvQN9CNuwLd6O7jzJG0XP5fXBJ/XHOYz3al8KfoYIZ19Wr299WaSUAJIcSvqNU3Rg/erKCknIs5RVzIKeJiThEFWZmMDQ/F4Gx3yz56B7jy9fOD2XQkg7kbj/Hk0l8Y3s2LP40OJsjg1FxvpVWTgBJCiDpysNUSZHCqDJjk5MJqw+k6lUrF/aG+RAZ7899d5/jnjycZtWA7o0J86G5wItDTgQ4e9gR6OOCil76qX5OAEkKIJmar1fDssE6M7+vPgh9OsPmokbjE9CrbuDvoCPSwp6u3E6NCfRjaxbNOFzVbMwkoIYRoJu4OOv46NoS/jg2hqNTE+axCUq4UkHK5gJQrhZy7UsCGI+l8tS8Vdwcd94f4ENO7HQM6uDdo0tzWSgJKCCEsQK/T0M3HiW4+VfujSspNxJ+4zLeHLrJ6fxqf7zmPj7MdY3r5Et3LlxA/lzZzZCUBJYQQLYitVsOIHgZG9DBQWFrO90lG1h1K57OfU1iy4yw6jZquPo709HUhxM+ZHu1cCPZ1qpzt/dcDOdKyleVM7HQavBxt8XSyxcvRFq9rXz2ddPWaKb45tcyqhBBCYK/TMjbMj7FhfuQWlrHt5CWOpuVy9GIem5My+GpfKgBqlTIVVF5x2S1D4bVqFQZnO4rLTGQVlmI23/o6fQPdeCGiM1HdG7b+VmOTgBJCiFbAxd6GB3q344He7QBl5oqLucWVgXXq0lXc7G3wc7Wnnasd/tcuLPZ2sqtcpLHMVEFWQSmX8ku4dLWES/klXMwpYmXCBZ797z66Ghx5IaIzMb3btYjTiBJQQgjRCqlUKmWGC1c99/X0qdNzbDRqDM52twyNn35PF9YnpvPRT6eZ+fUh3v/uBM/e3ZHf9G9v0XkJJaCEEKKN02rUPNjHj7Fh7dh6PJOPfjrNW+uS+OePp7g32BsfZzu8nO0wONni7WyHt5PSh9XUR1kSUEIIIQDlqCyyu4HI7gb2pmTxn21n2Hr8EleullBh/vW2MLGvP/Mm9G6yeiSghBBC3KJ/B3f6X1txuNxUwZWCUjLzSsjML8Z47WtIO5cmrUECSgghxG1pq/RdNW0o3czywzSEEEKIakhACSGEaJEkoIQQQrRIElBCCCFaJAkoIYQQLZIElBBCiBapTgEVHx/PyJEjGTFiBIsXL77l8b179zJu3Dh69OjBpk2bqjx28eJFnn76ae6//35Gjx7NhQsXGqdyIYQQVq3W66BMJhOzZ89m2bJlGAwGJkyYQGRkJF26dKncxtfXl7lz57J06dJbnv/666/zwgsvEB4eTkFBAWq1HLQJIYSoXa0BlZiYSGBgIAEBAQBER0ezZcuWKgHl7+8PcEv4nDp1ivLycsLDwwFwcHBotMKFEEJYt1oDymg04uNzY6Zcg8FAYmJinXaekpKCs7Mz06dP58KFCwwePJhXX30VjebW2XETEhLqUXb1GmMf1kzap2bSNrcn7VMzaZuaNbRtmnSqo/Lycvbt28fatWvx9fXllVdeYfXq1UycOLHKdn379m3KMoQQQrRCtXYIGQwGMjIyKn82Go0YDIY67dzHx4fg4GACAgLQarVERUWRlJR059UKIYRoM2oNqNDQUFJSUkhNTaW0tJS4uDgiIyPrtPPQ0FDy8vLIysoCYM+ePVX6roQQQoiaqMzm6laor2rbtm28/fbbmEwmxo8fz4svvsiCBQsICQkhKiqKxMREpk+fTl5eHra2tnh6ehIXFwfAzp07eeeddwDo2bMns2fPRqfTNe27EkII0erVKaBauvj4eObMmUNFRQUTJ07kueees3RJFvXmm2/y008/4eHhwfr16wHIycnhlVdeIS0tDT8/Pz744ANcXJpv2vyWIj09nddee40rV66gUql4+OGHmTx5srQPUFJSwqRJkygtLcVkMjFy5EhefvllUlNTmTlzJjk5OfTs2ZN58+a12T8yr/+RbjAY+M9//iNtc5PIyEgcHBxQq9VoNBpWr17d4M9Vq78o6fp1WkuWLCEuLo7169dz6tQpS5dlUQ899BBLliypct/ixYsZPHgw3333HYMHD672guu2QKPR8MYbb7Bhwwa++uorVqxYwalTp6R9AJ1Ox2effca3337L2rVr2b59OwcPHuS9997jqaee4vvvv8fZ2ZmVK1daulSL+e9//0vnzp0rf5a2qeqzzz4jNjaW1atXAw3/vdPqA+rm67R0Ol3ldVptWf/+/W/5K2XLli08+OCDADz44IP88MMPFqjM8ry9venZsycAjo6OdOrUCaPRKO2Dstz39WsVy8vLKS8vR6VSsXv3bkaOHAnAuHHj2uznKyMjg59++okJEyYAYDabpW1q0dDPVasPqOqu0zIajRasqGW6cuUK3t7eAHh5eXHlyhULV2R5Fy5cIDk5md69e0v7XGMymRg7dixDhgxhyJAhBAQE4OzsjFarXJHi4+PTZj9fb7/9Nn/4wx8qJyTIzs6WtvmVZ555hoceeoivvvoKaPjvHVnyvQ1SqVSoVCpLl2FRBQUFvPzyy/zxj3/E0dGxymNtuX00Gg2xsbHk5eUxbdo0zpw5Y+mSWoStW7fi7u5OSEgIe/bssXQ5LdIXX3yBwWDgypUrTJkyhU6dOlV5/E4+V60+oBpynVZb4uHhQWZmJt7e3mRmZuLu7m7pkiymrKyMl19+mZiYGO677z5A2ufXnJ2dGThwIAcPHiQvL4/y8nK0Wi0ZGRlt8vO1f/9+fvzxR+Lj4ykpKeHq1avMmTNH2uYm19+7h4cHI0aMIDExscGfq1Z/iq8h12m1JZGRkaxduxaAtWvXEhUVZdmCLMRsNvOnP/2JTp06MWXKlMr7pX0gKyuLvLw8AIqLi9m1axedO3dm4MCBbN68GYA1a9a0yc/X73//e+Lj4/nxxx/5+9//zqBBg3j//felba4pLCzk6tWrld/v3LmToKCgBn+urGKYeXXXabVlM2fO5JdffiE7OxsPDw9++9vfcu+99/K73/2O9PR02rVrxwcffICrq6ulS212+/btY9KkSXTt2rWyL2HmzJn06tWrzbfPsWPHeOONNzCZTJjNZkaNGsX06dNJTU3llVdeITc3l+DgYN577702O5QalAkHli5dWjnMXNoGUlNTmTZtGqD0Y44ZM4YXX3yR7OzsBn2urCKghBBCWJ9Wf4pPCCGEdZKAEkII0SJJQAkhhGiRJKCEEEK0SBJQQgghWiQJKCGEEC2SBJQQQogW6f8DL+K/gmI+X5sAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model = keras.Sequential([\n",
    "    layers.Dense(128, activation='relu', input_shape=input_shape),\n",
    "    layers.Dense(64, activation='relu'),\n",
    "    layers.Dense(1)\n",
    "])\n",
    "model.compile(\n",
    "    optimizer='adam',\n",
    "    loss='mae',\n",
    ")\n",
    "history = model.fit(\n",
    "    X_train, y_train,\n",
    "    validation_data=(X_valid, y_valid),\n",
    "    batch_size=512,\n",
    "    epochs=50,\n",
    ")\n",
    "history_df = pd.DataFrame(history.history)\n",
    "history_df.loc[:, ['loss', 'val_loss']].plot()\n",
    "print(\"Minimum Validation Loss: {:0.4f}\".format(history_df['val_loss'].min()));"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d426d6b7",
   "metadata": {
    "papermill": {
     "duration": 0.058366,
     "end_time": "2022-02-05T23:16:11.257042",
     "exception": false,
     "start_time": "2022-02-05T23:16:11.198676",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# 2) Add Capacity\n",
    "\n",
    "What is your evaluation of these curves? Underfitting, overfitting, just right?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "4c5eef9a",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-05T23:16:11.381091Z",
     "iopub.status.busy": "2022-02-05T23:16:11.380497Z",
     "iopub.status.idle": "2022-02-05T23:16:11.384445Z",
     "shell.execute_reply": "2022-02-05T23:16:11.383967Z",
     "shell.execute_reply.started": "2022-02-05T23:12:07.154033Z"
    },
    "papermill": {
     "duration": 0.069013,
     "end_time": "2022-02-05T23:16:11.384568",
     "exception": false,
     "start_time": "2022-02-05T23:16:11.315555",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"outcomeType\": 1, \"valueTowardsCompletion\": 0.25, \"interactionType\": 1, \"questionType\": 4, \"questionId\": \"2_Q2\", \"learnToolsVersion\": \"0.3.4\", \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\"}}, \"*\")"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<span style=\"color:#33cc33\">Correct:</span> \n",
       "\n",
       "Now the validation loss begins to rise very early, while the training loss continues to decrease. This indicates that the network has begun to overfit. At this point, we would need to try something to prevent it, either by reducing the number of units or through a method like early stopping. (We'll see another in the next lesson!)"
      ],
      "text/plain": [
       "Correct: \n",
       "\n",
       "Now the validation loss begins to rise very early, while the training loss continues to decrease. This indicates that the network has begun to overfit. At this point, we would need to try something to prevent it, either by reducing the number of units or through a method like early stopping. (We'll see another in the next lesson!)"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# View the solution (Run this cell to receive credit!)\n",
    "q_2.check()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "97bf5256",
   "metadata": {
    "papermill": {
     "duration": 0.058793,
     "end_time": "2022-02-05T23:16:11.502860",
     "exception": false,
     "start_time": "2022-02-05T23:16:11.444067",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# 3) Define Early Stopping Callback\n",
    "\n",
    "Now define an early stopping callback that waits 5 epochs (`patience'`) for a change in validation loss of at least `0.001` (`min_delta`) and keeps the weights with the best loss (`restore_best_weights`)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "e6482223",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-05T23:16:11.630790Z",
     "iopub.status.busy": "2022-02-05T23:16:11.628670Z",
     "iopub.status.idle": "2022-02-05T23:16:11.634600Z",
     "shell.execute_reply": "2022-02-05T23:16:11.634156Z",
     "shell.execute_reply.started": "2022-02-05T23:14:21.538422Z"
    },
    "lines_to_next_cell": 0,
    "papermill": {
     "duration": 0.072167,
     "end_time": "2022-02-05T23:16:11.634711",
     "exception": false,
     "start_time": "2022-02-05T23:16:11.562544",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"outcomeType\": 1, \"valueTowardsCompletion\": 0.25, \"interactionType\": 1, \"questionType\": 2, \"questionId\": \"3_Q3\", \"learnToolsVersion\": \"0.3.4\", \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\"}}, \"*\")"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<span style=\"color:#33cc33\">Correct</span>"
      ],
      "text/plain": [
       "Correct"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from tensorflow.keras import callbacks\n",
    "\n",
    "# YOUR CODE HERE: define an early stopping callback\n",
    "early_stopping = callbacks.EarlyStopping(min_delta=0.001, patience=5, restore_best_weights=True)\n",
    "\n",
    "# Check your answer\n",
    "q_3.check()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "bbcdc67c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-05T23:16:11.759289Z",
     "iopub.status.busy": "2022-02-05T23:16:11.758653Z",
     "iopub.status.idle": "2022-02-05T23:16:11.762135Z",
     "shell.execute_reply": "2022-02-05T23:16:11.761683Z",
     "shell.execute_reply.started": "2022-02-05T23:14:23.817598Z"
    },
    "papermill": {
     "duration": 0.067728,
     "end_time": "2022-02-05T23:16:11.762252",
     "exception": false,
     "start_time": "2022-02-05T23:16:11.694524",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Lines below will give you a hint or solution code\n",
    "#q_3.hint()\n",
    "#q_3.solution()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9a109842",
   "metadata": {
    "papermill": {
     "duration": 0.060216,
     "end_time": "2022-02-05T23:16:11.882777",
     "exception": false,
     "start_time": "2022-02-05T23:16:11.822561",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Now run this cell to train the model and get the learning curves. Notice the `callbacks` argument in `model.fit`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "458e347e",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-05T23:16:12.014660Z",
     "iopub.status.busy": "2022-02-05T23:16:12.013833Z",
     "iopub.status.idle": "2022-02-05T23:16:15.122797Z",
     "shell.execute_reply": "2022-02-05T23:16:15.122370Z",
     "shell.execute_reply.started": "2022-02-05T23:14:28.510942Z"
    },
    "papermill": {
     "duration": 3.179438,
     "end_time": "2022-02-05T23:16:15.122934",
     "exception": false,
     "start_time": "2022-02-05T23:16:11.943496",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/50\n",
      "49/49 [==============================] - 1s 5ms/step - loss: 0.2187 - val_loss: 0.2071\n",
      "Epoch 2/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1974 - val_loss: 0.2028\n",
      "Epoch 3/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1928 - val_loss: 0.2027\n",
      "Epoch 4/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1908 - val_loss: 0.2025\n",
      "Epoch 5/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1901 - val_loss: 0.1999\n",
      "Epoch 6/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1876 - val_loss: 0.2023\n",
      "Epoch 7/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1864 - val_loss: 0.2012\n",
      "Epoch 8/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1856 - val_loss: 0.1995\n",
      "Epoch 9/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1842 - val_loss: 0.1995\n",
      "Epoch 10/50\n",
      "49/49 [==============================] - 0s 3ms/step - loss: 0.1833 - val_loss: 0.1991\n",
      "Minimum Validation Loss: 0.1991\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABATElEQVR4nO3dfVxUZf7/8dcMw8id3MMAiogCioJSUmoiFqYYSGhq7dpam7ndb5ab281u7G7fLTe1Nldb/bkVbWtt21pGiWbFpmjeo4UkipYo3jDecKdyP3N+fxwcJTBABmaAz/PxmAfNOWfOfM6V8va6zjnX0SiKoiCEEELYGa2tCxBCCCGaIwElhBDCLklACSGEsEsSUEIIIeySBJQQQgi7JAElhBDCLrUqoLKzs0lMTGTChAmsXLmyyfr09HSSkpJISUnh3nvv5cSJEwDk5+dz1113kZycTEpKCuvWrbN8pqioiBkzZjBhwgSeeOIJamtrrXRIQgghugNNS/dBmUwmEhMTSU9Px2AwMH36dF599VXCwsIs22zfvp3hw4fj7OzMe++9x86dO3nttdc4cuQIGo2G/v37YzQamTZtGuvWrcPd3Z25c+cyceJEkpOTSUtLY/DgwcycObPRd+fk5HTMUQshhLArI0aMaLJM19KHcnNzCQkJITg4GIDk5GSysrIaBdSoUaMs/x0TE8Mnn3wCQGhoqGW5wWDA29ubkpISevfuzfbt23nllVcAmDp1KsuWLWsSUFcruq3y8/OJjIxs9356CmmvtpH2ajtps7bp7u11tc5IiwFlNBoJCAiwvDcYDOTm5l51+9WrVxMfH99keW5uLnV1dfTr14/S0lLc3d3R6dSvDwgIwGg0Nru//Pz8lkpsUXV1tVX201NIe7WNtFfbSZu1TU9trxYDqi0yMjLIy8tj1apVjZafPn2a+fPn8/LLL6PVtu26DGv8q6G7/+vD2qS92kbaq+2kzdqmu7fXNfegDAYDxcXFlvdGoxGDwdBku61bt7JixQpWrVqFXq+3LL9w4QIPPvggTz75JDExMQB4eXlRUVFBfX09Op2O4uLiZvcphBCi52qxOxMdHU1hYSFFRUXU1taSmZlJQkJCo232799PWloay5cvx8fHx7K8traWRx99lNTUVCZNmmRZrtFoGDlyJBs2bABgzZo1TfYphBCiZ2uxB6XT6UhLS2POnDmYTCamTZtGeHg4S5YsISoqivHjx7Nw4UIqKyuZO3cuAIGBgaxYsYL169eze/duysrKWLNmDQB/+ctfiIyMZP78+Tz55JO89tprREZGMmPGjI49UiGEEF1Kq85BjRs3jnHjxjVadimMAN5+++1mP5eamkpqamqz64KDg1m9enUryxRCCNHTyEwSQggh7JIElBBCdHHXXXedrUvoEN06oC7W1PPb1d9SVm2ydSlCCCHaqFsHVHlVHR/sPs7GHy7YuhQhhOhwiqLw8ssvM3ny5Ebzn54+fZq7776b1NRUJk+ezO7duzGZTDzzzDOWba92LYEtWfVGXXsT5OnMAF9Xck5W2roUIUQP8GHOcT7YXWTVfd4ZG8wQl9Zt+/nnn3PgwAEyMjIoLS1l+vTpxMbGsnbtWuLi4nj44YcxmUxUVVWRn5+P0Whk7dq1AFRUVFi1bmvo1j0ogPgIP/YVV1NdJ8N8QojuLScnh+TkZBwcHPD19eWGG25g3759REdH89FHH7F06VIKCgpwc3MjODiYoqIi/u///o/s7Gzc3NxsXX4T3boHBRAf4cvbWwvZXVhKXLivrcsRQnRj00b0ZdqIvlbfb3vn4bvhhhtYtWoVmzZt4plnnuG+++5jypQpZGRksGXLFt5//33Wr1/PggULrFSxdXT7HtSoAT7otJB96IytSxFCiA4VGxvL+vXrMZlMlJSUsHv3boYNG8aJEyfw9fXlzjvvZMaMGXz33XeUlJSgKAqJiYk88cQT7N+/39blN9Hte1Aueh1D/Z3ILjjDc0ndd7JFIYSYMGECe/fuJTU1FY1Gw/z58/Hz82PNmjW8+eab6HQ6XFxcePnllzl9+jTPPvssZrMZgHnz5tm4+qa6fUABXB/kQvqeEowV1RjcnWxdjhBCWNXevXsBdZ7Tp59+mqeffrrR+qlTpzJ16tQmn7s0BZ296vZDfAAj+jgDsPnQWRtXIoQQorV6RECFeunxdetFdoGchxJCiK6iRwSUVqMhPtyXLYfPYjYrti5HCCFEK/SIgAL1fqiSi7XknSy3dSlCCCFaoccE1KV7oGSYTwghuoYeE1C+br2I6uNOdoFcKCGEEF1BjwkogPhwP/YcK+V8dZ2tSxFCCNGCnhVQEX7UmxW2fn/O1qUIIYTN/NTzo44fP87kyZM7sZqr61EBdX0/L1z1DnIeSgghuoAeMZPEJXqdltEDfeSGXSFEx/jm37B3lXX3ed0voFfMT26yePFiAgMDufvuuwFYunQpDg4O7Nixg4qKCurr65k7dy633nprm766pqaGP/7xj+Tl5eHg4MAzzzzDqFGjOHToEM8++yx1dXWYzWaWLl2Kv78/TzzxBMXFxZjNZh555BGSkpKu9aiBHhZQoA7zfZl/msKzF+nv62rrcoQQot2SkpJ46aWXLAG1fv163nzzTe655x7c3NwoKSnhrrvuYvz48Wg0mlbv99133wXg008/5fvvv+f+++9nw4YNvP/++9xzzz3cfvvt1NbWYjab2bRpE/7+/qxcuRKA8+fPt/u4el5AhfsB6uzmElBCCKuK+bn6srYWHrcxZMgQzp07h9FopLS0FHd3d3x9fVmwYAG7du1Cq9ViNBo5e/Ysfn5+rf7anJwcfvGLXwAwcOBAgoKCOHLkCDExMaxYsYLi4mImTpxI//79iYiI4OWXX2bRokXccsstxMbGtuuQoYedgwLo7+tKP28XOQ8lhOhWJk2axIYNG1i3bh1JSUl8+umnlJSU8NFHH5GRkYGvry81NTVW+a6UlBSWL1+Ok5MTDzzwANu2bSM0NJSPPvqIiIgIXnvtNZYtW9bu7+lxAQXqQwy3fX+O2nqzrUsRQgirSEpKYt26dWzYsIFJkyZx/vx5fHx8cHR0ZPv27Zw4caLN+4yNjeXTTz8F4MiRI5w6dYoBAwZQVFREcHAw99xzD+PHj+fgwYMYjUacnZ1JTU3l/vvvt8rzpVoVUNnZ2SQmJjJhwgTL+OKV0tPTSUpKIiUlhXvvvbdRQ9x///3Exsby4IMPNvrMM888Q0JCAqmpqaSmprb7iZFtER/ux8VaEzlHSzvtO4UQoiOFh4dz8eJF/P398ff3JyUlhby8PFJSUsjIyGDAgAFt3ufMmTNRFIWUlBSefPJJFixYgF6vZ/369UyePJnU1FQKCgqYMmUKBQUFTJ8+ndTUVJYtW8bDDz/c/oNSWlBfX6+MHz9eOXbsmFJTU6OkpKQohw4darTNtm3blMrKSkVRFOXdd99V5s6da1m3detWJSsrS3nggQcafebpp59W1q9f/5PfvXv37pbKa5X9+/c3el9RVasMfDZT+cv6fKvsv7v5cXuJnybt1XbSZm3T3dvrar/rW+xB5ebmEhISQnBwMHq9nuTkZLKyshptM2rUKJyd1WcuxcTEUFxcbFk3evRoXF3t62KE3k6OXN/PS85DCSGEHWvxKj6j0UhAQIDlvcFgIDc396rbr169mvj4+FZ9+V//+ldef/11Ro8ezVNPPYVer2+yjTWG/qqrq5vsZ7CXwjt7K9i6Zx9ezj3uYsaf1Fx7iauT9mo7abO26Yj2Kiws5LXXXmu0zNHRkUWLFln1e9rDqr+ZMzIyyMvLY9Wqlm9UmzdvHn5+ftTV1fH888+zcuVKHnvssSbbRUZGtruu/Pz8JvuZ3ruMd/Z+zWmNNzdF9mn3d3QnzbWXuDppr7aTNmubjmivyMhIbrvtNqvu81rl5OQ0u7zFIT6DwdBoyM5oNGIwGJpst3XrVlasWMHy5cub7Qn9mL+/PxqNBr1ezx133MG+ffta/Iw1RQV54O2ql2E+IYSwUy0GVHR0NIWFhRQVFVFbW0tmZiYJCQmNttm/fz9paWksX74cHx+fVn3x6dOnAVAUhS+//JLw8PBrKP/aabUa4sJ8yT4kT9kVQgh71OIQn06nIy0tjTlz5mAymZg2bRrh4eEsWbKEqKgoxo8fz8KFC6msrGTu3LkABAYGsmLFCkC9TPGHH36gsrKS+Ph4XnzxRcaOHctTTz1FaWkpiqIwePBg/vSnP3XskTYjPsKPT749SX5xBUODPDr9+4UQQlxdq85BjRs3jnHjxjVadimMAN5+++2rfva9995rdvk777zTmq/uUPGWp+yelYASQgg70yNnkrjE392JwQG95TyUEELYoR4dUADjIvzYfbSEizX1ti5FCCHEFXp8QI0N96POpLD9B3nKrhBC2JMeH1Cx/b1wctTKMJ8QQtiZHh9QTo4OjBogT9kVQgh70+MDCtTZzX84e5GikkpblyKEEKKBBBTq/VCgPmVXCCGEfZCAAgb6udLH01nOQwkhhB2RgAI0Gg3xEb5sPXyOOpM8ZVcIIeyBBFSD+HA/ztfU801Rma1LEUIIgQSUxU1hvmg1yDCfEELYCQmoBh7OjsQEe0pACSGEnZCAukJ8hB+5J8opvVhr61KEEKLHk4C6QnyEH4oCWw7LTbtCCGFrElBXGN7XEw9nRxnmE0IIOyABdQUHy1N2z6Ao8pRdIYSwJQmoH4mP8MVYUUOB8YKtSxFCiB5NAupHLNMeyTCfEELYlATUjwR6OBPu7ybz8gkhhI1JQDUjPsKPHUdKqKo12boUIYTosSSgmjE23JfaejM7jshTdoUQwlYkoJoxMtQHvU4rDzEUQggbkoBqhrPegZGh3nKhhBBC2FCrAio7O5vExEQmTJjAypUrm6xPT08nKSmJlJQU7r33Xk6cOGFZd//99xMbG8uDDz7Y6DNFRUXMmDGDCRMm8MQTT1Bba1/TC8WH+3Ho9AVOllXZuhQhhOiRWgwok8nECy+8wBtvvEFmZiZr167l8OHDjbaJjIzkww8/5NNPPyUxMZFFixZZ1s2ZM4eFCxc22e/ixYv55S9/yRdffIG7uzurV6+2wuFYz6XLzTfL1XxCCGETLQZUbm4uISEhBAcHo9frSU5OJisrq9E2o0aNwtnZGYCYmBiKi4st60aPHo2rq2uj7RVFYfv27SQmJgIwderUJvu0tQiDGwHuTmQXyHkoIYSwBV1LGxiNRgICAizvDQYDubm5V91+9erVxMfH/+Q+S0tLcXd3R6dTvz4gIACj0djstvn5+S2V2KLq6upr2k+0v45NB43kfbcfB62m3XV0FdfaXj2VtFfbSZu1TU9trxYDqi0yMjLIy8tj1apVVttnZGRku/eRn59/Tfu5vdaDLw7vpbZ3INf382p3HV3FtbZXTyXt1XbSZm3T3dsrJyen2eUtDvEZDIZGQ3ZGoxGDwdBku61bt7JixQqWL1+OXq//yX16eXlRUVFBfX09AMXFxc3u09biwnzRyFN2hRDCJloMqOjoaAoLCykqKqK2tpbMzEwSEhIabbN//37S0tJYvnw5Pj4+LX6pRqNh5MiRbNiwAYA1a9Y02ac98HLVM6yPhwSUEELYQIsBpdPpSEtLY86cOSQlJXHbbbcRHh7OkiVLLBc2LFy4kMrKSubOnUtqaioPPfSQ5fMzZ85k7ty5bNu2jfj4eDZv3gzA/PnzSU9PZ8KECZSVlTFjxowOOsT2iY/w45uiMsqr6mxdihBC9CitOgc1btw4xo0b12jZ3LlzLf/99ttvX/Wz7733XrPLg4OD7e7S8ubER/ix9H+H2Xr4LLdFB9q6HCGE6DFkJokWxAR70ruXTmY3F0KITiYB1QJHBy03hfmQXXBWnrIrhBCdSAKqFeIj/DhRVsX3Zy7auhQhhOgxJKBaIT5cnrIrhBCdTQKqFYK9XRjg6yrnoYQQohNJQLXS2HBftv9wjuo6ecquEEJ0BgmoVoqP8KO6zszuwlJblyKEED2CBFQrjRrgg6ODRh6/IYQQnUQCqpVce+mIDfFmk1woIYQQnUICqg3iI/w4UHye0xXVti5FCCG6PQmoNoiP8AUg+5A8xFAIITpa9w8os9lqu4oMcMfXrZfcDyWEEJ2gewfUeSO8GIBhz1+hvqbdu9NqNcSH+7Ll8FnMZpn2SAghOlL3Dig3f4idjfeh/8CbE6Hkh3bvMj7Cj5KLteSdLLdCgUIIIa6meweURgO3/YWiMS9D6RFYEQ95H7Zrl3HhDeehZJhPCCE6VPcOqAYX+o6Dh7aAfySsng2fzoW6qmval69bL4YGuZNdIBdKCCFER+oRAQWAZz+4bx2MeQJy3oZ/jIczB69pV/ERfuw5Vsr5annKrhBCdJSeE1AADo4w4U9w94dwwQgrb4Zvmn/i70+JD/ej3qyw7ftz1q9RCCEE0NMC6pLwW9Uhvz4j4OOH4aMHoeZCqz8+IsQLV72DzG4uhBAdqGcGFIB7INyTATc/C7n/UXtTxfta9VG9TsvogT5yHkoIITpQzw0oAK0D3PwM3PsJ1JxXz0vtehNa8Wj3+Ag/jpVUUnhWnrIrhBAdoWcH1CWh8eqQX+hYyJwH//0lVP/0fU6Wp+zKMJ8QQnQICahL3Pxg5n/h1j9B/qewYiycyLnq5v19Xenn7SL3QwkhRAdpVUBlZ2eTmJjIhAkTWLlyZZP16enpJCUlkZKSwr333suJEycs69asWcPEiROZOHEia9assSyfNWsWiYmJpKamkpqayrlzdnBFnFYLcU/AfetBMcObibDt9asO+Y0N92Xb9+eorbfefH9CCCFULQaUyWTihRde4I033iAzM5O1a9dy+PDhRttERkby4Ycf8umnn5KYmMiiRYsAKCsrY9myZXzwwQf897//ZdmyZZSXXx46W7x4MRkZGWRkZODj42PlQ2uHfiPhwWyISIQNz8G/fw6VJU02i4/w42KtiZyj8pRdIYSwthYDKjc3l5CQEIKDg9Hr9SQnJ5OVldVom1GjRuHs7AxATEwMxcXFAGzZsoUxY8bg6emJh4cHY8aMYfPmzR1wGB3AxRvuWgW3LYTvs2BFHBzd1miTmwb6oNPKU3aFEKIjtBhQRqORgIAAy3uDwYDRaLzq9qtXryY+Pr5Vn33uuedITU3l9ddfR2nFlXOdTqOBkQ/C/Z+Dgx7eTobNr1ge4dHbyZHr+3nJhRJCCNEBdNbcWUZGBnl5eaxatarFbRcvXozBYODChQs8/vjjZGRkMGXKlCbb5efnt7uu6urqdu7HCe0tbxCw+y94ZL3AhbzPODnqD5icfBjspfDO3gq27cnD09mh3bXag/a3V88i7dV20mZt01Pbq8WAMhgMliE7UHtFBoOhyXZbt25lxYoVrFq1Cr1eb/nszp07G332xhtvtKwDcHNzY/LkyeTm5jYbUJGRkW07ombk5+dbZT9E/xf2vIPb+t8S8eV9MO0fTB8Twzt7v8ao8WJ0ZJ/2f4cdsFp79RDSXm0nbdY23b29cnKav2K6xSG+6OhoCgsLKSoqora2lszMTBISEhpts3//ftLS0li+fHmjix3i4uLYsmUL5eXllJeXs2XLFuLi4qivr6ekRL3ooK6ujo0bNxIeHt6e4+scGg2MuBd+9T9w9oJ3phB9cBl+Lg5yubkQQlhZiz0onU5HWloac+bMwWQyMW3aNMLDw1myZAlRUVGMHz+ehQsXUllZydy5cwEIDAxkxYoVeHp68sgjjzB9+nQAHn30UTw9PamsrGTOnDnU1dVhNpsZPXo0d955Z8ceqTUZhsIDX8G636LZvIh/Ow/j8YLHMJuHo9VqbF2dEEJ0CxrFLq9OUOXk5DBixIh276dDu8ff/of6T+ZSUe/A+duWEjLqjo75nk7U3YcTrE3aq+2kzdqmu7fX1X7Xy0wS7TX8LspmfUmx4kPIZ/fB57+H+lpbVyWEEF2eBJQV+PaP4mmvV/ncJQW2LoX026D0qK3LEkKILk0CykpGD+rDo+UzqZmaDmcL1Ln89mfYuiwhhOiyJKCsJD7cjzqTwte9xsBDm8E3DD64BzKfgrpqW5cnhBBdjgSUlcT298LJUas+xNCrP9z3GYx+DHb9A968Fc4ebnEfQgghLpOAshInRwdGDfC5fD+UTg+JL8LP/wPlx2HlOMj9wLZFCiFEFyIBZUXx4X78cPYiRSWVlxcOmgQPfQ0Bw+CjX0HGo1ArT+EVQoiWSEBZUXzEVZ6y69EH7v0U4ufD3nfhHwlwuufNqyWEEG1h1clie7qBfq4EeTiRXXCGu0eGNF7poIOE30PIGPjoAVh5CwQOB60OtA4NP3Xg4Nj4vfbH73Xqvq58r3Vo2E7XeH8Ojj/a/4/fN/d9OhwvHIcKD3DoBbqGl1anTvUkhBCdRALKijQaDfERfmTmnqLOZMbRoZkO6sBb4KEtkPUn9dyU2QT1NWC+AOZ69b25Xn2Z6hq/b+7VAcIAMpscHeic1HNrOqfG4eWg/9G6K9479Lr6Ost+WrmuV281TIUQPYIElJXFR/jx/q4ivikq44b+3s1v1NsAU/7e/i9TlGYCzATmuh+9vxR2Vwbg1bc5WXSUIH9vMNWq4VlfA6YaqK9WZ8mor25mXQ3UVqpPHjbVXt720rr6GvU728PZG2LvgxvmgHtQ+9tPCGHXJKCsbMxAX7Qa2Fxw5uoBZS0ajTrc52Dd/43lunyCOmLeL7O5cdBdGV4/DsFG66rVn8e2weZX4eslMGQKjHoE+rZ/rsZux1QHB9fDwXXgZoCAaDBEgU+Y1f+sCNGR5E+rlXm4OBIT7MmmQ2eZN3GQrcuxL1otaJ3B0fnaPn/TY1ByBHb+A/a8A3mroe+NMOohiLxdPZ/Wk5UehT3/hL2r4IJRfSRMzYXLPVedE/gNVsMqIEr9aRgKLh38DykhrpEEVAeIj/BjSdYhSi/W4uWqt3U53Yt3KEx6CW5+Br55D3asgNWzwb0P3PgruP7envUL11QHBZ9BzttwOEvtVYdPhBH3QfgEdfj2bAEY89RXcR4c2gDfXPHUa/c+jUMrIBq8B8j5PmFzElAdID7Cj9e+PMSWw2dJGS7nSjqEk7vac7rxV3Doc9j+d/jyj7DxZRj+Mxj1MPh14x5s2THIudRbKobeQTDuabh+Fnj0vbyd1kENnoCoxp8/bwTjPjB+p4aWMQ++z7p84Y3OGfwjG0IrWu1pGYaCs2enHaIQElAdYHhfTzycHckuOCMB1dG0DjDoNvVVnKf2qL55D3LSYeB4NagGjleHF7s6U/0VvaUv1WXhEyH2NQib0LbzS70N6ivs1svL6mvgzMHLPS3jPshfqw6nXuLRryG0hl7ubXmFdo/2FXZHAqoDOGg1xIX5kn3oDIqioJH7hzpHQBSkLoNb/wi702HXG/DudPAJV3tbw38OeldbV9l2Zcdgz79g77/g/CnoHQjjfgvXzQLPYOt9j64XBA5TX5coCpwvbgitfZfDq+AzUMzqNo4u4D+k8RCh/xC1lytEO0hAdZCx4b5k7jtFgfECgwJ627qcnsXVF8bNhzFzYf/H6vBf5m8g6wX1HNWND1j3F3tHMNWrQ5c56XDoC3VZ+ARIfgXCEzvvajyNBtwD1Vf4hMvL66rgzIHLw4PFefDdGrV3d4lnyOUrCC/1ujz7S29LtJoEVAexTHtUcEYCylZ0ehh2J0TPgKKdalBtWwbbXofIFHX4L3ikfc2QUX5cHVLb8y84f1LtLcXPV88tefazdXWXOTpD0HXq6xJFgYoTl4cHL53fOpAJKOo2+t5gGIKf22BwnQl9b5CLMcRVSUB1kCBPZ8L83cg+dIZfxQ+wdTk9m0YD/Uaqr7Ii9REoOW+rvaug69T7qYZMUQPNFkz1cPgLdVjy8BfqL/qwWyFpEURM6jr3Lmk06gUaHn3VSZIvqa1U5568FFqnvsXnwCrI/6d683XYrRCRCGHj1UvjhWjQRf7kd03x4X68u+Mo1XUmnBzlX4l2wTMYJrygXvH27b9h+wp1lvnPn1dnqIi9Tx0i7Azlxy+fW6o4od5UGzcPrr8HvEJa/nxXoXdRb6i+4qbqgm93MsjhOBRsUEN53wegcYB+o9QLPyImqVdh2lPvVnQ6CagOFB/hy1tfH2HHkRLGNQz5CTuhd1UDacRs+P5/6vDfV3+G7EXqsOCoh9VzJtZmNqnnlHLS1XNMigIDE+C2lxt6Sz3jZmOzvjdE3gFRd6htcmKPeuHFoQ3w5R/Ul2c/tU3CE6F/HDg62bps0ckkoDrQyFAf9Dot2QVnJKDslVYL4beqrzMHGy5T/7faqwmNV4f/whPbf2K//IS6zz3/gorjDb2lJxt6S/2tcihdltYBgm9QX+OfV9vq0Odq72rPv2DnSvVKwQE3N/SuEmUuxh5CAqoDOesdGBnqffkpu8K++Q2CyX+FhOfVKYN2/gP+/TP1Pp+RD8F1d6szqreW2aTer5TzdsNl2Q29pUkL1Pu2ekhvqc08+qhDrbH3qVcLFm5Rw6pggzq/IKhXB17qXfW5Xi606KZa9c/C7OxsEhMTmTBhAitXrmyyPj09naSkJFJSUrj33ns5ceKEZd2aNWuYOHEiEydOZM2aNZbleXl5pKSkMGHCBP785z+jKIoVDsf+xIf7cej0BU6WVdm6FNFaLt5q72butzA9HVz94LOn4dUh8Nmz6nyAP6XiJGxaCEuGw3t3wvHdMOYJmPsNzPoIhsi8ga3m6Nxwef1ieCIXHtmu3uem761OHPzmrbA4AtY8pF7mXl1u64qFNSktqK+vV8aPH68cO3ZMqampUVJSUpRDhw412mbbtm1KZWWloiiK8u677ypz585VFEVRSktLlYSEBKW0tFQpKytTEhISlLKyMkVRFGXatGnK3r17FbPZrNx///3Kxo0bm3z37t27WyqvVfbv32+V/VyLA6cqlJCn1yrv7zxqsxraypbtZbeKdivK6vsV5U/eivIHD0X590xFObJZUcxmtb1M9YpS8LmivPdzRfmjl6L8wV1R/pmqKN99rCj1tbau3u5Y5c/YxXOKkvtfRVk9R1H+EqK2+Z+8FSU9WVG+/puinD6oKGZz+7/HDnT3v5NX+13f4hBfbm4uISEhBAerNzYmJyeTlZVFWFiYZZtRo0ZZ/jsmJoZPPvkEgC1btjBmzBg8PT0BGDNmDJs3b+bGG2/kwoULxMTEADBlyhSysrIYN26ctXLXbkQY3DC49yK74Cx33WBH97GItuk7Avq+oV4BuOtN2P0WHFgLhmj8PYfBZ5uh/Jja2xrzuHpDsHeoravu3ly8IXq6+jKb4PgudSi14HP4/PfqyytUPWcVkag+zVrXy9ZVizZoMaCMRiMBAQGW9waDgdzc3Ktuv3r1auLj46/6WaPR2GR5QEAARqOx2f3l5+e3fBQtqK6utsp+rtUwP0c2HTSS991+HLT2f9msrdvL7gVNR5M0GY+jG/Aq+A8+xne5YLiBsqEPcT5orDp8Z6wGo7Th1XTMnzEP6HMX9LkL3cVi3E59jdvJrbjuTke7YwUmnQsXDTdwIWgMFwNvot65k24nsIKe+nfSqhdJZGRkkJeXx6pVq1reuJUirfDgvPz8fKvs51ql1Hrwxfd7qesdSFQ/+78R0dbt1WVEXwfK0xzcl8OgYbG42bqeLqTj/4xFAreo/1lbCYWbcSj4DPeCz3HftUldHhijXmgRMRECr7PrKZi6+9/JnJycZpe3GFAGg4Hi4mLLe6PRiMFgaLLd1q1bWbFiBatWrUKv11s+u3PnzkafvfHGG5vss7i4uNl9dhdjw3zRaCC74CzXdYGAEm2g0WB27IIT0PYkepfLw3yKAqf3Xx4KzF4Im/4Crv7qJez+g9VtFHPDy/Sj9829Wlr/o23MpjbvI7iqBvL6qjNtuHirP529m7538ug6M4+0QotHEh0dTWFhIUVFRRgMBjIzM3nllVcabbN//37S0tJ444038PHxsSyPi4vj1VdfpbxcvbJmy5YtzJs3D09PT9zc3Pjmm28YPnw4H3/8MbNmzbLyodkPL1c9w/p4kH3oDHNvDbd1OUL0XBrN5Wdbjf0NVJaotwIUfAYHPm38IMdmP6/9iZemhfUtva7+eYe6i1Ccq9ZbXXZ5JvnmOHk0BNZVQqzR+4aXk6dd9iBbDCidTkdaWhpz5szBZDIxbdo0wsPDWbJkCVFRUYwfP56FCxdSWVnJ3LlzAQgMDGTFihV4enryyCOPMH36dAAeffRRywUTf/jDH3j22Weprq4mPj7ect6qu4qP8OPvG7+nvKoOD2e5xFgIu+Dirc4cMuxOtWdTV/nTAWIjhVcO8ZnNUFMOVaVQWar+rCpp+FmqhtiVy0p+UH9Wl2OZtLcJjfowyhZDzbPx+17uHdouGkWx3xuQcnJyGDFiRMsbtsAexm93FZYwY8U2lt99PbdFB9q0lpbYQ3t1JdJebSdt1jZWaS+zSQ2pHwdY5RXh1tyymoqr79MtAB7fqw6jtsPVftd3n8FKOxcT7EnvXjqyD52x+4ASQnRDWge11+Pi3bbPmeqgqqz5ANO7gK7j5kiUgOokjg5abgrzIbvgrDxlVwjRdTg4gpuf+upk9ndWrBsbG+7HibIqvj9z0dalCCGE3ZOA6kTjrnjKrhBCiJ8mAdWJgr1dCPV1ZfMhCSghhGiJBFQniw/3ZfsPJdTUm2xdihBC2DUJqE4WH+FHVZ2J3YWlti5FCCHsmgRUJxs1wAdHB42chxJCiBZIQHUy1146YkO82SQBJYQQP0kCygbiI/w4UHye0xXVti5FCCHslgSUDYwNV59D89+c4zauRAgh7JcElA0MCXQnLsyXRRsO8vuP98kVfUII0QwJKBvQajWk33cDD8QPYNX2Y9y5YhvHSyttXZYQQtgVCSgbcXTQ8lxSJCt+MYIfzlxk8tItbDx42tZlCSGE3ZCAsrFJUQF88us4AtyduO/tXbz6RQEms90+AUUIITqNBJQdCPV1Zc0jY5h6XR/+lnWIX6bvpORira3LEkIIm5KAshPOegdemTGcBXdEs+NICZP/tpm9x2S2CSFEzyUBZUc0Gg0/v7EfHz50E1qthjv/3zbe2VaIHT/0WAghOowElB2K7uvB2l/HMTbcj7SM75j7/jdcrKm3dVlCCNGpJKDslKeLnjfuiWV+4iDW5p4k9fWvOXz6vK3LEkKITiMBZce0Wg2P3hLGv+4fSenFWm5f9jWffnvS1mUJIUSnkIDqAsaE+bL28TgiA9359b/38sdPvqO23mzrsoQQokNJQHURgR7OvP/AKGaPCeXtrYX8bOU2TpVX2bosIYToMK0KqOzsbBITE5kwYQIrV65ssn7Xrl1MnTqVIUOG8NlnnzVat2jRIiZPnszkyZNZt26dZfkzzzxDQkICqamppKamkp+f385D6f4cHbSkpQxh2czrOFh8nuS/bWHLobO2LksIITqErqUNTCYTL7zwAunp6RgMBqZPn05CQgJhYWGWbQIDA1mwYAFvvfVWo89u3LiR/fv38/HHH1NbW8usWbOIj4/Hzc0NgN/+9rdMmjTJyofU/U0eFsTgAHceXpXDrLd2MO/WCB69JQytVmPr0oQQwmpa7EHl5uYSEhJCcHAwer2e5ORksrKyGm3Tt29fBg8ejFbbeHeHDx8mNjYWnU6Hi4sLgwYNIjs727pH0EOF+bvx8aNjuH14EK98UcD9/9xFWaXMPiGE6D5a7EEZjUYCAgIs7w0GA7m5ua3a+eDBg1m2bBmzZ8+mqqqKHTt2NOp5/fWvf+X1119n9OjRPPXUU+j1+ib7sMbQX3V1dbcdQnxwWC/69vLh/+06w8RXvuJ3NxuI8O3Vrn125/bqCNJebSdt1jY9tb1aDKj2iIuLY9++ffzsZz/D29ubmJgYSy9r3rx5+Pn5UVdXx/PPP8/KlSt57LHHmuwjMjKy3XXk5+dbZT/2asgQmBBbxqPv7mH+Z6f4w+1DmHljPzSaaxvy6+7tZW3SXm0nbdY23b29cnJyml3e4hCfwWCguLjY8t5oNGIwGFr9xQ8//DAZGRmkp6cDEBoaCoC/vz8ajQa9Xs8dd9zBvn37Wr1P0VRMsCef/jqOUQN9+N2aPH7zwbdU1cqDEIUQXVeLARUdHU1hYSFFRUXU1taSmZlJQkJCq3ZuMpkoLVUnPD1w4AAHDx5kzJgxAJw+rT77SFEUvvzyS8LDw6/1GEQDb1c96b+8gSduDWfNNyeY8vrX/HDmgq3LEkKIa9LiEJ9OpyMtLY05c+ZgMpmYNm0a4eHhLFmyhKioKMaPH09ubi6PPfYYFRUVfPXVVyxdupTMzEzq6+u5++67AXBzc2PRokXodOpXPvXUU5SWlqIoCoMHD+ZPf/pTxx5pD+Gg1fDErRFc18+Lue/v5fZlX7N4xjAmRQXaujQhhGgTjWLHU2Xn5OQwYsSIdu+nu4/fXs2JsioeeXcP3xaV8auxofx20mAcHVq+9a2ntte1kvZqO2mztunu7XW13/Uyk0Q31sfTmQ8eHMU9o0P4x+YjzPzHdowV1bYuSwghWkUCqpvrpXPghdQolvwshrwTFST/bQvbvj9n67KEEKJFElA9RGpMHzIeG4O7s46739jO3zcexmy229FdIYSQgOpJIgy9+eSxOG6LDmThZwd54F85lFfV2bosIYRolgRUD+PWS8eyn19H2uQhbDx4mpSlW/juZLmtyxJCiCYkoHogjUbD7LhQ/vPgKGrrzdzx9618sKvI1mUJIUQjElA92IgQb9Y+Hkdsfy9++2Euv139LdV1MvuEEMI+SED1cL5uvXhn9kgeuyWMD3Yf546/b+VISY2tyxJCiI6dLFZ0DQ5aDU8lDuL6EE+e/M+3PPJpBYN2VpAUHUjysEDC/N1sXaIQogeSgBIWCYMNZP1mHG9+vpec0wqvZRXw1y8LGBzQm+ToQJKGBTLQT8JKCNE5JKBEI75uvbg90oOn74ikuLya9XmnWLfvFK98UcArX6hhNXlYIEnRgQyQsBJCdCAJKHFVAR5O3DcmlPvGhHKqvIr1+4rJ3HeKxZ8XsPjzAiID3S1hFerrautyhRDdjASUaJVAD2dmx4UyOy6Uk2VVrM8rJjP3JIs2HGTRhoMMCXQneVggydGB9JewEkJYgQSUaLMgT2fujwvl/oawWrfvFJn7TlnCKqqPu3qBRXQgIT4SVkKIayMBJdolyNOZOWMHMGfsAI6XVlqGARd+dpCFnx0kuo+HpWcV7O1i63KFEF2IBJSwmr5eLvwqfgC/ih9AUUkl6/NOkbmvmL+sP8Bf1h9gWF8P9WpACSshRCtIQIkOEeztwgPxA3kgfiBFJZWWYcAF6w+wYP0BhvdVe1ZJ0YH09ZKwEkI0JQElOlywtwsPjhvIg+MGcuxcJevyTpGZe4qX1h3gpXUHGB7syeSG+6z6eDrbulwhhJ2QgBKdqp+PCw+NG8hDDWGVue8UmftO8uK6fF5cl891/TxJjg7ktmgJKyF6OgkoYTP9fFx4+OaBPHzzQArPXrT0rP6cmc+fM/O5vp8nSQ3nrIIkrITocSSghF3o7+vKIzeH8cjNYRw5e1E9Z3VFWI0I8WJchB/B3s708XQhyNOJAHcndA4y37EQ3ZUElLA7ob6uPHpLGI/eEsYPZy6wbt8p1uae4tUvChptp9VAgLsTQZ7OBHk608er4aenkyXEejs52ugohBDtJQEl7NoAPzceSwjnsYRwqmpNnCyv4kRpFSfL1Nfxhp/fFJWxPu8UdSal0ed7O+no4+lMn4YQuxRkfTzVYPPv7YSDVmOjoxNC/JRWBVR2djYvvvgiZrOZGTNm8MADDzRav2vXLl566SUOHjzIq6++yqRJkyzrFi1axKZNmwB45JFHSEpKAqCoqIh58+ZRVlbG0KFDWbhwIXq93lrHJbohZ70DA/3crjqjusmscPZCDSfKGofYibIqTpRVs/toKeVVdY0+o9NqCPBQw6rvVULMRS//jhPCFlr8m2cymXjhhRdIT0/HYDAwffp0EhISCAsLs2wTGBjIggULeOuttxp9duPGjezfv5+PP/6Y2tpaZs2aRXx8PG5ubixevJhf/vKXJCcnk5aWxurVq5k5c6b1j1D0GA5aDQZ3JwzuTlzfz6vZbS7U1FtC62SjIKtmx5ESiiuqMZkb98K8XBwvB1ej3pgTfbycMStKs98lhGifFgMqNzeXkJAQgoODAUhOTiYrK6tRQPXt2xcArbbxCevDhw8TGxuLTqdDp9MxaNAgsrOzue2229i+fTuvvPIKAFOnTmXZsmUSUKLDufXSEWHoTYShd7Pr601mTp+vuRxgV4TYsXOVbPv+HBdq6ht9xtVRy89Gwi9v6i8zZAhhRS0GlNFoJCAgwPLeYDCQm5vbqp0PHjyYZcuWMXv2bKqqqtixYwdhYWGUlpbi7u6OTqd+fUBAAEaj8RoPQQjr0TloLb2lqymvqms0fJj1bSH/3FpI+tdHmDgkgNlxodzQ3wuNRs5tCdEeHTq4HhcXx759+/jZz36Gt7c3MTExTXpZLcnPz293HdXV1VbZT08h7dU6QUCQJ0SP9OD+WB8yD1awruA0n31XTLiPnimRHozt74ajgwTVj8mfsbbpqe3VYkAZDAaKi4st741GIwaDodVf8PDDD/Pwww8D8Jvf/IbQ0FC8vLyoqKigvr4enU5HcXHxVfcZGRnZ6u+6mvz8fKvsp6eQ9mqb/Px8rouMJD4W/lhr4qO9x3lryxEWbTnDP7+t4J7RIcwcGYK3q1wEdIn8GWub7t5eOTk5zS5vsTsTHR1NYWEhRUVF1NbWkpmZSUJCQqu+1GQyUVpaCsCBAwc4ePAgY8aMQaPRMHLkSDZs2ADAmjVrWr1PIeyZs96Bu0eG8MWT43j7vhsYHOjO4s8LGL0gi2c/yqXAeN7WJQrRZbTYg9LpdKSlpTFnzhxMJhPTpk0jPDycJUuWEBUVxfjx48nNzeWxxx6joqKCr776iqVLl5KZmUl9fT133303AG5ubixatMhy3mn+/Pk8+eSTvPbaa0RGRjJjxoyOPVIhOpFWq+HmQf7cPMifQ8bzvPV1IR/tOc6/dxYxNtyX2XGhjAv3Qyv3YAlxVRpFsd9rZHNychgxYkS799Pdu8fWJu3VNq1tr5KLtfx75zH+ubWQ0+drGOjnyn1jQpl2fV+c9Q6dUKn9kD9jbdPd2+tqv+tlIjMhOom3q55Hbwljy9MJvHZXDC56Hb//OI9RC7J4+bMDFJdX27pEIeyK3CIvRCfT67RMua4PqTFB7D5ayltbjvD/Nn3PP7J/ICk6kPvjQhke7GnrMoWwOQkoIWxEo9FwQ39vbujvTVFJJf/cWsh/dhXxybcnGRHixewxoSQONciM7aLHkj/5QtiBYG8Xfj95CNueG88fUoZw5nwNj763h3GLNrIy+/smcwgK0RNIQAlhR9x66bhvTChfPXUzK2eNoK+XMy+tO8DoBVn8ISOPI2cv2rpEITqNDPEJYYcctBomDg1g4tAA8k6Uk/51Ie/tPMY7248yfrA/s+NCGT3AR6ZTEt2a9KCEsHNRfTx45c7hfP1MAr9OCGfvsTJm/mMHSX/bwn93F1FdZ7J1iUJ0CAkoIboI/95OzJsQwdfPJLBw2jAURWH+6lziXv4ff/2igDPna2xdohBWJUN8QnQxTo4O3HlDMDNi+7L1+3O8ueUIS7IOsXzj99weE8TsMaEMCXK3dZlCtJsElBBdlEajYUyYL2PCfPnhzAXSvy5kdc5xVuccZ/QAH+4b058xYb649pK/5qJrkj+5QnQDA/zc+L8pUTw1cRDv71KnU3rgXzloNDDA15WoPh5EBXkwtI87Q4M88HB2tHXJQrRIAkqIbsTDxZEHxw1kdlwoWw6f5duiMvJOVLDzSAkZ35y0bNfP24XoPmpgRQV5MDTIHR+3XjasXIimJKCE6IYcHbTcMsifWwb5W5advVDDdycryDtRzncny9l3opzMfacs64M8nBja0NOK6uNOVB8PDO5OtihfCEACSogew9etF+Mi/BgX4WdZVl5Zx3cny8k7WU7eiQryTpbzZb6RS8848Ovdi6ggNayGNgRXH09nuf9KdAoJKCF6MA8XR24K8+WmMF/Lsgs19eSfUntaeScq+O5kOdmHzmIyq6nl6eJoOZ+l9rY8CPF2kWdbCauTgBJCNOLWS2eZxPaS6joTB4rPN4SW2uN6a8sR6kxqaPXupWNIQ08rqiG4Bvi54SChJdpBAkoI0SInRwdigj2JueIxILX1ZgqM59UhwobhwXd3HKW6zgyAs6MDkYG9G11BGGHojaPMzi5aSQJKCHFN9DptQ4/Jg7tuUJfVm8z8cPaiZXgw70Q5H+Yc551tR9XPOGgZFNAbv14moo870N/XhX7eroT4uODjqpdzW6IRCSghhNXoHLREGHoTYejNHdery8xmhcJzF8k7WcF3DcOD+05d4KsjhywXYwC46h3o5+NKfx8X+vm4ENIQXP28XQjydJbhwh5IAkoI0aG0Wg0D/NwY4OfG7cODAMjPzyc0LILjpVUcK7lI4dlKjpVUcvTcRQ4az5OVf5pak9myD0cHDcFel4LLxRJkIT4u9PVywcnRwVaHJzqQBJQQwiacHB0I83cjzN+tyTqTWeFUeRXHzlVytKSSo+cqOVZykaPnKtldWMqFmnrLthoNBLg70c/bhf4+rmqINfTA+vm4yKwZXZgElBDC7jhoNfT1UntHN/1onaIolFys5WhJpRpg59Se19GSSrIOnObshcazunu6OBLi40qIt4tlyDCkoQfm17uXnPeyYxJQQoguRaPR4OPWCx+3Xlzfz6vJ+os19ZbhwqMNPbBj5yrZW1TK2tyTmK847+Xs6EA/78tDh4MD3RkZ6k1fL7kZ2R5IQAkhuhXXXjoiA92JDGz6yJHaejMnyqo4eu5iQ4ipr8KzF8kuOENNvXreK9DDiRv6e3NjqDcjQ70J83eTwLKBVgVUdnY2L774ImazmRkzZvDAAw80Wr9r1y5eeuklDh48yKuvvsqkSZMs6xYuXMimTZswm82MGTOG3/3ud2g0GmbNmsXp06dxclLn+nrrrbfw8fGx4qEJIURjep2WUF9XQn1dm6wzmxUOnb7AziPn2HGkhO0/nOOTb9UJdr1d9cSGeDUElg+Rgb3Ryf1cHa7FgDKZTLzwwgukp6djMBiYPn06CQkJhIWFWbYJDAxkwYIFvPXWW40+u2fPHvbs2cMnn3wCwMyZM9m5cycjR44EYPHixURHR1vzeIQQ4ppotRoGBfRmUEBvZo3uj6IoHCupZMeREnYeKWFXYQmf7zcC6mwb14d4MTJU7WUN6+tBL51cSWhtLQZUbm4uISEhBAcHA5CcnExWVlajgOrbty8AWm3jf1FoNBpqa2upq6tDURTq6urw9fVFCCHsnUajUS+u8HHlzlj1919xeTU7C0vY1RBaizYcBNSeWUywpyWwru/nJQ+KtIIWW9BoNBIQEGB5bzAYyM3NbdXOr7vuOkaOHElcXByKovCLX/yCgQMHWtY/99xzaLVaJk6cyCOPPNLsGG9+fn6rvuunVFdXW2U/PYW0V9tIe7VdV26zcD2ED3Jg5iA/Kqq9+e50NXnGavKMF3n9qxKW/g+0Ggjz7kWUwYlogxNDDU707nXtPayu3F7t0aERf/ToUb7//ns2bdoEwOzZs9m9ezexsbEsXrwYg8HAhQsXePzxx8nIyGDKlClN9hEZGdnuOvLz862yn55C2qttpL3arju12cgr/vtCTT17jpay80gJOwtLWFtQxkf7ywEYZOjNjQ09rBtDvdv0rK3u1F7NycnJaXZ5iwFlMBgoLi62vDcajRgMhlZ96RdffMHw4cNxdVVPSI4dO5a9e/cSGxtr2YebmxuTJ08mNze32YASQoiuwq2XjvgIP+IbnrlVXWci93g5uwpL2HGkhI/2HOdf29V5CUN8XLjRcqWgD8Hecmn7j7UYUNHR0RQWFlJUVITBYCAzM5NXXnmlVTsPCgrigw8+oL6+HkVR2LVrF/feey/19fVUVFTg7e1NXV0dGzduZPTo0e0+GCGEsCdOjg6WHtOjt6iT6e4/VaH2sI6U8GW+kf/mHAfA4N6LG0N9Ll/a7ufW45+x1WJA6XQ60tLSmDNnDiaTiWnTphEeHs6SJUuIiopi/Pjx5Obm8thjj1FRUcFXX33F0qVLyczMJDExke3bt5OSkoJGo2Hs2LEkJCRQWVnJnDlzqKurw2w2M3r0aO68887OOF4hhLAZnYOWYX09GdbXkzljB2A2Kxw+c4EdRy5fePFpw6Xtni6O3NBfDSsvczX+wTV497AZ3zWKcuV8wvYlJyeHESNGtHs/3X381tqkvdpG2qvtpM2apygKRSVV7CwsYeeRc+w8UkLhuUrL+t69dIT4Xp6qSf3Z9adtutrverkOUggh7IRGo6Ffw+NGpo9Qb985XVHNuu15mF18OHruIoXnKvnuRDmf5RVjumLeJmdHB3WSXB910lxLiPm6Euju1CWHCyWghBDCjvm7OzEy2JXIyNBGy+tMZk6WVVHYMFlu4Vn15+HTF/jqwJlGjyvR67QNs7037X0FeTrZ7awYElBCCNEFOTpoLTcSg1+jdSazQnFFNUfPXrwcYA2T5245fJbqusvhpdNq6Ovl3Di4GoYR+3o523SGDAkoIYToZhy0Gvp4OtPH05mbwhqvUxSF0+drKDyrBtal4Co8d5Gco42ftaXVQJCnc8OQYcNzthp6Xv28XXDWd2x4SUAJIUQPotFoMLg7YXB3YuSAxhN0X3rW1uVe1+WfmftOUVZZ12j7yEB31jxyU4c90VgCSgghBND4WVsjQpo+a6ussrZRr8tkVtB34PkrCSghhBCt4umix9NFz/Bgz075Pvu8dEMIIUSPJwElhBDCLklACSGEsEsSUEIIIeySBJQQQgi7JAElhBDCLklACSGEsEsSUEIIIeySBJQQQgi7ZPcPLBRCCNH9NffAQrsOKCGEED2XDPEJIYSwSxJQQggh7JIElBBCCLvUrQMqOzubxMREJkyYwMqVK21djt07deoUs2bNIikpieTkZP75z3/auqQuwWQyMWXKFB588EFbl2L3KioqePzxx5k0aRK33XYbe/futXVJdu/tt98mOTmZyZMnM2/ePGpqamxdUqfptgFlMpl44YUXeOONN8jMzGTt2rUcPnzY1mXZNQcHB5555hnWrVvHf/7zH9577z1ps1Z45513GDhwoK3L6BJefPFFxo4dy2effUZGRoa0WwuMRiPvvPMOH374IWvXrsVkMpGZmWnrsjpNtw2o3NxcQkJCCA4ORq/Xk5ycTFZWlq3Lsmv+/v4MHToUADc3NwYMGIDRaLRxVfatuLiYjRs3Mn36dFuXYvfOnz/Prl27LG2l1+txd3e3cVX2z2QyUV1dTX19PdXV1fj7+9u6pE7TbQPKaDQSEBBgeW8wGOSXbRscP36c/Px8hg8fbutS7NpLL73E/Pnz0Wq77V8lqzl+/Dje3t48++yzTJkyhd/97ndUVlbauiy7ZjAYmD17NrfccgtxcXG4ubkRFxdn67I6jfytEk1cvHiRxx9/nOeeew43Nzdbl2O3vvrqK7y9vYmKirJ1KV1CfX09+/fv5+c//zkff/wxzs7Ocm64BeXl5WRlZZGVlcXmzZupqqoiIyPD1mV1mm4bUAaDgeLiYst7o9GIwWCwYUVdQ11dHY8//jgpKSlMnDjR1uXYtT179vC///2PhIQE5s2bx/bt23nqqadsXZbdCggIICAgwNIrnzRpEvv377dxVfZt69at9O3bF29vbxwdHZk4cWKPurCk2wZUdHQ0hYWFFBUVUVtbS2ZmJgkJCbYuy64pisLvfvc7BgwYwH333Wfrcuzeb37zG7Kzs/nf//7Hq6++yqhRo1i8eLGty7Jbfn5+BAQE8MMPPwCwbds2uUiiBUFBQXz77bdUVVWhKEqPazOdrQvoKDqdjrS0NObMmYPJZGLatGmEh4fbuiy7lpOTQ0ZGBhEREaSmpgIwb948xo0bZ+PKRHfx/PPP89RTT1FXV0dwcDALFiywdUl2bfjw4SQmJjJ16lR0Oh2RkZHcddddti6r08hcfEIIIexStx3iE0II0bVJQAkhhLBLElBCCCHskgSUEEIIuyQBJYQQwi5JQAkhhLBLElBCCCHs0v8HNbVmGFBK4NkAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model = keras.Sequential([\n",
    "    layers.Dense(128, activation='relu', input_shape=input_shape),\n",
    "    layers.Dense(64, activation='relu'),    \n",
    "    layers.Dense(1)\n",
    "])\n",
    "model.compile(\n",
    "    optimizer='adam',\n",
    "    loss='mae',\n",
    ")\n",
    "history = model.fit(\n",
    "    X_train, y_train,\n",
    "    validation_data=(X_valid, y_valid),\n",
    "    batch_size=512,\n",
    "    epochs=50,\n",
    "    callbacks=[early_stopping]\n",
    ")\n",
    "history_df = pd.DataFrame(history.history)\n",
    "history_df.loc[:, ['loss', 'val_loss']].plot()\n",
    "print(\"Minimum Validation Loss: {:0.4f}\".format(history_df['val_loss'].min()));"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2ef03999",
   "metadata": {
    "papermill": {
     "duration": 0.133295,
     "end_time": "2022-02-05T23:16:15.329961",
     "exception": false,
     "start_time": "2022-02-05T23:16:15.196666",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# 4) Train and Interpret\n",
    "\n",
    "Was this an improvement compared to training without early stopping?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "cd7c8422",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-02-05T23:16:15.567028Z",
     "iopub.status.busy": "2022-02-05T23:16:15.564965Z",
     "iopub.status.idle": "2022-02-05T23:16:15.572432Z",
     "shell.execute_reply": "2022-02-05T23:16:15.571756Z",
     "shell.execute_reply.started": "2022-02-05T23:14:49.500745Z"
    },
    "papermill": {
     "duration": 0.128759,
     "end_time": "2022-02-05T23:16:15.572597",
     "exception": false,
     "start_time": "2022-02-05T23:16:15.443838",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"outcomeType\": 1, \"valueTowardsCompletion\": 0.25, \"interactionType\": 1, \"questionType\": 4, \"questionId\": \"4_Q4\", \"learnToolsVersion\": \"0.3.4\", \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\"}}, \"*\")"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<span style=\"color:#33cc33\">Correct:</span> \n",
       "\n",
       "The early stopping callback did stop the training once the network began overfitting. Moreover, by including `restore_best_weights` we still get to keep the model where validation loss was lowest."
      ],
      "text/plain": [
       "Correct: \n",
       "\n",
       "The early stopping callback did stop the training once the network began overfitting. Moreover, by including `restore_best_weights` we still get to keep the model where validation loss was lowest."
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# View the solution (Run this cell to receive credit!)\n",
    "q_4.check()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "63024e16",
   "metadata": {
    "papermill": {
     "duration": 0.114028,
     "end_time": "2022-02-05T23:16:15.803455",
     "exception": false,
     "start_time": "2022-02-05T23:16:15.689427",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "If you like, try experimenting with `patience` and `min_delta` to see what difference it might make.\n",
    "\n",
    "# Keep Going #\n",
    "\n",
    "Move on to [**learn about a couple of special layers**](https://www.kaggle.com/ryanholbrook/dropout-and-batch-normalization): batch normalization and dropout."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "23035b98",
   "metadata": {
    "papermill": {
     "duration": 0.070303,
     "end_time": "2022-02-05T23:16:15.948298",
     "exception": false,
     "start_time": "2022-02-05T23:16:15.877995",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "---\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "*Have questions or comments? Visit the [course discussion forum](https://www.kaggle.com/learn/intro-to-deep-learning/discussion) to chat with other learners.*"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.12"
  },
  "papermill": {
   "default_parameters": {},
   "duration": 47.36732,
   "end_time": "2022-02-05T23:16:19.610650",
   "environment_variables": {},
   "exception": null,
   "input_path": "__notebook__.ipynb",
   "output_path": "__notebook__.ipynb",
   "parameters": {},
   "start_time": "2022-02-05T23:15:32.243330",
   "version": "2.3.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}