intro_to_deep_learning/02-deep-neural-networks.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "fc89edd7",
   "metadata": {
    "papermill": {
     "duration": 0.017082,
     "end_time": "2022-01-29T23:32:31.783135",
     "exception": false,
     "start_time": "2022-01-29T23:32:31.766053",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "**This notebook is an exercise in the [Intro to Deep Learning](https://www.kaggle.com/learn/intro-to-deep-learning) course.  You can reference the tutorial at [this link](https://www.kaggle.com/ryanholbrook/deep-neural-networks).**\n",
    "\n",
    "---\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1d96354a",
   "metadata": {
    "papermill": {
     "duration": 0.013681,
     "end_time": "2022-01-29T23:32:31.815095",
     "exception": false,
     "start_time": "2022-01-29T23:32:31.801414",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Introduction #\n",
    "\n",
    "In the tutorial, we saw how to build deep neural networks by stacking layers inside a `Sequential` model. By adding an *activation function* after the hidden layers, we gave the network the ability to learn more complex (non-linear) relationships in the data.\n",
    "\n",
    "In these exercises, you'll build a neural network with several hidden layers and then explore some activation functions beyond ReLU. Run this next cell to set everything up!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "a3876d02",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-01-29T23:32:31.854158Z",
     "iopub.status.busy": "2022-01-29T23:32:31.853505Z",
     "iopub.status.idle": "2022-01-29T23:32:36.477365Z",
     "shell.execute_reply": "2022-01-29T23:32:36.476277Z",
     "shell.execute_reply.started": "2022-01-29T22:51:12.842849Z"
    },
    "papermill": {
     "duration": 4.648974,
     "end_time": "2022-01-29T23:32:36.477533",
     "exception": false,
     "start_time": "2022-01-29T23:32:31.828559",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "\n",
    "# Setup plotting\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.style.use('seaborn-whitegrid')\n",
    "# Set Matplotlib defaults\n",
    "plt.rc('figure', autolayout=True)\n",
    "plt.rc('axes', labelweight='bold', labelsize='large',\n",
    "       titleweight='bold', titlesize=18, titlepad=10)\n",
    "\n",
    "# Setup feedback system\n",
    "from learntools.core import binder\n",
    "binder.bind(globals())\n",
    "from learntools.deep_learning_intro.ex2 import *"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "10122749",
   "metadata": {
    "papermill": {
     "duration": 0.012563,
     "end_time": "2022-01-29T23:32:36.502783",
     "exception": false,
     "start_time": "2022-01-29T23:32:36.490220",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "In the *Concrete* dataset, your task is to predict the compressive strength of concrete manufactured according to various recipes.\n",
    "\n",
    "Run the next code cell without changes to load the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "fe9809f3",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-01-29T23:32:36.534523Z",
     "iopub.status.busy": "2022-01-29T23:32:36.533833Z",
     "iopub.status.idle": "2022-01-29T23:32:36.572844Z",
     "shell.execute_reply": "2022-01-29T23:32:36.573363Z",
     "shell.execute_reply.started": "2022-01-29T22:51:12.851193Z"
    },
    "papermill": {
     "duration": 0.05791,
     "end_time": "2022-01-29T23:32:36.573534",
     "exception": false,
     "start_time": "2022-01-29T23:32:36.515624",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Cement</th>\n",
       "      <th>BlastFurnaceSlag</th>\n",
       "      <th>FlyAsh</th>\n",
       "      <th>Water</th>\n",
       "      <th>Superplasticizer</th>\n",
       "      <th>CoarseAggregate</th>\n",
       "      <th>FineAggregate</th>\n",
       "      <th>Age</th>\n",
       "      <th>CompressiveStrength</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>540.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>162.0</td>\n",
       "      <td>2.5</td>\n",
       "      <td>1040.0</td>\n",
       "      <td>676.0</td>\n",
       "      <td>28</td>\n",
       "      <td>79.99</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>540.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>162.0</td>\n",
       "      <td>2.5</td>\n",
       "      <td>1055.0</td>\n",
       "      <td>676.0</td>\n",
       "      <td>28</td>\n",
       "      <td>61.89</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>332.5</td>\n",
       "      <td>142.5</td>\n",
       "      <td>0.0</td>\n",
       "      <td>228.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>932.0</td>\n",
       "      <td>594.0</td>\n",
       "      <td>270</td>\n",
       "      <td>40.27</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>332.5</td>\n",
       "      <td>142.5</td>\n",
       "      <td>0.0</td>\n",
       "      <td>228.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>932.0</td>\n",
       "      <td>594.0</td>\n",
       "      <td>365</td>\n",
       "      <td>41.05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>198.6</td>\n",
       "      <td>132.4</td>\n",
       "      <td>0.0</td>\n",
       "      <td>192.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>978.4</td>\n",
       "      <td>825.5</td>\n",
       "      <td>360</td>\n",
       "      <td>44.30</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Cement  BlastFurnaceSlag  FlyAsh  Water  Superplasticizer  CoarseAggregate  \\\n",
       "0   540.0               0.0     0.0  162.0               2.5           1040.0   \n",
       "1   540.0               0.0     0.0  162.0               2.5           1055.0   \n",
       "2   332.5             142.5     0.0  228.0               0.0            932.0   \n",
       "3   332.5             142.5     0.0  228.0               0.0            932.0   \n",
       "4   198.6             132.4     0.0  192.0               0.0            978.4   \n",
       "\n",
       "   FineAggregate  Age  CompressiveStrength  \n",
       "0          676.0   28                79.99  \n",
       "1          676.0   28                61.89  \n",
       "2          594.0  270                40.27  \n",
       "3          594.0  365                41.05  \n",
       "4          825.5  360                44.30  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "concrete = pd.read_csv('../input/dl-course-data/concrete.csv')\n",
    "concrete.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "542ef499",
   "metadata": {
    "papermill": {
     "duration": 0.013325,
     "end_time": "2022-01-29T23:32:36.600652",
     "exception": false,
     "start_time": "2022-01-29T23:32:36.587327",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# 1) Input Shape #\n",
    "\n",
    "The target for this task is the column `'CompressiveStrength'`. The remaining columns are the features we'll use as inputs.\n",
    "\n",
    "What would be the input shape for this dataset?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "3bf385ad",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-01-29T23:32:36.632803Z",
     "iopub.status.busy": "2022-01-29T23:32:36.632030Z",
     "iopub.status.idle": "2022-01-29T23:32:36.637078Z",
     "shell.execute_reply": "2022-01-29T23:32:36.637552Z",
     "shell.execute_reply.started": "2022-01-29T22:53:01.555767Z"
    },
    "lines_to_next_cell": 2,
    "papermill": {
     "duration": 0.023995,
     "end_time": "2022-01-29T23:32:36.637723",
     "exception": false,
     "start_time": "2022-01-29T23:32:36.613728",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"outcomeType\": 1, \"valueTowardsCompletion\": 0.3333333333333333, \"interactionType\": 1, \"questionType\": 2, \"questionId\": \"1_Q1\", \"learnToolsVersion\": \"0.3.4\", \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\"}}, \"*\")"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<span style=\"color:#33cc33\">Correct</span>"
      ],
      "text/plain": [
       "Correct"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# YOUR CODE HERE\n",
    "input_shape = [concrete.shape[1] - 1]\n",
    "\n",
    "# Check your answer\n",
    "q_1.check()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "4f7b4fb4",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-01-29T23:32:36.670123Z",
     "iopub.status.busy": "2022-01-29T23:32:36.669429Z",
     "iopub.status.idle": "2022-01-29T23:32:36.671720Z",
     "shell.execute_reply": "2022-01-29T23:32:36.672171Z",
     "shell.execute_reply.started": "2022-01-29T22:53:04.294696Z"
    },
    "papermill": {
     "duration": 0.020526,
     "end_time": "2022-01-29T23:32:36.672356",
     "exception": false,
     "start_time": "2022-01-29T23:32:36.651830",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Lines below will give you a hint or solution code\n",
    "#q_1.hint()\n",
    "#q_1.solution()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e3ba8337",
   "metadata": {
    "papermill": {
     "duration": 0.014277,
     "end_time": "2022-01-29T23:32:36.700804",
     "exception": false,
     "start_time": "2022-01-29T23:32:36.686527",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# 2) Define a Model with Hidden Layers #\n",
    "\n",
    "Now create a model with three hidden layers, each having 512 units and the ReLU activation.  Be sure to include an output layer of one unit and no activation, and also `input_shape` as an argument to the first layer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "f085049c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-01-29T23:32:36.736058Z",
     "iopub.status.busy": "2022-01-29T23:32:36.735379Z",
     "iopub.status.idle": "2022-01-29T23:32:38.019145Z",
     "shell.execute_reply": "2022-01-29T23:32:38.019657Z",
     "shell.execute_reply.started": "2022-01-29T22:56:29.467587Z"
    },
    "lines_to_next_cell": 0,
    "papermill": {
     "duration": 1.304886,
     "end_time": "2022-01-29T23:32:38.019843",
     "exception": false,
     "start_time": "2022-01-29T23:32:36.714957",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "User settings:\n",
      "\n",
      "   KMP_AFFINITY=granularity=fine,noverbose,compact,1,0\n",
      "   KMP_BLOCKTIME=0\n",
      "   KMP_SETTINGS=1\n",
      "   KMP_WARNINGS=0\n",
      "\n",
      "Effective settings:\n",
      "\n",
      "   KMP_ABORT_DELAY=0\n",
      "   KMP_ADAPTIVE_LOCK_PROPS='1,1024'\n",
      "   KMP_ALIGN_ALLOC=64\n",
      "   KMP_ALL_THREADPRIVATE=128\n",
      "   KMP_ATOMIC_MODE=2\n",
      "   KMP_BLOCKTIME=0\n",
      "   KMP_CPUINFO_FILE: value is not defined\n",
      "   KMP_DETERMINISTIC_REDUCTION=false\n",
      "   KMP_DEVICE_THREAD_LIMIT=2147483647\n",
      "   KMP_DISP_NUM_BUFFERS=7\n",
      "   KMP_DUPLICATE_LIB_OK=false\n",
      "   KMP_ENABLE_TASK_THROTTLING=true\n",
      "   KMP_FORCE_REDUCTION: value is not defined\n",
      "   KMP_FOREIGN_THREADS_THREADPRIVATE=true\n",
      "   KMP_FORKJOIN_BARRIER='2,2'\n",
      "   KMP_FORKJOIN_BARRIER_PATTERN='hyper,hyper'\n",
      "   KMP_GTID_MODE=3\n",
      "   KMP_HANDLE_SIGNALS=false\n",
      "   KMP_HOT_TEAMS_MAX_LEVEL=1\n",
      "   KMP_HOT_TEAMS_MODE=0\n",
      "   KMP_INIT_AT_FORK=true\n",
      "   KMP_LIBRARY=throughput\n",
      "   KMP_LOCK_KIND=queuing\n",
      "   KMP_MALLOC_POOL_INCR=1M\n",
      "   KMP_NUM_LOCKS_IN_BLOCK=1\n",
      "   KMP_PLAIN_BARRIER='2,2'\n",
      "   KMP_PLAIN_BARRIER_PATTERN='hyper,hyper'\n",
      "   KMP_REDUCTION_BARRIER='1,1'\n",
      "   KMP_REDUCTION_BARRIER_PATTERN='hyper,hyper'\n",
      "   KMP_SCHEDULE='static,balanced;guided,iterative'\n",
      "   KMP_SETTINGS=true\n",
      "   KMP_SPIN_BACKOFF_PARAMS='4096,100'\n",
      "   KMP_STACKOFFSET=64\n",
      "   KMP_STACKPAD=0\n",
      "   KMP_STACKSIZE=8M\n",
      "   KMP_STORAGE_MAP=false\n",
      "   KMP_TASKING=2\n",
      "   KMP_TASKLOOP_MIN_TASKS=0\n",
      "   KMP_TASK_STEALING_CONSTRAINT=1\n",
      "   KMP_TEAMS_THREAD_LIMIT=4\n",
      "   KMP_TOPOLOGY_METHOD=all\n",
      "   KMP_USE_YIELD=1\n",
      "   KMP_VERSION=false\n",
      "   KMP_WARNINGS=false\n",
      "   OMP_AFFINITY_FORMAT='OMP: pid %P tid %i thread %n bound to OS proc set {%A}'\n",
      "   OMP_ALLOCATOR=omp_default_mem_alloc\n",
      "   OMP_CANCELLATION=false\n",
      "   OMP_DEFAULT_DEVICE=0\n",
      "   OMP_DISPLAY_AFFINITY=false\n",
      "   OMP_DISPLAY_ENV=false\n",
      "   OMP_DYNAMIC=false\n",
      "   OMP_MAX_ACTIVE_LEVELS=1\n",
      "   OMP_MAX_TASK_PRIORITY=0\n",
      "   OMP_NESTED: deprecated; max-active-levels-var=1\n",
      "   OMP_NUM_THREADS: value is not defined\n",
      "   OMP_PLACES: value is not defined\n",
      "   OMP_PROC_BIND='intel'\n",
      "   OMP_SCHEDULE='static'\n",
      "   OMP_STACKSIZE=8M\n",
      "   OMP_TARGET_OFFLOAD=DEFAULT\n",
      "   OMP_THREAD_LIMIT=2147483647\n",
      "   OMP_WAIT_POLICY=PASSIVE\n",
      "   KMP_AFFINITY='noverbose,warnings,respect,granularity=fine,compact,1,0'\n",
      "\n",
      "2022-01-29 23:32:37.928512: I tensorflow/core/common_runtime/process_util.cc:146] Creating new thread pool with default inter op setting: 2. Tune using inter_op_parallelism_threads for best performance.\n"
     ]
    },
    {
     "data": {
      "application/javascript": [
       "parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"outcomeType\": 1, \"valueTowardsCompletion\": 0.3333333333333333, \"interactionType\": 1, \"questionType\": 2, \"questionId\": \"2_Q2\", \"learnToolsVersion\": \"0.3.4\", \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\"}}, \"*\")"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<span style=\"color:#33cc33\">Correct</span>"
      ],
      "text/plain": [
       "Correct"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from tensorflow import keras\n",
    "from tensorflow.keras import layers\n",
    "\n",
    "# YOUR CODE HERE\n",
    "model = keras.Sequential([\n",
    "    # hidden ReLU layers\n",
    "    layers.Dense(units=512, activation='relu', input_shape=input_shape),\n",
    "    layers.Dense(units=512, activation='relu'),\n",
    "    layers.Dense(units=512, activation='relu'),\n",
    "    # output layer\n",
    "    layers.Dense(units=1)\n",
    "])\n",
    "\n",
    "# Check your answer\n",
    "q_2.check()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "d0ec7e1c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-01-29T23:32:38.054591Z",
     "iopub.status.busy": "2022-01-29T23:32:38.053958Z",
     "iopub.status.idle": "2022-01-29T23:32:38.056827Z",
     "shell.execute_reply": "2022-01-29T23:32:38.057359Z",
     "shell.execute_reply.started": "2022-01-29T22:56:46.627473Z"
    },
    "papermill": {
     "duration": 0.022143,
     "end_time": "2022-01-29T23:32:38.057513",
     "exception": false,
     "start_time": "2022-01-29T23:32:38.035370",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Lines below will give you a hint or solution code\n",
    "#q_2.hint()\n",
    "#q_2.solution()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "850c19d9",
   "metadata": {
    "papermill": {
     "duration": 0.015486,
     "end_time": "2022-01-29T23:32:38.088446",
     "exception": false,
     "start_time": "2022-01-29T23:32:38.072960",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# 3) Activation Layers #\n",
    "\n",
    "Let's explore activations functions some.\n",
    "\n",
    "The usual way of attaching an activation function to a `Dense` layer is to include it as part of the definition with the `activation` argument. Sometimes though you'll want to put some other layer between the `Dense` layer and its activation function. (We'll see an example of this in Lesson 5 with *batch normalization*.) In this case, we can define the activation in its own `Activation` layer, like so:\n",
    "\n",
    "```\n",
    "layers.Dense(units=8),\n",
    "layers.Activation('relu')\n",
    "```\n",
    "\n",
    "This is completely equivalent to the ordinary way: `layers.Dense(units=8, activation='relu')`.\n",
    "\n",
    "Rewrite the following model so that each activation is in its own `Activation` layer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "b2f64498",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-01-29T23:32:38.123166Z",
     "iopub.status.busy": "2022-01-29T23:32:38.122533Z",
     "iopub.status.idle": "2022-01-29T23:32:38.156849Z",
     "shell.execute_reply": "2022-01-29T23:32:38.157336Z",
     "shell.execute_reply.started": "2022-01-29T23:04:51.911825Z"
    },
    "lines_to_next_cell": 0,
    "papermill": {
     "duration": 0.053505,
     "end_time": "2022-01-29T23:32:38.157507",
     "exception": false,
     "start_time": "2022-01-29T23:32:38.104002",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"outcomeType\": 1, \"valueTowardsCompletion\": 0.3333333333333333, \"interactionType\": 1, \"questionType\": 2, \"questionId\": \"3_Q3\", \"learnToolsVersion\": \"0.3.4\", \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\"}}, \"*\")"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<span style=\"color:#33cc33\">Correct</span>"
      ],
      "text/plain": [
       "Correct"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "### YOUR CODE HERE: rewrite this to use activation layers\n",
    "model = keras.Sequential([\n",
    "    layers.Dense(32, input_shape=[8]),\n",
    "    layers.Activation('relu'),\n",
    "    layers.Dense(32),\n",
    "    layers.Activation('relu'),\n",
    "    layers.Dense(1),\n",
    "])\n",
    "\n",
    "# Check your answer\n",
    "q_3.check()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "daec53dc",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-01-29T23:32:38.193767Z",
     "iopub.status.busy": "2022-01-29T23:32:38.193037Z",
     "iopub.status.idle": "2022-01-29T23:32:38.195644Z",
     "shell.execute_reply": "2022-01-29T23:32:38.196111Z",
     "shell.execute_reply.started": "2022-01-29T23:05:05.899799Z"
    },
    "papermill": {
     "duration": 0.022399,
     "end_time": "2022-01-29T23:32:38.196260",
     "exception": false,
     "start_time": "2022-01-29T23:32:38.173861",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Lines below will give you a hint or solution code\n",
    "#q_3.hint()\n",
    "#q_3.solution()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "575f8652",
   "metadata": {
    "papermill": {
     "duration": 0.016303,
     "end_time": "2022-01-29T23:32:38.229168",
     "exception": false,
     "start_time": "2022-01-29T23:32:38.212865",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Optional: Alternatives to ReLU #\n",
    "\n",
    "There is a whole family of variants of the `'relu'` activation -- `'elu'`, `'selu'`, and `'swish'`, among others -- all of which you can use in Keras. Sometimes one activation will perform better than another on a given task, so you could consider experimenting with activations as you develop a model. The ReLU activation tends to do well on most problems, so it's a good one to start with.\n",
    "\n",
    "Let's look at the graphs of some of these. Change the activation from `'relu'` to one of the others named above. Then run the cell to see the graph. (Check out the [documentation](https://www.tensorflow.org/api_docs/python/tf/keras/activations) for more ideas.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "0d38d16a",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-01-29T23:32:38.265181Z",
     "iopub.status.busy": "2022-01-29T23:32:38.264582Z",
     "iopub.status.idle": "2022-01-29T23:32:38.963105Z",
     "shell.execute_reply": "2022-01-29T23:32:38.962564Z",
     "shell.execute_reply.started": "2022-01-29T23:31:41.311783Z"
    },
    "lines_to_next_cell": 0,
    "papermill": {
     "duration": 0.717626,
     "end_time": "2022-01-29T23:32:38.963241",
     "exception": false,
     "start_time": "2022-01-29T23:32:38.245615",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsgAAAGoCAYAAABbtxOxAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABa4klEQVR4nO3dd3hUdf728fekd0ICCaETaiiChCoKihQVAUVxFWHF1bX+ZBUFEekICjbsBQvYd10LgmJDVASlRDqB0AIEUggJ6WUyc54/BrLxESRkBs7M5H5dV65Mkpkz92ciX24P55yxGIZhICIiIiIiAPiYHUBERERExJ2oIIuIiIiIVKGCLCIiIiJShQqyiIiIiEgVKsgiIiIiIlWoIIuIiIiIVKGCLAJ8+umntG3blrZt2zJmzBiz44iIeLS1a9dWrqn9+/c3O47IWVNBFhERERGpQgVZRERERKQKFWTxOkVFRWZHEBEREQ+mgiwe6YUXXqg8vm3SpEmsWrWKG2+8kS5dujBq1CgAjh8/zoIFCxg2bBgXXnghF1xwAUOGDOGFF16odon+q2OTJ02aVPmzF154weUzioi4m5KSEhYuXMj1119P165d6dixI4MGDeLxxx8nJyfnjI//q2OT//91XcRMfmYHEHHWhg0bWLJkCXa7vfJ7Bw4c4O9//zsZGRl/uO+ePXt48cUX+fbbb3n33XeJjIw8z2lFRDxTTk4Ot9xyCykpKX/4/oEDB1i0aBHLly/n/fffp0mTJiYlFHEd7UEWj3fo0CHi4+OZP38+b775JqNHj2bChAmV5bhnz5689NJLvPrqq/To0QOAlJQU5s6da2ZsERGPMmvWrMpynJCQwDPPPMPChQsZPHgwAJmZmdrzK15DBVk8XnBwMIsXL2b48OFcfPHFdO7cmc2bNwPg7+/PHXfcQWRkJOHh4YwePbrycV999ZWOVxYRqYb8/Hy+/fbbyq9vv/12YmNjCQkJ4eabb8bf3x9w/Ivevn37zIop4jI6xEI8XteuXalXr17l13v27Km8bbVaue222075OKvVyv79++nYseM5zygi4slSU1Ox2WyVXz/44IOnve/u3bt1+Jp4PO1BFo9Xv379Gj+2uLj4L39usVgqb1dUVPzhZ9U5IUVEpLb5q3VVa6p4Cu1BFo9XdcEFaNmyZeXtoKAgfvnlF8LDw//0uOLiYkJCQv5y2xEREZW3MzMzK2/n5+eTlJRU08giIh6lefPm+Pr6Vu5F/vrrr2nRosWf7ndyXV27du0pt1N1Tc3NzaW8vJyAgAAqKipYtWrVuQkvUgMqyOJ12rZtS6dOndi6dSulpaXccsstjBkzhgYNGpCbm0taWhq//fYbdrudRYsW/eW2mjdvXnn78OHDTJo0iY4dO/Lf//6XwsLCczuIiIibiIiIYODAgXz99dcA3HHHHdx22200a9aM/Px8jhw5wvr169m3b1/lfU6lSZMm+Pn5UVFRQXl5OePGjeOSSy7h66+/5tChQ+drHJEzUkEWr/TUU09xyy23kJGRwfbt2095ZvXJK1r8lZYtW9KnTx9Wr14NwGeffcZnn32Gv78/zZs3JzU11dXRRUTc0vTp09m3bx8pKSkcPHiQ6dOn/+k+jRo1+stthIaGMmLECP7zn/8AsHLlSlauXInFYqF169bs3r37nGQXOVsqyOKVmjdvzhdffMHixYtZuXIlqampVFRUUK9ePRo1asTFF1/MwIEDq7WtefPmMWfOHFatWoXNZqNLly488MADfPjhhyrIIlJrREVF8fHHH/PBBx/wzTffsHfvXkpLS6lbty4NGzakV69e1VpXT+6w+PbbbykpKSEhIYF77rmHLVu2qCCL27AYhmGYHUJERERExF3oKhYiIiIiIlWoIIuIiIiIVKGCLCIiIiJShQqyiIiIiEgVKsgiIiIiIlWoIIuIiIiIVKGCLCIiIiJShQqyiIiIiEgV5+Wd9JKSks7H04iIeJ3ExMRq3U/rrIhIzZxqnT1vbzVd3UX+VJKTk0lISHBhGnN4yxygWdyRt8wBmuWksy29NV1n9Xq7J2+ZxVvmAM3ijpyd43TrrA6xEBERERGpQgVZRERERKQKFWQRERERkSpUkEVEREREqlBBFhERERGpQgVZRERERKQKFWQRERERkSpUkEVEREREqnD6jULKysq4+eabKS8vx2azMXjwYMaNG+eKbCIiIiIi553TBTkgIIDFixcTGhqK1Wpl1KhR9O3bly5durggnoiIiIjI+eX0IRYWi4XQ0FAAKioqqKiowGKxOB1MRMSTHT5ewvj/bOKHfQVmRxER8T52O2x4m0arJ0NFmcs3bzEMw3B2IzabjREjRnDw4EFGjRrFhAkT/vDzpKQkQkJCarz90tJSgoKCnI1pOm+ZAzSLO/KWOcCzZ7HZDZYk5/HuplwM4KHedbk4PrJG2youLiYxMbFa93VmnfXk1/v/p1ncj7fMAZrFXQTk7SNuwxOEZG8hP6YHhy9dAJaa7fM93Trr9CEWAL6+vixZsoT8/HzuvfdeUlJSaNOmzR/uk5CQUOPtJycnO/V4d+Etc4BmcUfeMgd47ixb0/J45LMtbDucT/92Mcwa3oGCjAM1niUpKems7l/T5/HU1/tUNIv78ZY5QLOYzloKPz8Jq5+DwHC45hUOB3QhoX37Gm/ydOusSwrySREREfTs2ZNVq1b9qSCLiHirorIKnv42hUVr9hMdFshLo7pyVacGWCwWkjPMTici4gX2/QTL7oecfdD5Jhg0B0KjITn5nDyd0wU5JycHPz8/IiIiKC0tZc2aNfzzn/90RTYREbf3/Y5Mpi3ZRnp+KTf3bMrEK9oREeRvdiwREe9QdAy+fRQ2fwhR8fD3JRB/6Tl/WqcLclZWFpMmTcJms2EYBldccQWXXXaZK7KJiLitzPxSZnyxneXbMmgTG8Z/R/UmsVmU2bFERLyDYcCmD+DbKVCWD5c8BH0fAv/g8/L0Thfkdu3a8fnnn7sgioiI+7PZDd5fe4Anv95Fuc3OhMFt+ecl8QT46X2XRERcInuP43CK1FXQpCcMfQ5izu/x0i49BllExJslp+fzyKdb2XToOH1aRTPnmk40rxdqdiwREe9QUeY4Ae/np8AvCK5+FrqOBZ/zvwNCBVlE5AxKym08t2I3C1fto06wP8/+rTPXdGmka76LiLjKgTWw9H7I3gUdroUrnoDwBqbFUUEWEfkLP6cc5dHPt3Iop4SRiY2ZfFUCdUMDzI4lIuIdSnLhu+nw+2Ko0xRGfQxtBpmdSgVZRORUsgvLmL1sB0s2HSG+Xigf/rMXvVtGmx1LRMQ7GAZs+wS+ngTFOXDRfXDpIxDgHoetqSCLiFRhtxt8nHSIuV/tpKTcxrjLW3PPpS0J8vc1O5qIiHfITYVl42HvCmjYFUZ/AnGdzU71ByrIIiIn7MkqYPKn21iXmkOPFlHMvbYTrWLCzI4lIuIdbFb49SX48Qnw8YUr50P32x233YwKsojUemUVNl5euZeXf9xDSIAf867rxMjEJvj46CQ8ERGXSEuCpeMgcxu0u9pRjus0MjvVaakgi0it9uveYzz62Vb2ZRcxvEtDpl7dnnphgWbHEhHxDqX58MNsWLcQwuPgb+9DwtVmpzojFWQRqZVyi8qZ+1UyHyel0SQqmMX/6EG/NvXNjiUi4h0MA5KXwvKJUJABPe6A/lMgKMLsZNWigiwitYphGHy+6TCzlyWTV2Llrn4t+dflrQkOcL9j4EREPFJeGnw1AXZ9BbGdHHuNGyeaneqsqCCLSK1x4FgRUz7fxqrd2XRpEsnjIzqREOcZezNERNye3QbrXocfHgPDDgNnQ6+7wdff7GRnTQVZRLxeeYWdhav28fyK3QT4+jB7eAdG9WyGr07CExFxjfTNsPRfcGQjtBoAQ56Gus3NTlVjKsgi4tWSDuQw+dNt7Mos4MqODZg+tAMN6gSZHUtExDuUF8HKufDbKxASDde9CR2vA4tn74BQQRYRr5RXYmX+1zt5f+1BGtYJ4o2/d2NA+1izY4mIeI+Ub+DLByHvECSOhQEzILiu2alcQgVZRLyKYRh8uTWdmUt3cKywjNsubsH4gW0IDdRyJyLiEgUZsPxh2PE51G8Ht34NzXqbncql9DeGiHiNtNxipn6+jZW7jtKxUQRv3dKdTo3rmB1LRMQ72O2Q9DZ8PxMqSh2XbbvoX+AXYHYyl1NBFhGPV2Gz8/bqVJ75LgWLBaYMSWDsRc3x8/UxO5qIiHfI3AHL7odDa6FFX7h6AUS3NDvVOaOCLCIebUvacR75dCvbj+RzebsYZg7vQOO6IWbHEhHxDtYS+Gk+rHkeAiPgmleh840efxLemaggi4hHKiyr4Olvd7F4TSr1wgJ5+eauXNmxARYvX7RFRM6bvSth2QOQux86j4JBj0FotNmpzgsVZBHxON/tyGTakm1k5Jdyc8+mTLyiHRFBnnchehERt1SUDd9Mhi3/hqh4+PsXEN/P7FTnlQqyiHiMjLxSZnyxna+3Z9A2NpwXR3UlsZl3XFJIRMR0hgGbPoBvH4WyQug7AS55CPxr37XjVZBFxO3Z7Abvrz3A/K93YbXZmXhFW/55STz+OglPRMQ1snc7DqdIXQVNesHQ5yCmndmpTKOCLCJubceRfCZ/tpVNh45zSet6PHZNR5pFh5odS0TEO1SUwS8LYNVT4B/sKMYX/h18avcOCKcLcnp6OhMnTuTYsWNYLBZuuOEGbrnlFldkE5FarLi8gue+380bv+wnMtifBX/rwvAuDXUSnoiIq6Sudly6LTvF8fbQgx+HcL3jKLigIPv6+jJp0iQ6dOhAYWEh1113HX369KFVq1auyCcitdCGw8X884ufScst4YZujZl8VQKRId53IXoREVMU5xC3bg7sXwqRTeHmT6D1ALNTuRWnC3JMTAwxMTEAhIWFER8fT2ZmpgqyiJy1owVlzFq2g6WbM4ivH8pHd/SiV3ztuKSQiMg5Zxiw9b/wzSPUKc6Bi8bBpZMgQIet/f8shmEYrtpYWloao0ePZtmyZYSFhVV+PykpiZCQml+4v7S0lKAgzz+D0lvmAM3ijjx5Drth8O3uAt5MyqG0ws71CeHcdGE9Anw9/3AKZ34vxcXFJCYmVuu+zqyznvzfzv9Ps7gfb5kDPHsW/8LDNEh6krCM3yiJas+BCx7AiO1kdiynOfs7Od0667KT9IqKihg3bhyTJ0/+Qzk+KSEhocbbTk5Odurx7sJb5gDN4o48dY7dmQVM/mwr61Nz6dkiijnXdsJ67JBHznIqzvxekpKSzur+NX0eT/1v51Q0i/vxljnAQ2exWeHXF+HHeeDjB1fOJ7j77Ri7UjxvllNw9ndyunXWJQXZarUybtw4hg4dyqBBg1yxSRHxcqVWGy+v3MMrP+0lNNCP+ddfwMjExlgsFpKPmZ1ORMQLHFoPS/8FWduh3dVw5Xyo08jsVB7B6YJsGAaPPvoo8fHx3Hrrra7IJCJebs3ebB79bBv7s4u49sJGTBmSQHRYoNmxRES8Q2k+rJgF69+AiIZw4wfQbojZqTyK0wU5KSmJJUuW0KZNG4YPHw7A+PHj6devdr0loYicWU5ROXO+TOaT39NoGhXCu7f14JLW9c2OJSLiHQwDkr+A5Q9DQQb0vBP6T4HAcLOTeRynC3K3bt3YtWuXK7KIiJcyDINPfz/MY1/uoKC0grsvbcm4/q0JDvA1O5qIiHc4fgi+mgApy6FBJ7jxfWhUvZN85c/0Tnoick7tzy5iyudbWb3nGF2bRjJ3RCfaNYgwO5aIiHewVcC61+GHxwADBj0GPe8GX1U8Z+jVE5FzorzCzus/7+X5H/YQ6OvD7Gs6cnOPpvj4eP6l20RE3MKRTbB0HKRvhtaD4KqnoG4zs1N5BRVkEXG5Dak5PPLpVnZnFTKkUxzTh7YnJsIzrx0qIuJ2ygph5VxY+wqE1IPr34YO14JFOyBcRQVZRFwmr8TKvK938sHagzSKDObNW7pxeUKs2bFERLxHyjfw5YOQdwgSb4UB0yG4rtmpvI4Ksog4zTAMlm1JZ+bSHeQUlXH7xS14YGAbQgO1xIiIuER+Onz9MOxYAvXbwT++gaa9zE7ltfS3l4g45VBOMVOXbOPHXUfp1KgOi27tTsdGdcyOJSLiHex2SHoLvp8JFWXQfypcNA78AsxO5tVUkEWkRipsdt78ZT/Pfp+Cj8XCtKvbc8tFzfHVSXgiIq6Rud3xTnhp66FFP7j6WYhuaXaqWkEFWUTO2uZDx3nk063sSM9nQEIss4Z3oGFksNmxRES8g7UEfpoPa56HoDpw7etwwQ06Ce88UkEWkWorLKvgqW92sfjXVOqHBfLq6K4M7tAAixZtERHX2PsDLHsAclOhy82O6xqHRJmdqtZRQRaRavl2ewbTlmwns6CUMb2a8dDgtkQE+ZsdS0TEOxQehW8mw9b/QFRLuGUptOhrdqpaSwVZRP5Sel4J05ds59sdmbRrEM7Lo7vStakuKSQi4hKGARvfg2+nQHkR9J0IlzwI/rp2vJlUkEXklGx2g/d+O8CT3+yiwm5n0pXtuO3iFvj7+pgdTUTEO2TvhqX3w4FfoGlvuHoBxLQzO5Wggiwip7DjSD6PfLaVzYeOc0nresy5phNNo0PMjiUi4h0qyuCXZ2HV0+AfDEOfhwvHgI92QLgLFWQRqVRcXsFz3+/mjV/2UzfEn+du7MKwzg11Ep6IiKuk/uLYa3xsN3QaCYPnQliM2ank/6OCLCIArNyVxdTPt5GWW8LfujXhkavaERmiC9GLiLhEcQ58N9VxvHFkMxj9CbQaYHYqOQ0VZJFaLquglFlLd7BsSzot64fy7zt60TM+2uxYIiLewTBgy38cV6goyYU+90O/hyFAh625MxVkkVrKbjf4aP0hnlieTKnVzv0DWnP3pS0J9PM1O5qIiHfI2QfLxsO+ldAoEf7+OTToZHYqqQYVZJFaaHdmAZM/28r61Fx6xUcx59pOtKwfZnYsERHvYLM63gXvp/ng4w9XPQXd/gE+2gHhKVSQRWqRUquNl1bu4dWf9hIa6Mf86y9gZGJjnYQnIuIqh9bB0n9B1g5IGApXzoeIhmankrOkgixSS6zZk82jn29jf3YRIy5sxKNDEogOCzQ7loiIdyjNg+9nwoa3HIX4xg+h3VVmp5IaUkEW8XI5ReU89uUOPv39MM2iQ3jvtp5c3Lqe2bFERLyDYcCOJbD8YSjKgl53w2WTITDc7GTiBBVkES9lGAaf/H6YOV/uoKC0gnsva8l9/VsT5K9j4EREXOL4IfjqIUj5GhpcADd9CI26mp1KXMAlBfmRRx7hxx9/JDo6mmXLlrlikyLihP3ZRTz62VbW7D1GYrO6zL22E20baG+GiIhL2Cpg7auwci5gwKDHoOfd4Kv9jt7CJb/JESNGMHr0aB5++GFXbE5Eaqi8ws5rP+3lhZV7CPTzYc61Hbmpe1N8fHQSnoiISxzZ6DgJL30ztB4MQ56CyKZmpxIXc0lB7t69O2lpaa7YlIjU0LbMUu5bvoo9WYUMuSCO6Ve3JyYiyOxYIiLeoayA2N+fhT0fQ2gMjFwE7a8BXQXIK+nfAkQ8XF6xlSe+TubDdUdoFBnM22O7c1m7GLNjiYh4j51fwVcTiMpPg263wYDpEFTH7FRyDlkMwzBcsaG0tDTuuuuuUx6DnJSUREhIzd9SsbS0lKAgz98T5i1zgGZxB4Zh8FNqEa+vO0ZemY2rW4dya7f6BPn7mB3NaZ76OzkVZ2YpLi4mMTGxWvd1Zp3V6+2evGUWT57Dr+Qosb8/Q0TaSkoj4jnYeTy2ht3MjuUSnvx7qcrZOU63zp63PcgJCQk1fmxycrJTj3cX3jIHaBazHcopZsrn2/gp5SgXNK7Du9d2wjf/iMfNcTqe+Ds5HWdmSUpKOqv71/R59Hq7J2+ZxSPnsNsc1zNeMQts5XD5NIJ634dt917Pm+U0PPL3cgrOznG6dVaHWIh4EKvNzlu/7OfZ71PwtViYdnV7brmoOb4+FpLzj5gdT0TE82Vsc5yEd3gDxF8KQ56B6JZmp5LzzCUFefz48axbt47c3Fz69u3Lfffdx8iRI12xaRE5YdOh40z6ZAs7MwoY2D6WmcM60DAy2OxYIiLeobwYfpoHv74IQZEwYiF0GqmT8GoplxTkZ555xhWbEZFTKCyr4KlvdrH411RiwgN5dXQiV3RsYHYsERHvsWcFLHsAjh+AC0fDwNkQEmV2KjGRDrEQcWPfbM9g+pLtZBaU8vdezXhocFvCg/zNjiUi4h0Ks+CbybD1Y4huDWO/hOYXm51K3IAKsogbSs8rYdqS7Xy3I5N2DcJ5ZXRXLmxa1+xYIiLewW6Hje/Cd9PAWgz9JsHFD4C/51/VQVxDBVnEjdjsBu/8mspT3+zCZhg8cmU7/nFxC/x9Pf/SbSIibuHoLlh6PxxcA836wNULoH4bs1OJm1FBFnET24/kMfnTrWxOy6Nfm/o8dk1HmkTV/PrhIiJSRUUZrHoGVj0NAaEw7AXoMhp8tANC/kwFWcRkxeUVLPh+N2/+sp+6If48f9OFDL0gDovOnBYRcY39q2DZ/XBsj+PKFIMfh7D6ZqcSN6aCLGKilTuzmPL5Ng4fL+GmHk2YdEUCdUJ0Ep6IiEsU58C3U2HTe1C3OYz+FFpdbnYq8QAqyCImyMovZeayHXy5JZ1WMWH8587e9GihSwqJiLiEYcCWfzuuUFGa5zgBr+9ECNBha1I9Ksgi55HdbvDh+oM8sXwnZRV2HhzYhjv7tSTAT8fAiYi4xLG98OV42PcjNO4BQxdAbAezU4mHUUEWOU9SMgt45NOtJB3IpXd8NHOu7Uh8/TCzY4mIeIeKcljzPPz8JPgGwJCnIfEfOglPakQFWeQcK7XaePGHPbz2817CAv14amRnruvaSCfhiYi4ysG1sPRfcDQZ2g+HK+ZBRJzZqcSDqSCLnEOr92Tz6GdbST1WzIiujZgypD1RoQFmxxIR8Q4lx2HFTNjwFkQ0hps+grZXmp1KvIAKssg5cKywjDlfJvPpxsM0jw7h/dt70qdVPbNjiYh4B8OAHZ/D8oeh6Cj0uhcumwyBOmxNXEMFWcSFDMPg46Q05n6VTFFZBff1b8W9l7UiyN/X7GgiIt7h+EH48kHY/S3EdYZR/4aGF5qdSryMCrKIi+w7Wsijn23j133H6NasLo+P6ETr2HCzY4mIeAdbBax9FVbOASwweC70uBN8VWXE9fRflYiTyipsvPrjPl5auYdAfx/mXNuRm7o3xcdHJ+GJiLjE4d8dJ+FlbIE2V8BVT0FkE7NTiRdTQRZxwrr9OUz+bCt7sgq5+oI4pg1tT0x4kNmxRES8Q1kB/DAH1r0GoTFwwzuQMAx0FSA5x1SQRWogr9jK48uT+Wj9IRpFBvP22O5c1i7G7FgiIt5j51fw1UOQfwS63waXT4OgOmanklpCBVnkLBiGwdIt6cxaup3cYit39o3nXwNaExKgP0oiIi6RfwS+mgA7l0FMBxi5GJp0NzuV1DL6W12kmg7lFPPo59v4OeUonRvXYfE/etChofZmiIi4hN0G69+EFbPAboUBM6D3/4Gvv9nJpBZSQRY5A6vNzhur9vPcihR8LRZmDG3PmN7N8dVJeCIirpGx1XES3uEkiL8Mrn4GouLNTiW1mAqyyF/YeDCXRz7dys6MAga1j2Xm8A7E1Qk2O5aIiHcoL4Ifn4BfX4LgujDiDeh0vU7CE9OpIIucQkGplSe/2cW7vx0gNjyI18YkMrhDA7NjiYh4j93fw5cPON7448IxMHAWhESZnUoEUEEW+QPDMPhmewbTv9hOVkEZt/RuzoOD2hAepGPgRERcojALvp4E2z6B6NYw9ito3sfsVCJ/4JKC/PPPPzNnzhzsdjsjR47kjjvucMVmRc6rI8dLmLZkO98nZ5IQF8HrY7rRuUmk2bFERLyD3Q4b34HvpoG1BC59BC5+APwCzU4m8idOF2SbzcasWbN4++23iY2N5frrr6d///60atXKFflEzjmrzeCtX/bz9Le7sBsw+ap2/KNPC/x8fcyOJiLiHY7ucpyEd/BXaH4JXP0s1GttdiqR03K6IG/ZsoVmzZrRpInjLR+HDBnCihUrXFaQDx8vIfloKSWhuS7ZnplSvWQO8I5ZDueW8N2OTL7fkU6x1eDStvWZPbwjTaJCzI4mIuIdrCXU3/oa7HwPAsNg+EvQ5WadhCduz+mCnJmZSYMG/zt5KTY2li1btji72UqXP/0jpVY7cMRl2zSXt8wB3jBLdGgAlzQPY9QlCfRtXQ+LFm0REdfY+wN8+SD1cvbBBTfCoMcgrL7ZqUSq5bydpJecnFyjxy24qiGHc4sJCAhwcaLzr7y83CvmAO+YJTTAhzbRgVjLywiyZbNzZ7bZkZxSWlpa4z9n7kaz1ExNn0evt3vy1Fn8itKJ3fQ8EWkrKQtrwsGLnqGiyUVwKBvQOusuvGWWczWH0wU5NjaWjIyMyq8zMzOJjY390/0SEhJqtP0EHIt+TR/vTrxlDtAs7shb5gDNclJSUtJZ3b+mz6PX2z153CzlxfDri7DqGcfXl00h8KL7qNiz37Pm+Ase9zv5C94yi7NznG6ddbogd+rUidTUVA4dOkRsbCxffvklTz/9tLObFREREU9gt8PW/zjeIjr/MCQMg8FzIbKJ2clEaszpguzn58e0adO4/fbbsdlsXHfddbRurTNTRUREvJphOI4zXjET0jdDXBcY8To0v9jsZCJOc8kxyP369aNfv36u2JSIiIi4u4NrHXuMD/wCdZrAta9Dp5Hgo8tjinfQO+mJiIhI9RxcCz894dhzHBoDVz4JibfozT7E66ggi4iIyOkZBuz/yXHy3f6fIKQeDJgJPf4JAaFmpxM5J1SQRURE5M/sNkj+An5ZAOmbICwWBs2BbreqGIvXU0EWERGR/yk5DhvfhXWvw/GDENUShj7neLMP/yCz04mcFyrIIiIiApnbYf2bsPkjsBZBsz6OPcbthoCPr9npRM4rFWQREZHaqrzYcRhF0iI4+Cv4BkLHEdDzLmjYxex0IqZRQRYREalNDAMO/w6b3oOt/4WyfKjbAgbOhgtHQ0iU2QlFTKeCLCIiUhvkHnC8493mf8Ox3eAXBO2Hw4VjHIdT6BrGIpVUkEVERLxV/hHY/jls/xTS1ju+16wPXHSfoxwHR5qZTsRtqSCLiIh4k5x9kLzMcWzxyVLcoBMMmAEdRkDdZqbGE/EEKsgiIiKezG6DtA2Qshx2LYejOx3fb3AB9J8CCcOhfhtzM4p4GBVkERERT5OfDvtWwu7vHG/7XHocfPyg2UXQ9RZoeyVEtTA7pYjHUkEWERFxdyW5cGAN7F/lKMYn9xKHxTquU9xqALTsr2OKRVxEBVlERMTdFB4l/NAPkLoYDq6B9C2A4bjyRNPe0GUUxF8GsR119QmRc0AFWURExEwV5ZC13XEccdp6OLQOcvfTGMAvGBp3g0snQfNLHLf9As1OLOL1VJBFRETOF5sVju6C9E2QvhmObHTsHbaVOX4eGgNNekC3W0m1x9G89zXgF2BmYpFaSQVZRETkXCjKhsztkLXD8TljK2Ql/68MB4Q5rjTR45/QKBEadYXIZmCxAFCSnKxyLGISFWQREZGaMgwoSIfsFMje7dg7fHSn46Po6P/uFxLtKMM973RckziuC0S30vHDIm5KBVlEROSv2O1QmAE5+yF3v+Nzzl44tgeO7QNr0f/uGxgB9dtCmyugfjuIbQ8xHSAspnLPsIi4PxVkERGp3WxWx17gvMOQlwZ5B+H4ITh+EI4fcNw+eVgEgMUXIps69gA3uxiiW0K9No6P8AYqwiJeQAVZRES8k90GxcegMNPxUZDpKMIFGY7P+UccnwszwbD/8bEh0VCnCcS0h7ZXOd6euW5ziIp3fN/X35SRROT8UEEWERHPUFHueMe44mNQnAMlOY4T4YqPOT6KjtIkKxV+LIHCLCjO/nPxBQiKdOzpjWjoOAQiopHjo04jiGgMdRpDYNh5Hk5E3IkKsoiInB8VZVBWCGX5Jz4KoPTE7dJ8KMuD0hMfJccdZbgkF0ryHJ/LC06/7YAwCK2Hj08YRDdxXBEiNMZx7G9YrKMQh8VAeBz4B5+viUXEQzlVkJcvX86LL77I3r17+fjjj+nUqZOrcomIyPlms4K1xFFkK0rAWvq/z9ZiqDjxubzYcT9rkeNzedH/vl9e6Pi6vMhxu6zQUWzLCsFuPXMG/xDHiW5BdSC4rmPPbkwHCIlyfB1c98TtKMfnkHqOwyH8gwA4kJxMQkLCOX6hRMTbOVWQ27RpwwsvvMD06dNdlUdERM6nTR/SZtl4x0lohq1m2/APgYDQE5/DICDEcTssxvF1YFiVz+EQFAGB4Sc+TpThk5913V8RcQNOFeSWLVu6KoeIiJghrjPHW15LdEwc+AU53sbYP9hxu+rnyo8Qx/dOFmK/IF3LV0S8jsUwDMPZjYwZM4aJEyee9hCLpKQkQkJCarz90tJSgoKCavx4d+Etc4BmcUfeMgdolpOKi4tJTEys1n2dWWf1ersnb5nFW+YAzeKOnJ3jdOvsGfcgjx07luzs7D99//7772fAgAHVDuDMMWHJXnJMmbfMAZrFHXnLHKBZTkpKSjqr+9f0efR6uydvmcVb5gDN4o6cneN06+wZC/KiRYtq/KQiIiIiIp5GB46JiIiIiFThVEH+7rvv6Nu3Lxs3buTOO+/ktttuc1UuERERERFTOHUVi4EDBzJw4EBXZRERERERMZ0OsRARERERqUIFWURERESkChVkEREREZEqVJBFRERERKpQQRYRERERqUIFWURERESkChVkEREREZEqVJBFRERERKpQQRYRERERqUIFWURERESkChVkEREREZEqVJBFRERERKpQQRYRERERqUIFWURERESkChVkEREREZEqVJBFRERERKpQQRYRERERqUIFWURERESkChVkEREREZEqVJBFRERERKpQQRYRERERqUIFWURERESkChVkEREREZEqLIZhGOf6SZKSks71U4iIeKXExMRq3U/rrIhIzZxqnT0vBVlERERExFPoEAsRERERkSpUkEVEREREqlBBFhERERGpQgVZRERERKQKFWQRERERkSpUkEVEREREqlBBFhERERGpQgVZRERERKQKFWQRERERkSpUkEVEREREqlBBFqli0qRJtG3blrZt2/LCCy+YHUdExOOtXbu2cl3t379/jbbxwgsvVG5j0qRJLk4o8mcqyCIiIiIiVfiZHUBERES8V/v27Xn//fcBCAwMNDmNSPWoIIuIiMg5Ex4eTrdu3cyOIXJWdIiFeKT8/HzmzZvHFVdcwQUXXEDHjh25+OKLGT16NPPmzaOkpKTyvnv37uXRRx/l8ssvp1OnTnTt2pUbb7yRTz/9FMMwqvV8f3Vscv/+/St/tnbtWpfOKSJyrlRnHZ0yZUrl+vbOO+9UPvbFF1+s/P5///vfyu/Pmzev8vsvvvgi8NfHIC9ZsoRRo0bRvXt32rdvT8+ePRk6dCiTJk1i06ZNp82+ZcsWbr31Vi688EISExO5//77OXbsmGtfIKnVtAdZPNI999zD+vXr//C9o0ePcvToUdavX88//vEPgoOD+f777xk/fjxlZWWV9ysvL2fjxo1s3LiRNWvW8OSTT2KxWM73CCIipqrOOtqrVy8+/vhjADZs2MDf//53gD88bsOGDVx//fV/+n6vXr3+8vk/+eQTJk+e/IfvHT9+nOPHj5OSkkJ8fDxdunT50+M2btzIsmXLsFqtld9bvnw5BQUFvPnmm9WYXOTMVJDF4+Tk5FQuwnFxcUycOJG6dety9OhRUlJS+Omnn7BYLOTk5DBhwoTKcnzjjTcyYMAAcnJyeO655zh8+DBLly6lV69elYu7iEhtUN11tGrJTUpKAsBqtbJ58+bK72/YsAGAwsJCduzYAUBISAidO3f+ywzffvtt5e377ruPxMREioqKSEtL45dffjnt8cqpqan06dOH0aNHs3379so91b/88gv79u0jPj7+bF8OkT9RQRaPExoaiq+vLzabjfDwcJo1a0arVq0qF9OHHnoIgPfee4/i4mIA2rRpw9ChQwFo1KgRQ4cO5dVXXwXg448/VkEWkVqluusoQOvWrdm9ezfZ2dns27eP/Px8SkpKaNq0KdnZ2Rw6dIjMzEx27dqFzWYDIDExEX9//7/MEBYWVnm7RYsWtG3blqioKADGjh172sfVrVuXl19+maCgIPr3789XX33Fvn37AEd5VkEWV1BBFo8TGBjINddcwyeffEJKSgojRozAx8eHuLg4OnfuzIgRI7jkkkvYs2dP5WNSUlK4+eabT7m9lJSU8xVdRMQtVHcdBcehErt37wYce5Hz8vIA6NmzJ4cPH2bNmjVs2LCBXbt2VW7/TIdXANxwww0sX74cm83G+PHjAahTpw5t27bl0ksvZdSoUQQHB//pcV26dCEoKKjy68jIyMrbJ7OJOEsFWTzS7Nmz6dWrFz/++CO7d+/mwIEDHD58mMOHD/PVV1/x0ksvVXtbJ/cy/5WqxyhXVFT84We5ubnVDy4i4iaqs44OGDCAXr168e677wKOY4zz8/MB6NatG3FxcaxZs4b169ezc+fOym337t37jM/fs2dPPvnkEz799FO2bdvG/v37yc3NZd26daxbt47ff//9lGt51UIM4Of3vypT3ROvRc5EBVk8ko+PD8OGDWPYsGEA2O123n77bebPnw/Al19+SdeuXSvvf+GFF/LRRx+dclvVKcgRERGVtzMzMytv//rrr9V6vIiIu6nOOjpgwAB69OhReTjGhg0bKCwsBKB79+40atQIgDVr1nDkyBHAUWATEhLO+PyGYZCQkMCjjz5a+b2DBw8yfPhwiouLWbFiBSUlJafciyxyrqkgi0caOHAg/fr1o0OHDsTExGCz2f5w9nRZWRlXXXUVzzzzDMXFxWzcuJFx48Zx9dVXEx4eTmZmJvv37+enn35iwIAB/N///d9fPl/z5s0rby9btozGjRsTEBCgM6ZFxGNVZx0Fxw6C9u3bs3XrVg4fPgw4Tuxr1KgR9evXJyAggAMHDlQ+rkePHvj4nPkqsnPnzuXQoUP06dOHuLg4wsLC2LFjB6WlpYCjQJeXl6sgiylUkMUjZWZm8t57753259dccw3R0dHMnz+fBx98kLKyMr755hu++eabP9338ssvP+PzDR06lAULFnD8+HGsVmvltZAbNGhARERE5T85ioh4iuqsoyf16tWLrVu3Vn598o0/AgICuOCCCyqvZHHyvtVRVlbGypUrWbly5Sl/3r9/f+rUqVOtbYm4mt4oRDzSgw8+SP/+/WnUqBEhISH4+voSFRXFJZdcwuuvv86gQYMAxx6Szz77jL/97W80a9aMwMBAQkJCaNasGZdddhkzZ85k1KhRZ3y+sLAwFi5cSGJiIgEBAURGRjJ8+HD+85//EB4efq7HFRFxuequo/DnY4q7d+9+yttQ/YJ89dVXM3LkSNq0aUNkZCS+vr6EhITQvn17xo0bx4IFC2o+nIiTLIaOaBcRERERqaQ9yCIiIiIiVaggi4iIiIhUoYIsIiIiIlKFCrKIiIiISBUqyCIiIiIiVaggi4iIiIhUoYIsIiIiIlKFCrKIiIiISBXn5a2mk5KSzsfTiIh4ncTExGrdT+usiEjNnGqdPS8F+XRPXl3JyckkJCS4MI05vGUO0CzuyFvmAM1y0tmW3pqus3q93ZO3zOItc4BmcUfOznG6dVaHWIiIiIiIVKGCLCIiIiJShQqyiIiIiEgVKsgiIiIiIlWoIIuIiIiIVHHermIhIlKbZBVn8fzvz9PUaEoCnn+muIhIbaI9yCIiLmQYBp/t/oxrPr+Gb1K/IdQv1OxIIiJylrQHWUTERTKKMpjx6wxWH15N15iuzOozi+LDxWbHEhGRs+R0QS4rK+Pmm2+mvLwcm83G4MGDGTdunCuyiYh4BMMw+GT3Jzy14Snshp1JPSZxU7ub8LH4kHw42ex4IiJylpwuyAEBASxevJjQ0FCsViujRo2ib9++dOnSxQXxRETc2+HCw8xYM4Pf0n+jR4MezLhoBk3Cm5gdS0REnOB0QbZYLISGOo6xq6iooKKiAovF4nQwERF3Zjfs/HvXv3k26VksWJjaayrXt7keH4tO7RAR8XQWwzAMZzdis9kYMWIEBw8eZNSoUUyYMOEPP09KSiIkJKTG2y8tLSUoKMjZmKbzljlAs7gjb5kD3H+WjNIMXtv/GtsLtnNBxAXc2eJO6gfWP+V9nZmluLiYxMTEat3XmXXW3V/vs6FZ3I+3zAGaxR05O8fp1lmXnKTn6+vLkiVLyM/P59577yUlJYU2bdr84T4JCTW/zFFycrJTj3cX3jIHaBZ35C1zgPvOYjfsfJD8Ac/veB5fiy8zL5rJta2u/ct/NXNmlqSkpLO6f02fx11f75rQLO7HW+YAzeKOnJ3jdOusS69iERERQc+ePVm1atWfCrKIiCdLzUtl+prp/J71O5c0uoRpvafRILSB2bFEROQccPpguZycHPLz8wHHbu41a9YQHx/vdDAREXdgs9tYtG0R1y+9nt3HdzPn4jm8dPlLKsciIiaryMmBpN/Pybad3oOclZXFpEmTsNlsGIbBFVdcwWWXXeaKbCIiptp7fC/TVk9jS/YWLmtyGVN7TaV+yKmPNRYRkfOn4IcfSJ86DYqKsF9/HT4uPp7a6YLcrl07Pv/8cxdEERFxDxX2ChZtX8TLm14mxD+EeZfM48oWV+oKPSIiJrMVFJA593HyPvuMwIQEbHfe4fJyDHonPRGRP0jJTWHq6qnsOLaDgc0GMrnnZOoF1zM7lohIrVf0228cmTyZioxMou++i/p3383OvXvPyXOpIIuIAFa7lTe2vsHrW14nIiCCp/s9zaDmg8yOJSJS69lLSsh65lly332XgObNaf7hBwR37nxOn1MFWURqvZ05O5m6eio7c3ZyZYsreaTHI9QNqmt2LBGRWq9kyxaOPDyJ8v37qTt6NDEPjscnOPicP68KsojUWlablde2vMabW9+kTmAdFly2gMubXm52LBGRWs8oLyf71VfJfu11/OrXp+nbbxHau/d5e34VZBGplbZnb2fK6insOb6HYS2HMbH7ROoE1jE7lohIrVe2Zw9HJj5M6Y4d1Bk+nNhHJ+MbEXFeM6ggi0itUmYr4+VNL7No+yLqBdfjpctfom/jvmbHEhGp9Qy7nZx33uHoM8/iExpKo+efI2KQOeeCqCCLSK2x+ehmpq6eyv68/YxoPYKHuj1EeEC42bFERGq98rTDpE+eTPG6dYT170/crJn41TPvCkIqyCLi9UorSnlx44u8s+MdYkNjeXXAq/Rp1MfsWCIitZ5hGOR9+hmZc+cCEDdnDnVGXGv6dedVkEXEq/2e+TvT1kzjQP4BRrYZyfjE8YQFhJkdS0Sk1qs4doz0adMpXLGCkO7diXv8cQIaNzI7FqCCLCJeqthazPMbn+eD5A9oGNaQNwa9Qc+4nmbHEhERoGDFCtKnTsNeWEjMxIlEjb0Fi4+P2bEqqSCLiNdZn7GeaaunkVaYxk3tbuL+rvcT4h9idiwRkVrPVlhI5py5jreKbp9Ao3mLCGzd2uxYf6KCLCJeo8haxLNJz/LvXf+mSXgT3h78Nt0adDM7loiIAEVr15H+yCNYMzKIvutO6t9zD5aAALNjnZIKsoh4hV+P/MqMNTNIL0pnTPsx3HfhfQT7nft3WxIRkb9mLyvj6LMLyFm8GP+mTWj+wfsEd+lidqy/pIIsIh6toLyApzc8zSe7P6F5RHPeufIdusR0MTuWiIgApTt2cHjiRMr37CXyphuJnTABnxD3P+RNBVlEPNaqtFXM/HUmR0uOcmvHW7mn8z0E+QWZHUtEpNYzKio49sYbHH3xJfzq1qXJwoWEXXKx2bGqTQVZRDxOXlke89fP54u9X9CyTkuevfRZOtXvZHYsEREBylNTOfLwJEo2bybiqitpMG0avpGRZsc6KyrIIuJRVh5cyezfZpNTmsM/O/2TuzrfRYCve57kISJSmxiGwfF//5vMefOx+PvT8KmnqHP1ELNj1YgKsoh4hOOlx3l83eN8tf8r2tRtw4uXv0j76PZmxxIREcCamUX6lCkUrVpF6EUXETd3Dv4NGpgdq8ZUkEXE7X1/4Htm/zab/LJ87ul8D7d3uh1/X3+zY4mICJD/9ddkTJ+BvayM2KlTqHvTTW71ph81oYIsIm7rWMkx5q6dy7cHviUhKoHXB75O26i2ZscSERHAlpdHxmNzyF+6lKALLqDhE08QGN/C7FguoYIsIm7HMAy+3v81c9fOpdBayLgLxzG241j8fbTXWETEHRStWcORRyZTcewY9cbdR7077sDi5z210nsmERGvkF2SzdN7nmZd7jo61evErItm0apuK7NjiYgIYC8pIevpZ8h97z0C4uNp/uKLBHfqaHYsl3O6IKenpzNx4kSOHTuGxWLhhhtu4JZbbnFFNhGpRQzDYNm+ZTyx7glKrCWMTxzPmPZj8PPR/8eLiLiDkq1bOTLxYcr376fumDHEPDgenyDvvPa803/z+Pr6MmnSJDp06EBhYSHXXXcdffr0oVUr7fERkerJLMpk9m+z+SntJzrX78zYBmMZ0HGA2bFERAQwrFayX3ud7Fdewa9+fZq+/RahvXubHeuccrogx8TEEBMTA0BYWBjx8fFkZmaqIIvIGRmGwed7PufJ9U9itVuZ0G0CNyfcTMquFLOjiYgIULZvH0cenkTp1q1EDB1Kg6lT8I2IMDvWOefSf7tMS0sjOTmZzp07u3KzIuKF0gvTmfnrTFYfWU1ibCKzLppF04imZscSERHAsNvJff8Dsp56Cp+gIBotWEDEFYPNjnXeWAzDMFyxoaKiIsaMGcNdd93FoEGD/vCzpKQkQkJCarzt0tJSgrzgGBdvmQM0izvylDkMw2DF0RW8e/Bd7Ni5ucnNDIoZhI/lf9fM9JRZqsOZWYqLi0lMTKzWfZ1ZZ/V6uydvmcVb5oBaNEt2NrzwImzZAl27wr33QFTU+Q1YTc7+Tk63zrpkD7LVamXcuHEMHTr0T+X4pISEhBpvPzk52anHuwtvmQM0izvyhDkOFx5m+prprE1fS8+4nszoPYPG4Y3/dD9PmKW6nJklKSnprO5f0+fR6+2evGUWb5kDvH8WwzDIX7qUjNmPYdhsxM6cSeQNI7FYLCalPDNnfyenW2edLsiGYfDoo48SHx/Prbfe6uzmRMQL2Q07/971b55NehYfiw/Tek/j+tbXu/WiKyJSm1Tk5pIxYyYF33xD8IUX0nDeEwQ0rb2HvTldkJOSkliyZAlt2rRh+PDhAIwfP55+/fo5HU5EPN/B/INMXzOdDZkb6NOwD9N7TycuLM7sWCIickLBjz+SPmUqtrw86j84nuh//AOLr6/ZsUzldEHu1q0bu3btckUWEfEiNruND3Z+wPO/P4+/jz+zLprFNa2u0V5jERE3YSssImveExz/+L8EtmlD0zffIKhtW7NjuQVdgV9EXG5/3n6mrZ7GpqOb6Nu4L9N6TSM2NNbsWCIictKOHey/bxzWI0eI/uc/qXff/+ETEGB2KrehgiwiLlNhr+CdHe/w0saXCPILYu7Fc7k6/mrtNRYRcRP2sjKOPvc8vP02NGlCs/feJaRrV7NjuR0VZBFxiT25e5i6eirbjm2jf5P+TOk1hfoh9c2OJSIiJ5Rs3076pEmU7d4DgwcTP3cOPqGhZsdySyrIIuKUCnsFb297m1c2v0Kofyjz+87niuZXaK+xiIibMKxWsl9/nexXXsUvKoomC1/nUL16Ksd/QQVZRGpsV84upq6eSnJOMoOaDWJyz8lEB0ebHUtERE4o27vX8VbR27YRcfXVNJjyKL6RkZCcbHY0t6aCLCJnzWqz8sbWN3h96+tEBETwzKXPMLDZQLNjiYjICYbdTs7idzj67LP4hITUureKdpYKsoiclR3HdjB19VRSclO4qsVVPNLjESKDIs2OJSIiJ5QfOkT6I5Mp3rCBsP79iZs1E7969cyO5VFUkEWkWspt5by6+VXe2vYWUUFRPHfZc/Rv2t/sWCIicoJhGBz/97/JnP8kFh8f4h5/nDrXDNc5ITWggiwiZ7T16FamrZnGnuN7GNZyGBO7T6ROYB2zY4mIyAnWjAzSH51C0erVhF50EXFzHsM/Tu9aWlMqyCJyWqUVpby86WUW71hMveB6vHT5S/Rt3NfsWCIicoJhGOR99jmZjz+OYbPRYMZ0Iv/2N+01dpIKsoic0qasTUxdPZXU/FSua30dD3Z7kPCAcLNjiYjICdbMLDKmT6fwxx8J7pZIw7lzCWja1OxYXkEFWUT+oKSihBc2vsB7O96jQWgDXhvwGhc1usjsWCIicoJhGOQvW0bGY3MwSkuJfWQSdceMweLjY3Y0r6GCLCKVkjKTmLZ6GgcLDvK3tn/jgcQHCPXXheRFRNxFxdGjpM+YSeGKFQR36ULc3LkExrcwO5bXUUEWEYqtxSz4fQEf7vyQRmGNeHPQm/SI62F2LBEROcEwDPK//IrM2bOxl5QQM2ECUWNvweLra3Y0r6SCLFLLrU1fy/Q10zlSeISbE25m3IXjCPEPMTuWiIicUHH0KOkzZ1L4/QqCOl9Aw7lzCWzZ0uxYXk0FWaSWKiwv5JmkZ/g45WOahjfl7SveJjE20exYIiJywsljjTMfm3Nir/FDRI0dq73G54EKskgttObwGmb8OoOMogxuaX8L9154L8F+wWbHEhGRE6xZWWTMmEnhDz8Q3LkzcY/PJTA+3uxYtYYKskgtkl+ez1Prn+KzPZ/Rok4L3rnyHbrEdDE7loiInGAYBnmfL3Fc17isjJiHHybq72O01/g8U0EWqSV+TvuZmb/OJLskm9s63sbdXe4m0DfQ7FgiInKC9cgR0qfPoGjVKoITE4l7bDaBLXSFCjOoIIt4ubyyPOavn88Xe7+gVWQrnrvsOTrW62h2LBEROcGw2zn+n/+Q9eRTGIZB7JQp1B11k65rbCIVZBEv9sPBH5j922xyS3O544I7uPOCOwnwDTA7loiInFCemkr6lKkUb9hA6EW9aTBrFgGNG5sdq9ZTQRbxQrmluTy+7nGW719O27ptefnyl0mITjA7loiInGBUVHDs7bfJfvElLAEBxM15jDojRmCxWMyOJqggi3idb1O/Zc7aOeSX53NPl3u4vdPt+Pv4mx1LREROKN2xgyNTplC2I5nwgQOInTIV/9gYs2NJFS4pyI888gg//vgj0dHRLFu2zBWbFJGzlGfNY/yP4/nuwHckRCWwcNBC2tRtY3YsERE5wV5SQvZLL3Hs7UX4RtWl0fPPETFokNmx5BRcUpBHjBjB6NGjefjhh12xORE5C4ZhsHz/ch7b+hil9lL+1fVfjO0wFj8f/QORiIi7KFy9mowZM7EeOkTkyOuJmTAB34gIs2PJabjkb9Du3buTlpbmik2JyFk4WnyU2b/NZuWhlbQKbcVTA56iZaTeflRExF1U5OSQ+cQT5H+xlIDmzWn6zmJCe/QwO5acgXYxiXggwzBYum8p89bNo8xWxoOJD9Ldp7vKsYiImzAMg7xPPyNr/nxsxcVE330X9e66C59AXX/eE1gMwzBcsaG0tDTuuuuuUx6DnJSUREhISI23XVpaSlBQkDPx3IK3zAGaxUw55Tm8tv81NuZtpG1YW+5ucTcNgxt63Bx/RbM4FBcXk5iYWK37OrPO6vV2T94yi7fMAWcxy6E0eO012L4dEhLg7rugSZNzH/AseMvvxdk5TrfOnrc9yAkJNb/EVHJyslOPdxfeMgdoFjMYhsHnez7nyY1PYrVbebj7w9zU7iZ8fRxvP+opc1SHZnFISko6q/vX9Hn0ersnb5nFW+aAM89iLy0l+9VXOfbmW/iEhBAzexaR113nlm/44S2/F2fnON06q0MsRDxAemE6M36dwZoja+gW241ZF82iSYR77Y0QEanNCn/6iYzZj2FNS6PO8OHEPDwRv6gos2NJDbmkII8fP55169aRm5tL3759ue+++xg5cqQrNi1Sq9kNO/9N+S9Pb3gaA4NHez7KDW1vwMfifnsjRERqI+uRI2Q+/jgF331PQMuWNF20iNBePc2OJU5ySUF+5plnXLEZEakirSCN6Wumsy5jHT3jejLzopk0CmtkdiwREQHs5eXkvPU22a+9BkD9B8cTfcstWAICTE4mrqBDLETcjN2w89HOj1jw+wJ8LD5M6z2N61tfr7cfFRFxE4WrVpH52BzKDxwgfOBAYic9jH8j7cDwJirIIm7kQP4Bpq2exu9Zv9OnYR+m955OXFic2bFERAQgI4NDL7xI4Q8/ENCsGU0WLiTskovNTiXngAqyiBuw2W28l/weL258EX8ff2b3mc3wlsO111hExA3Yi4vJXrgQ3niTIn9/6j84nqhbbsFHh1N4LRVkEZPty9vHtNXT2Hx0M/0a92Nqr6nEhsaaHUtEpNYz7Hbyly0j6+lnqMjMhL59aTl7Nv6xMWZHk3NMBVnEJBX2Ct7Z8Q4vbXyJIL8g5l48l6vjr9ZeYxERN1C8cSNZT8yjZPNmgjp2pNGzz3IgOEjluJZQQRYxwZ7cPUxdPZVtx7ZxedPLmdJrCvWC65kdS0Sk1itPO8zRZ54m/6vl+NWvT9zcudS5ZrjjzT6Sk82OJ+eJCrLIeWS1W3l729u8uvlVwvzDeLLfkwxuNlh7jUVETGbLz+fY66+T8867YLFQ7567ib7tNnxCQ82OJiZQQRY5T3bl7GLq6qkk5yQzuPlgJvecTFSQ3mVJRMRM9vJyjn/0Edkvv4ItL486w4ZR//5/4R+nKwjVZirIIueY1WZl4daFLNyykIjACJ699FkGNBtgdiwRkVrNsNvJ//JLjj73PNa0NEJ69yJ2wgSC2rc3O5q4ARVkkXNo+7HtTFs9jZTcFIbED2FS90lEBkWaHUtEpNYyDIOin38ma8FzlCUnE5iQQJM33iC0z0U63E0qqSCLnAPltnJe3fwqb217i6igKJ6/7Hkua3qZ2bFERGq14g0byHp2ASVJSfg3aULDJ+cTMWSI4wQ8kSpUkEVcbOvRrUxdPZW9eXsZ1nIYE7tPpE5gHbNjiYjUWiWbNnH0+RcoWrMGv/r1aTBjOpEjRmDRG33Iaaggi7hIaUUpL29+mcXbF1M/uD4vX/4ylzS+xOxYIiK1VsnmzRx9+WWKfvoZ36goYiZMoO6om/AJDjY7mrg5FWQRF9iUtYmpq6eSmp/Kda2v48FuDxIeEG52LBGRWqn4941kv/wyRb/8gm+dOo63hh41Spdsk2pTQRZxQklFCc///jzvJ79PXGgcrw98nd4Ne5sdS0Sk1jEMg+LffiP71dcoXrsW37p1iXnoQSJvvAnfMBVjOTsqyCI1tCFjA9PWTONQwSH+1vZvPJD4AKH+WoRFRM4nw2aj4IcfOLbwDUq3bMGvfn1iJk6k7o1/wyckxOx44qFUkEXOUrG1mAW/L+DDnR/SOKwxbw56kx5xPcyOJSJSq9jLyshbsoSct96mPDUV/yZNaDBzJnWuvQYfnXwnTlJBFjkLv6X/xow1MzhSeISbE25m3IXjCPHXHgoRkfOlIjub3A8/IvfDD7Hl5BDUoQONFjxL+MCBWHx9zY4nXkIFWaQaCssLeTrpaf6b8l+aRTRj0RWL6Brb1exYIiK1Rsn27eS+9z75X36JUV5O2KWXEjX2FkJ69tQbfIjLqSCLnMHqw6uZ8esMsoqzGNthLPd2uZcgvyCzY4mIeD17eTkF33xL7gcfULJxI5aQEOpcN4KoMX8nML6F2fHEi6kgi5xGfnk+T61/is/2fEZ8nXjevfJdLqh/gdmxRES8XvnBgxz/z384/smn2HJz8W/alJhJDxM5YgS+ERFmx5NaQAVZ5BR+TvuZmb/O5FjJMW7vdDt3db6LQN9As2OJiHgte1kZBd9+x/H//pfitWvB15ewyy6l7o03EXpRb70dtJxXKsgiVeSV5TFv3TyW7ltKq8hWPH/Z83So18HsWCIiXskwDEo2biLv88/JX74ce0EB/o0bU//+f1Hn2mvxj401O6LUUi4pyD///DNz5szBbrczcuRI7rjjDldsVuS8WnFgBbN/m01eWR53db6LOzrdgb+vv9mxRES8Ttm+feQv+5K8L5dhPXAQS3Aw4QMHEHnttY6T7rS3WEzmdEG22WzMmjWLt99+m9jYWK6//nr69+9Pq1atXJFP5JwrrChkwk8T+Dr1a9pFtePVga/SLqqd2bFERLxK+YED5H/9DfnffE3ZjmSwWAjp1ZN6d95F+KBBerc7cStOF+QtW7bQrFkzmjRpAsCQIUNYsWKFCrJ4hEMFh5iyYwpZ5Vnc2+Vebut0G/4+2mssIuIswzAoS9kN//4P+zZvpiw5GYCgzhcQM+lhIq66Cv+YGJNTipya0wU5MzOTBg0aVH4dGxvLli1b/nS/5BN/MGqitLTUqce7C2+ZA7xjlj2Fe5iXMo8Ko4KpbaeSEJDAnl17zI5VY97wOzlJs9RMTZ9Hr7d78shZrFbYvh02JMGGDZCZiWGxUNa2DYy9BS66iNL69SkFso4dg2PHzE58Vjzyd3Ia3jLLuZrjvJ2kl5CQUOPHJicnO/V4d+Etc4Dnz/LL4V+YlTSL6OBoHmrxEAO6DjA7ktM8/XdSlWZxSEpKOqv71/R59Hq7J0+YxTAMylNTKVq9hqJffqFo3TqM4mIsgYGE9OpJ+D33kNGkMQkXXWR2VJfwhN9JdXnLLM7Ocbp11umCHBsbS0ZGRuXXmZmZxOqsU3Fjv6X/xr9++BfxkfG8MuAVjqYeNTuSiIhHMAwDa1oaxes3ULx2LUVr11JxogP4N21KneHDCOvXj9CePfEJDgYgwwv2Ukrt43RB7tSpE6mpqRw6dIjY2Fi+/PJLnn76aVdkE3G5jVkbGffDOJpGNGXhwIVEBkVyFBVkEZFTMcrLKd21i5JNmynZuJHipCQqMjMB8K1bl5CePQnt1ZPQiy4ioGlTk9OKuI7TBdnPz49p06Zx++23Y7PZuO6662jdurUrsom41Lbsbdz9/d3EhsSycJCjHIvUdgXffw9PPU1qVBQ+4WH4hobhExbmuB0ejk9YOL7hYfiEh+MTduJ74RGO74WFYfH1NXsEcRGjvJyyffso3ZFM6bZtlG7fTunOnRhlZQD4NWhASGIiwd0SCUnsRmDrVrocm3gtlxyD3K9fP/r16+eKTYmcE1uObuGu7+4iMjCShYMWUi+4ntmRRNyCb1Q0xMZgwYIt+xjlqanYC4uwFxRglJef8fE+oaH4hIc7inNEBL5hYY7P4eH4RITjGx5R+dk34kS5jvjffS3+umrM+WZUVGBNS6Ns337Kdu+mbM8eylJSKNu3z3GSHeATEkJQhw7UHTWK4M6dCe7SGf8qJ+SLeDu9k554vY1ZG7n7+7uJCorircFv0SBUi7zISSFdL4RJk2h2ipNc7OXl2AsKsBcUYCsoxF5YgK2gAPvJ2/kF2ArysRcUOj7nF2A9moV93z5s+fnYCwrAbv/L57eEhOAbHn6iNNf5X7EOO1mwwx0FPCLif59P7smOiMAnIOBcvTQezZafjzU9HevhI1jT0ihPO4T14CHKDxyg/NAhqKiovK9fXByBrVsR1rcvge3aEpSQQEDz5to7LLWaCrJ4tfUZ67l3xb3EhsTyxqA3iA3VCaQi1eUTEIBPdDRER9fo8YZhYC8qxp6f5yjYBfnY8gv+V67z87CfLNn5+dgKCqnIysK2Z8+JUl6Ngu3v7zj8IzwM37Bw8PHhUGyMo2CHhTn2cIeF4hMaim+o47NPaCg+ISGVH5aTnwMCsFgsNZr1XDOsVmyFhY7X6fhxKnJzseUex5ZzjIrsY1RkZ1Nx9CgVmZlUZGZiLy7+w+N9QkLwb9KEwFYtCR8wgIAWLQiMb0FAq1b4hoWZNJWI+1JBFq+18uBKJvw8gcZhjXlj8Bs6rELkPLNYLPiGheIbFkpNDqSoLNgF+Sf2XBc49kwXFjq+Plm2CwodPysqhMwsrAcOUlZU5CiUhYVnLNmVfHzwCQ7GEhyMT2AgluAgfAKDsAQG4hMUiMU/wFGiAwOx+Ptj8fPD4u8Hfn5YfP2w+PmCjy/4WLBYfKBq2TYMDLsNbHbH5wobRkWF46O8HKOsDMNajr2kFHtpCUZxCeTmklJhxV5YhFFScvrXOSgIv+ho/GJiCGzThtBLLsY/tgH+jRri37Ah/o0a4RsV5bblX8QdqSCLV1qyZwnT10ynfXR7Xr78ZZ2QJ+KB/lCw4+Kq9Zjk5GTiqxwuYhgGRmkp9sJC7EVF2IqKMIqLKz/bS0ocJbykBHtJMUZJCfbSMozSEkdZLSvFKCvHXlyCUZ6HvbwMo9yKYbViVFjBeqLk2u1QUYFhs4FhnLqU+/iAry8WiwWLnx+cLNkBAfgEnCjfIcH4BAXjW78+REURHhfn2ONd5Thu38hIfOvWdXyOisYnNETlV8TFVJDFqxiGweLti3k66Wl6xfXiucueI8Q/xOxYImISi8Xi2CMcHAz165/X5zYMw6nimpycTJwXvJGDiCdSQRavUWGvYN66eXy06yMGNRvE45c8ToCvTuAREXNor66I51JBFq9QbC1m4s8T+SntJ27tcCv3J96Pj0VnYIuIiMjZU0EWj5demM6/Vv6LXbm7mNJzCn9r9zezI4mIiIgHU0EWj5aUmcT4H8dTbivnhf4v0LdxX7MjiYiIiIfTv0GLRzIMg49TPub2b28nIiCC94e8r3IsIiIiLqE9yOJxSipKeOy3x/hi7xf0adiH+f3mExEQYXYsERER8RIqyOJRUvNSGf/TePbk7uHuzndz5wV34uvja3YsERER8SIqyOIxlu5dypy1c/D38eflAS9zcaOLzY4kIiIiXkgFWdxeYXkhc9bOYdm+ZXSN6cq8vvNoENrA7FgiIiLipVSQxa39nvk7j/7yKEeKjnBPl3u4o9MdOqRCREREzikVZHFLZbYyXtz4Iou3L6ZhWEMWXbGIC2MuNDuWiIiI1AIqyOJ2NmVtYvqa6ezL28cNbW7gwW4PEuIfYnYsERERqSVUkMVtFFmLeO735/ho50fEhsbyyoBXdCKeiIiInHcqyGI6wzD44eAPPLH+CTKLMrmp3U2M6zqOUP9Qs6OJiIhILaSCLKY6kH+Ax9c9zurDq2ldtzVP9n2SLjFdzI4lIiIitZgKspiisLyQhVsX8u6OdwnwDeDh7g9zY7sb8fPRf5IiIiJiLrUROa8q7BV8tuczXtz4IjmlOQxrOYz7u95P/ZD6ZkcTERERAVSQ5Tw5eZzx8xufZ1/ePrrGdOXly1+mQ70OZkcTERER+QMfZx68fPlyhgwZQrt27di6daurMokXMQyD39J/Y/Ty0dz/4/3YDTvPXvosi65YpHIsIiIibsmpPcht2rThhRdeYPr06a7KI17CMAzWZazj5U0v83vW78SExDCj9wyGtxqu44xFRETErTnVVFq2bOmqHOIlDMPg57SfeWPrG2w6uomY4Bgm95zMiNYjCPQNNDueiIiIyBlZDMMwnN3ImDFjmDhxIp06dTrlz5OSkggJqfk7oZWWlhIUFFTjx7sLb5kD/jxLhb2CNTlr+CL9Cw6WHKReQD2GxQ2jf/3+BPgEmJj0zLzl9+Itc4BmOam4uJjExMRq3deZdVavt3vyllm8ZQ7QLO7I2TlOt86ecQ/y2LFjyc7O/tP377//fgYMGFDtAAkJCdW+7/8vOTnZqce7C2+ZA/43S15ZHh+nfMyHyR+SVZJFyzotmZM4hytbXIm/j7/ZMavFW34v3jIHaJaTkpKSzur+NX0evd7uyVtm8ZY5QLO4I2fnON06e8aCvGjRoho/qXivfUX7+GD1Byzfv5wyWxm943ozs89M+jTsg8ViMTueiIiISI3pbCmptiJrEV/v/5pPdn/C1uytBPsFM6zlMG5sdyNt6rYxO56IiIiISzhVkL/77jtmz55NTk4Od955JwkJCbz55puuyiZuwDAMfs/6nS/2fsHX+7+muKKYVpGtGNt0LHf0uYPwgHCzI4qIiIi4lFMFeeDAgQwcONBVWcSN7M/bz/L9y1m6dylphWkE+wUzuPlgrmt9HZ3rd2bnzp0qxyIiIuKVdIiFVEorSOP7A9/z1f6vSM5JxoKFHnE9uKfLPVze9HJC/Gt+JRIRERERT6GCXIsZhsH+vP38cOgHvjvwHTuO7QCgfXR7JnSbwBUtriAmJMbklCIiIiLnlwpyLWO1W9mUtYmf035m5aGVHMg/AEDH6I6MTxzPgGYDaBLexOSUIiIiIuZRQa4F0gvTWXNkDauPrOa3I79RYC3Az8ePHg16MDphNJc2uZQGoQ3MjikiIiLiFlSQvdDx0uOsz1zP2vS1rE1fS2p+KgAxwTEMbD6Qvo360jOuJ2EBYeYGFREREXFDKsheIKs4i41ZG9mQsYGkrCR25+4GINgvmMTYREa2GUmfRn2IrxOvN/EQEREROQMVZA9TbitnV84utmRvYcvRLWzK2sSRoiOAoxB3qd+FwV0G0zOuJx3qdfCYt3sWERERcRcqyG7MarOyN28vyceS2X5sOzuO7WBnzk6sdivgOGSic0xnRrcfTZf6XWgX3U6FWERERMRJKshu4ljJMXYf301KTgopuSnsyt3FnuN7qLBXABDmH0ZCdAI3J9xMp3qduKD+BTqxTkREROQcUEE+j+yGnYyiDPbn7Wdf3j725+1n7/G97D2+l9yy3Mr71QuuR+vI1oxpP4aEqATaRbWjWUQzfCw+JqYXERERqR1UkF3MbtjJKs4irSCNQwWHOFhwkIP5BzmQf4DUvFTK1pdV3jfcP5z4yHj6N+1Pq8hWtIxsSZu6bYgOjjZxAhEREZHaTQX5LFntVo4WHyW9KJ30onQyijI4XHiYI4VHOFJ4hMOFhyuPEQbws/jRMKwhzSKa0TKgJV2bd6VFnRbER8YTHRStq0qIiIiIuBkV5BNsdhvHy46TXZJNdkk2R0uOcrT4KFnFWZUfmcWZZJdkY2D84bFRQVE0DG1I67qtuazpZTQOa0yjsEY0DW9KXFgcfj6Olzk5OZmEdglmjCciIiIi1eS1Bdlqt5JXlkdeWR65pbmOz2W55JbmklOaQ25ZLjklORwrPUZOaQ45pTnYDfufthMREEFMSAwxITG0rtuaBqENiAmJoWFoQxqENaBBSANC/ENMmFBEREREzgW3L8h5ZXmkFqVSlFFEobWQgvICCsoL/nA7vzzf8VGW7yjF5XkUWYtOu80QvxDqBtUlOjiahmEN6VSvE9HB0UQHRVMvuB71Q+pTL7ge9YLrEewXfB6nFRERERGzuXVBNgyDKz+5kgJrAWz/88+DfIMIDwgnPCC8ck9vq8hW1AmsU/kRGRhZ+VE3qC6RgZEE+QWd/2FERERExCO4dUG2WCy8cPkLbNuzjbbxbQnzDyM8ILzyc4BvgNkRRURERMTLuHVBBkiMTSQkJ4SEOJ3cJiIiIiLnnt55QkRERESkChVkEREREZEqVJBFRERERKpQQRYRERERqUIFWURERESkCothGMaZ7+acpKSkc/0UIiJeKTExsVr30zorIlIzp1pnz0tBFhERERHxFDrEQkRERESkChVkEREREZEqVJBFRERERKrwiIK8YMEChg4dyvDhw/nHP/5BZmam2ZFqbN68eVxxxRUMHTqUe++9l/z8fLMj1djy5csZMmQI7dq1Y+vWrWbHOWs///wzgwcPZuDAgbz++utmx6mxRx55hN69e3P11VebHcVp6enpjBkzhquuuoohQ4awePFisyPVWFlZGddffz3Dhg1jyJAhPP/882ZH+ktaZ92Pp6+xoHXW3WiNPQuGBygoKKi8vXjxYmPq1KkmpnHOqlWrDKvVahiGYcyfP9+YP3++yYlqbs+ePcbevXuN0aNHG1u2bDE7zlmpqKgwLr/8cuPgwYNGWVmZMXToUGP37t1mx6qRdevWGdu2bTOGDBlidhSnZWZmGtu2bTMMw/HnftCgQR77e7Hb7UZhYaFhGIZRXl5uXH/99cbGjRvNDfUXtM66H09eYw1D66w70hpbfR6xBzksLKzydklJCRaLxcQ0zrn44ovx8/MDoEuXLmRkZJicqOZatmxJfHy82TFqZMuWLTRr1owmTZoQEBDAkCFDWLFihdmxaqR79+7UqVPH7BguERMTQ4cOHQDHn/v4+HiP3ZNpsVgIDQ0FoKKigoqKCrdeu7TOuh9PXmNB66w70hpbfX4u29I59uyzz/L5558THh7OO++8Y3Ycl/jkk0+48sorzY5RK2VmZtKgQYPKr2NjY9myZYuJieT/l5aWRnJyMp07dzY7So3ZbDZGjBjBwYMHGTVqlNvPonVWXEnrrHvTGvvX3KYgjx07luzs7D99//7772fAgAE88MADPPDAA7z22mu89957jBs3zoSU1XOmWQBeeeUVfH19GTZs2PmOd1aqM4uIqxUVFTFu3DgmT578hz2bnsbX15clS5aQn5/PvffeS0pKCm3atDEtj9ZZ96M1VsygNfbM3KYgL1q0qFr3Gzp0KHfccYdbL9xnmuXTTz/lxx9/ZNGiRW7/z5jV/b14mtjY2D/8s2tmZiaxsbEmJpKTrFYr48aNY+jQoQwaNMjsOC4RERFBz549WbVqlakFWeus+/HWNRa0zrorrbHV4xHHIKemplbeXrFihUcfk/Xzzz/zxhtv8MorrxAcHGx2nFqrU6dOpKamcujQIcrLy/nyyy/p37+/2bFqPcMwePTRR4mPj+fWW281O45TcnJyKq+eUFpaypo1a9x67dI6K66mddb9aI2tPo94q+n77ruP/fv3Y7FYaNSoETNnzvTY/wsdOHAg5eXlREZGAtC5c2dmzZplbqga+u6775g9ezY5OTlERESQkJDAm2++aXasavvpp5+YO3cuNpuN6667jrvvvtvsSDUyfvx41q1bR25uLtHR0dx3332MHDnS7Fg1smHDBm6++WbatGmDj4/j/9/Hjx9Pv379TE529nbu3MmkSZOw2WwYhsEVV1zB//3f/5kd67S0zrofT19jQeusu9EaW30eUZBFRERERM4XjzjEQkRERETkfFFBFhERERGpQgVZRERERKQKFWQRERERkSpUkEVEREREqlBBFhERERGpQgVZRERERKSK/wdmqwq1F4jL/QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x432 with 4 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# YOUR CODE HERE: Change 'relu' to 'elu', 'selu', 'swish'... or something else\n",
    "activation_layer = layers.Activation('swish')\n",
    "\n",
    "x = tf.linspace(-3.0, 3.0, 100)\n",
    "y = activation_layer(x) # once created, a layer is callable just like a function\n",
    "\n",
    "fig, axes = plt.subplots(nrows=2, ncols=2, figsize=(10, 6), sharex=True, sharey=True)\n",
    "axes = axes.ravel()  # array to 1D\n",
    "activations = ['relu', 'elu', 'selu', 'swish']\n",
    "colors = ['tab:blue', 'tab:orange', 'tab:green', 'tab:red']\n",
    "\n",
    "for activation, color, ax in zip(activations, colors, axes):\n",
    "    ax.plot(x, layers.Activation(activation)(x), color)\n",
    "    ax.set_title(activation)\n",
    "    \n",
    "fig.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "543d9326",
   "metadata": {
    "papermill": {
     "duration": 0.01757,
     "end_time": "2022-01-29T23:32:38.998781",
     "exception": false,
     "start_time": "2022-01-29T23:32:38.981211",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Keep Going #\n",
    "\n",
    "Now move on to Lesson 3 and [**learn how to train neural networks**](https://www.kaggle.com/ryanholbrook/stochastic-gradient-descent) with stochastic gradient descent."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "94b02f05",
   "metadata": {
    "papermill": {
     "duration": 0.017257,
     "end_time": "2022-01-29T23:32:39.033733",
     "exception": false,
     "start_time": "2022-01-29T23:32:39.016476",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "---\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "*Have questions or comments? Visit the [course discussion forum](https://www.kaggle.com/learn/intro-to-deep-learning/discussion) to chat with other learners.*"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.12"
  },
  "papermill": {
   "default_parameters": {},
   "duration": 19.585988,
   "end_time": "2022-01-29T23:32:41.799807",
   "environment_variables": {},
   "exception": null,
   "input_path": "__notebook__.ipynb",
   "output_path": "__notebook__.ipynb",
   "parameters": {},
   "start_time": "2022-01-29T23:32:22.213819",
   "version": "2.3.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}