diff --git a/module1-rnn-and-lstm/LS_DS_431_RNN_and_LSTM_Assignment.ipynb b/module1-rnn-and-lstm/LS_DS_431_RNN_and_LSTM_Assignment.ipynb
index a49e3827..7b6a402d 100644
--- a/module1-rnn-and-lstm/LS_DS_431_RNN_and_LSTM_Assignment.ipynb
+++ b/module1-rnn-and-lstm/LS_DS_431_RNN_and_LSTM_Assignment.ipynb
@@ -1,263 +1,1015 @@
 {
-  "cells": [
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "<img align=\"left\" src=\"https://lever-client-logos.s3.amazonaws.com/864372b1-534c-480e-acd5-9711f850815c-1524247202159.png\" width=200>\n",
+    "<br></br>\n",
+    "<br></br>\n",
+    "\n",
+    "## *Data Science Unit 4 Sprint 3 Assignment 1*\n",
+    "\n",
+    "# Recurrent Neural Networks and Long Short Term Memory (LSTM)\n",
+    "\n",
+    "![Monkey at a typewriter](https://upload.wikimedia.org/wikipedia/commons/thumb/3/3c/Chimpanzee_seated_at_typewriter.jpg/603px-Chimpanzee_seated_at_typewriter.jpg)\n",
+    "\n",
+    "It is said that [infinite monkeys typing for an infinite amount of time](https://en.wikipedia.org/wiki/Infinite_monkey_theorem) will eventually type, among other things, the complete works of Wiliam Shakespeare. Let's see if we can get there a bit faster, with the power of Recurrent Neural Networks and LSTM.\n",
+    "\n",
+    "This text file contains the complete works of Shakespeare: https://www.gutenberg.org/files/100/100-0.txt\n",
+    "\n",
+    "Use it as training data for an RNN - you can keep it simple and train character level, and that is suggested as an initial approach.\n",
+    "\n",
+    "Then, use that trained RNN to generate Shakespearean-ish text. Your goal - a function that can take, as an argument, the size of text (e.g. number of characters or lines) to generate, and returns generated text of that size.\n",
+    "\n",
+    "Note - Shakespeare wrote an awful lot. It's OK, especially initially, to sample/use smaller data and parameters, so you can have a tighter feedback loop when you're trying to get things running. Then, once you've got a proof of concept - start pushing it more!"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 85,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import requests\n",
+    "import pandas as pd"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 126,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "url = \"https://www.gutenberg.org/files/100/100-0.txt\"\n",
+    "\n",
+    "r = requests.get(url)\n",
+    "r.encoding = r.apparent_encoding\n",
+    "data = r.text\n",
+    "data = data.split('\\r\\n')\n",
+    "toc = [l.strip() for l in data[44:130:2]]\n",
+    "# Skip the Table of Contents\n",
+    "data = data[135:]\n",
+    "\n",
+    "# Fixing Titles\n",
+    "toc[9] = 'THE LIFE OF KING HENRY V'\n",
+    "toc[18] = 'MACBETH'\n",
+    "toc[24] = 'OTHELLO, THE MOOR OF VENICE'\n",
+    "toc[34] = 'TWELFTH NIGHT: OR, WHAT YOU WILL'\n",
+    "\n",
+    "locations = {id_:{'title':title, 'start':-99} for id_,title in enumerate(toc)}\n",
+    "\n",
+    "# Start \n",
+    "for e,i in enumerate(data):\n",
+    "    for t,title in enumerate(toc):\n",
+    "        if title in i:\n",
+    "            locations[t].update({'start':e})\n",
+    "            \n",
+    "\n",
+    "df_toc = pd.DataFrame.from_dict(locations, orient='index')\n",
+    "df_toc['end'] = df_toc['start'].shift(-1).apply(lambda x: x-1)\n",
+    "df_toc.loc[42, 'end'] = len(data)\n",
+    "df_toc['end'] = df_toc['end'].astype('int')\n",
+    "\n",
+    "df_toc['text'] = df_toc.apply(lambda x: '\\r\\n'.join(data[ x['start'] : int(x['end']) ]), axis=1)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 127,
+   "metadata": {
+    "nteract": {
+     "transient": {
+      "deleting": false
+     }
+    }
+   },
+   "outputs": [
     {
-      "cell_type": "markdown",
-      "source": [
-        "<img align=\"left\" src=\"https://lever-client-logos.s3.amazonaws.com/864372b1-534c-480e-acd5-9711f850815c-1524247202159.png\" width=200>\n",
-        "<br></br>\n",
-        "<br></br>\n",
-        "\n",
-        "## *Data Science Unit 4 Sprint 3 Assignment 1*\n",
-        "\n",
-        "# Recurrent Neural Networks and Long Short Term Memory (LSTM)\n",
-        "\n",
-        "![Monkey at a typewriter](https://upload.wikimedia.org/wikipedia/commons/thumb/3/3c/Chimpanzee_seated_at_typewriter.jpg/603px-Chimpanzee_seated_at_typewriter.jpg)\n",
-        "\n",
-        "It is said that [infinite monkeys typing for an infinite amount of time](https://en.wikipedia.org/wiki/Infinite_monkey_theorem) will eventually type, among other things, the complete works of Wiliam Shakespeare. Let's see if we can get there a bit faster, with the power of Recurrent Neural Networks and LSTM.\n",
-        "\n",
-        "This text file contains the complete works of Shakespeare: https://www.gutenberg.org/files/100/100-0.txt\n",
-        "\n",
-        "Use it as training data for an RNN - you can keep it simple and train character level, and that is suggested as an initial approach.\n",
-        "\n",
-        "Then, use that trained RNN to generate Shakespearean-ish text. Your goal - a function that can take, as an argument, the size of text (e.g. number of characters or lines) to generate, and returns generated text of that size.\n",
-        "\n",
-        "Note - Shakespeare wrote an awful lot. It's OK, especially initially, to sample/use smaller data and parameters, so you can have a tighter feedback loop when you're trying to get things running. Then, once you've got a proof of concept - start pushing it more!"
+     "data": {
+      "text/html": [
+       "<div>\n",
+       "<style scoped>\n",
+       "    .dataframe tbody tr th:only-of-type {\n",
+       "        vertical-align: middle;\n",
+       "    }\n",
+       "\n",
+       "    .dataframe tbody tr th {\n",
+       "        vertical-align: top;\n",
+       "    }\n",
+       "\n",
+       "    .dataframe thead th {\n",
+       "        text-align: right;\n",
+       "    }\n",
+       "</style>\n",
+       "<table border=\"1\" class=\"dataframe\">\n",
+       "  <thead>\n",
+       "    <tr style=\"text-align: right;\">\n",
+       "      <th></th>\n",
+       "      <th>title</th>\n",
+       "      <th>start</th>\n",
+       "      <th>end</th>\n",
+       "      <th>text</th>\n",
+       "    </tr>\n",
+       "  </thead>\n",
+       "  <tbody>\n",
+       "    <tr>\n",
+       "      <th>0</th>\n",
+       "      <td>THE TRAGEDY OF ANTONY AND CLEOPATRA</td>\n",
+       "      <td>-99</td>\n",
+       "      <td>14379</td>\n",
+       "      <td></td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>1</th>\n",
+       "      <td>AS YOU LIKE IT</td>\n",
+       "      <td>14380</td>\n",
+       "      <td>17171</td>\n",
+       "      <td>AS YOU LIKE IT\\r\\n\\r\\n\\r\\nDRAMATIS PERSONAE.\\r...</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>2</th>\n",
+       "      <td>THE COMEDY OF ERRORS</td>\n",
+       "      <td>17172</td>\n",
+       "      <td>20372</td>\n",
+       "      <td>THE COMEDY OF ERRORS\\r\\n\\r\\n\\r\\n\\r\\nContents\\r...</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>3</th>\n",
+       "      <td>THE TRAGEDY OF CORIOLANUS</td>\n",
+       "      <td>20373</td>\n",
+       "      <td>30346</td>\n",
+       "      <td>THE TRAGEDY OF CORIOLANUS\\r\\n\\r\\nDramatis Pers...</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>4</th>\n",
+       "      <td>CYMBELINE</td>\n",
+       "      <td>30347</td>\n",
+       "      <td>30364</td>\n",
+       "      <td>CYMBELINE.\\r\\nLaud we the gods;\\r\\nAnd let our...</td>\n",
+       "    </tr>\n",
+       "  </tbody>\n",
+       "</table>\n",
+       "</div>"
       ],
-      "metadata": {}
-    },
+      "text/plain": [
+       "                                 title  start    end  \\\n",
+       "0  THE TRAGEDY OF ANTONY AND CLEOPATRA    -99  14379   \n",
+       "1                       AS YOU LIKE IT  14380  17171   \n",
+       "2                 THE COMEDY OF ERRORS  17172  20372   \n",
+       "3            THE TRAGEDY OF CORIOLANUS  20373  30346   \n",
+       "4                            CYMBELINE  30347  30364   \n",
+       "\n",
+       "                                                text  \n",
+       "0                                                     \n",
+       "1  AS YOU LIKE IT\\r\\n\\r\\n\\r\\nDRAMATIS PERSONAE.\\r...  \n",
+       "2  THE COMEDY OF ERRORS\\r\\n\\r\\n\\r\\n\\r\\nContents\\r...  \n",
+       "3  THE TRAGEDY OF CORIOLANUS\\r\\n\\r\\nDramatis Pers...  \n",
+       "4  CYMBELINE.\\r\\nLaud we the gods;\\r\\nAnd let our...  "
+      ]
+     },
+     "execution_count": 127,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "#Shakespeare Data Parsed by Play\n",
+    "df_toc.head()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 128,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from tensorflow.keras.callbacks import LambdaCallback\n",
+    "from tensorflow.keras.models import Sequential\n",
+    "from tensorflow.keras.layers import Dense, LSTM\n",
+    "from tensorflow.keras.optimizers import RMSprop\n",
+    "\n",
+    "import numpy as np\n",
+    "import random\n",
+    "import sys\n",
+    "import os"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 129,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import re\n",
+    "\n",
+    "df_toc['text'] = df_toc['text'].str.replace(r'[\\n\\r\\t]', ' ')\n",
+    "df_toc['text'] = df_toc['text'].str.replace('  ', ' ')\n",
+    "df_toc= df_toc.drop(df_toc.index[0])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 132,
+   "metadata": {},
+   "outputs": [
     {
-      "cell_type": "code",
-      "source": [
-        "import requests\n",
-        "import pandas as pd"
-      ],
-      "outputs": [],
-      "execution_count": 1,
-      "metadata": {
-        "execution": {
-          "iopub.status.busy": "2020-06-15T18:18:20.442Z",
-          "iopub.execute_input": "2020-06-15T18:18:20.453Z",
-          "iopub.status.idle": "2020-06-15T18:18:20.513Z",
-          "shell.execute_reply": "2020-06-15T18:18:20.523Z"
-        }
-      }
-    },
+     "data": {
+      "text/plain": [
+       "1    AS YOU LIKE IT DRAMATIS PERSONAE. DUKE, living...\n",
+       "2    THE COMEDY OF ERRORS Contents ACT I Scene I. A...\n",
+       "3    THE TRAGEDY OF CORIOLANUS Dramatis Personae CA...\n",
+       "4    CYMBELINE. Laud we the gods; And let our crook...\n",
+       "5    THE TRAGEDY OF HAMLET, PRINCE OF DENMARK Conte...\n",
+       "Name: text, dtype: object"
+      ]
+     },
+     "execution_count": 132,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "df_toc['text'] = df_toc['text'].str.replace('  ', ' ')\n",
+    "df_toc['text'].head()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 133,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "small_data = df_toc['text'][:3].values\n",
+    "data = df_toc['text'].values"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 134,
+   "metadata": {},
+   "outputs": [
     {
-      "cell_type": "code",
-      "source": [
-        "url = \"https://www.gutenberg.org/files/100/100-0.txt\"\n",
-        "\n",
-        "r = requests.get(url)\n",
-        "r.encoding = r.apparent_encoding\n",
-        "data = r.text\n",
-        "data = data.split('\\r\\n')\n",
-        "toc = [l.strip() for l in data[44:130:2]]\n",
-        "# Skip the Table of Contents\n",
-        "data = data[135:]\n",
-        "\n",
-        "# Fixing Titles\n",
-        "toc[9] = 'THE LIFE OF KING HENRY V'\n",
-        "toc[18] = 'MACBETH'\n",
-        "toc[24] = 'OTHELLO, THE MOOR OF VENICE'\n",
-        "toc[34] = 'TWELFTH NIGHT: OR, WHAT YOU WILL'\n",
-        "\n",
-        "locations = {id_:{'title':title, 'start':-99} for id_,title in enumerate(toc)}\n",
-        "\n",
-        "# Start \n",
-        "for e,i in enumerate(data):\n",
-        "    for t,title in enumerate(toc):\n",
-        "        if title in i:\n",
-        "            locations[t].update({'start':e})\n",
-        "            \n",
-        "\n",
-        "df_toc = pd.DataFrame.from_dict(locations, orient='index')\n",
-        "df_toc['end'] = df_toc['start'].shift(-1).apply(lambda x: x-1)\n",
-        "df_toc.loc[42, 'end'] = len(data)\n",
-        "df_toc['end'] = df_toc['end'].astype('int')\n",
-        "\n",
-        "df_toc['text'] = df_toc.apply(lambda x: '\\r\\n'.join(data[ x['start'] : int(x['end']) ]), axis=1)"
-      ],
-      "outputs": [],
-      "execution_count": 23,
-      "metadata": {
-        "execution": {
-          "iopub.status.busy": "2020-06-15T18:25:49.778Z",
-          "iopub.execute_input": "2020-06-15T18:25:49.781Z",
-          "iopub.status.idle": "2020-06-15T18:25:51.467Z",
-          "shell.execute_reply": "2020-06-15T18:25:51.469Z"
-        }
-      }
-    },
+     "data": {
+      "text/plain": [
+       "3"
+      ]
+     },
+     "execution_count": 134,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "len(small_data)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 135,
+   "metadata": {},
+   "outputs": [
     {
-      "cell_type": "code",
-      "source": [
-        "#Shakespeare Data Parsed by Play\n",
-        "df_toc.head()"
-      ],
-      "outputs": [
-        {
-          "output_type": "execute_result",
-          "execution_count": 24,
-          "data": {
-            "text/plain": [
-              "                                 title  start    end  \\\n",
-              "0            ALL’S WELL THAT ENDS WELL   2777   7738   \n",
-              "1  THE TRAGEDY OF ANTONY AND CLEOPATRA   7739  11840   \n",
-              "2                       AS YOU LIKE IT  11841  14631   \n",
-              "3                 THE COMEDY OF ERRORS  14632  17832   \n",
-              "4            THE TRAGEDY OF CORIOLANUS  17833  27806   \n",
-              "\n",
-              "                                                text  \n",
-              "0  ALL’S WELL THAT ENDS WELL\\r\\n\\r\\n\\r\\n\\r\\nConte...  \n",
-              "1  THE TRAGEDY OF ANTONY AND CLEOPATRA\\r\\n\\r\\nDRA...  \n",
-              "2  AS YOU LIKE IT\\r\\n\\r\\nDRAMATIS PERSONAE.\\r\\n\\r...  \n",
-              "3  THE COMEDY OF ERRORS\\r\\n\\r\\n\\r\\n\\r\\nContents\\r...  \n",
-              "4  THE TRAGEDY OF CORIOLANUS\\r\\n\\r\\nDramatis Pers...  "
-            ],
-            "text/html": [
-              "<div>\n",
-              "<style scoped>\n",
-              "    .dataframe tbody tr th:only-of-type {\n",
-              "        vertical-align: middle;\n",
-              "    }\n",
-              "\n",
-              "    .dataframe tbody tr th {\n",
-              "        vertical-align: top;\n",
-              "    }\n",
-              "\n",
-              "    .dataframe thead th {\n",
-              "        text-align: right;\n",
-              "    }\n",
-              "</style>\n",
-              "<table border=\"1\" class=\"dataframe\">\n",
-              "  <thead>\n",
-              "    <tr style=\"text-align: right;\">\n",
-              "      <th></th>\n",
-              "      <th>title</th>\n",
-              "      <th>start</th>\n",
-              "      <th>end</th>\n",
-              "      <th>text</th>\n",
-              "    </tr>\n",
-              "  </thead>\n",
-              "  <tbody>\n",
-              "    <tr>\n",
-              "      <th>0</th>\n",
-              "      <td>ALL’S WELL THAT ENDS WELL</td>\n",
-              "      <td>2777</td>\n",
-              "      <td>7738</td>\n",
-              "      <td>ALL’S WELL THAT ENDS WELL\\r\\n\\r\\n\\r\\n\\r\\nConte...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>1</th>\n",
-              "      <td>THE TRAGEDY OF ANTONY AND CLEOPATRA</td>\n",
-              "      <td>7739</td>\n",
-              "      <td>11840</td>\n",
-              "      <td>THE TRAGEDY OF ANTONY AND CLEOPATRA\\r\\n\\r\\nDRA...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>2</th>\n",
-              "      <td>AS YOU LIKE IT</td>\n",
-              "      <td>11841</td>\n",
-              "      <td>14631</td>\n",
-              "      <td>AS YOU LIKE IT\\r\\n\\r\\nDRAMATIS PERSONAE.\\r\\n\\r...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>3</th>\n",
-              "      <td>THE COMEDY OF ERRORS</td>\n",
-              "      <td>14632</td>\n",
-              "      <td>17832</td>\n",
-              "      <td>THE COMEDY OF ERRORS\\r\\n\\r\\n\\r\\n\\r\\nContents\\r...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>4</th>\n",
-              "      <td>THE TRAGEDY OF CORIOLANUS</td>\n",
-              "      <td>17833</td>\n",
-              "      <td>27806</td>\n",
-              "      <td>THE TRAGEDY OF CORIOLANUS\\r\\n\\r\\nDramatis Pers...</td>\n",
-              "    </tr>\n",
-              "  </tbody>\n",
-              "</table>\n",
-              "</div>"
-            ]
-          },
-          "metadata": {}
-        }
-      ],
-      "execution_count": 24,
-      "metadata": {
-        "collapsed": true,
-        "jupyter": {
-          "source_hidden": false,
-          "outputs_hidden": false
-        },
-        "nteract": {
-          "transient": {
-            "deleting": false
-          }
-        },
-        "execution": {
-          "iopub.status.busy": "2020-06-15T18:26:12.630Z",
-          "iopub.execute_input": "2020-06-15T18:26:12.637Z",
-          "iopub.status.idle": "2020-06-15T18:26:12.643Z",
-          "shell.execute_reply": "2020-06-15T18:26:12.647Z"
-        }
-      }
-    },
+     "data": {
+      "text/plain": [
+       "42"
+      ]
+     },
+     "execution_count": 135,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "len(data)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 136,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Encoding\n",
+    "\n",
+    "#Gathering All Text\n",
+    "text = \" \".join(small_data)\n",
+    "\n",
+    "#Character count\n",
+    "chars = list(set(text))\n",
+    "\n",
+    "# Lookup tables - mapping chars to nums for numeric representation \n",
+    "char_int = {c:i for i, c in enumerate(chars)} \n",
+    "int_char = {i:c for i, c in enumerate(chars)} "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 137,
+   "metadata": {},
+   "outputs": [
     {
-      "cell_type": "markdown",
-      "source": [
-        "# Resources and Stretch Goals"
-      ],
-      "metadata": {
-        "colab_type": "text",
-        "id": "zE4a4O7Bp5x1"
-      }
+     "data": {
+      "text/plain": [
+       "76"
+      ]
+     },
+     "execution_count": 137,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "len(chars)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 138,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "sequences:  106178\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Create the sequence data\n",
+    "maxlen = 40\n",
+    "step = 5\n",
+    "\n",
+    "encoded = [char_int[c] for c in text]\n",
+    "\n",
+    "sequences = [] # Each element is 40 chars long\n",
+    "next_char = [] # One element for each sequence\n",
+    "\n",
+    "for i in range(0, len(encoded) - maxlen, step):\n",
+    "    sequences.append(encoded[i : i + maxlen])\n",
+    "    next_char.append(encoded[i + maxlen])\n",
+    "    \n",
+    "print('sequences: ', len(sequences))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 139,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Create x & y\n",
+    "x = np.zeros((len(sequences), maxlen, len(chars)), dtype=np.bool)\n",
+    "y = np.zeros((len(sequences),len(chars)), dtype=np.bool)\n",
+    "\n",
+    "for i, sequence in enumerate(sequences):\n",
+    "    for t, char in enumerate(sequence):\n",
+    "        x[i,t,char] = 1\n",
+    "    y[i, next_char[i]] = 1"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 140,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "(106178, 40, 76)"
+      ]
+     },
+     "execution_count": 140,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "x.shape"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 68,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "(115565, 76)"
+      ]
+     },
+     "execution_count": 68,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "y.shape"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 69,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# build the model: a single LSTM\n",
+    "model = Sequential()\n",
+    "model.add(LSTM(128, input_shape=(maxlen, len(chars))))\n",
+    "model.add(Dense(len(chars), activation='softmax'))\n",
+    "\n",
+    "model.compile(loss='categorical_crossentropy', optimizer='adam')"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 70,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def sample(preds):\n",
+    "    # helper function to sample an index from a probability array\n",
+    "    preds = np.asarray(preds).astype('float64')\n",
+    "    preds = np.log(preds) / 1\n",
+    "    exp_preds = np.exp(preds)\n",
+    "    preds = exp_preds / np.sum(exp_preds)\n",
+    "    probas = np.random.multinomial(1, preds, 1)\n",
+    "    return np.argmax(probas)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 71,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def on_epoch_end(epoch, _):\n",
+    "    # Function invoked at end of each epoch. Prints generated text.\n",
+    "    print()\n",
+    "    print('----- Generating text after Epoch: %d' % epoch)\n",
+    "\n",
+    "    start_index = random.randint(0, len(text) - maxlen - 1)\n",
+    "    \n",
+    "    generated = ''\n",
+    "    \n",
+    "    sentence = text[start_index: start_index + maxlen]\n",
+    "    generated += sentence\n",
+    "    \n",
+    "    print('----- Generating with seed: \"' + sentence + '\"')\n",
+    "    sys.stdout.write(generated)\n",
+    "    \n",
+    "    for i in range(400):\n",
+    "        x_pred = np.zeros((1, maxlen, len(chars)))\n",
+    "        for t, char in enumerate(sentence):\n",
+    "            x_pred[0, t, char_int[char]] = 1\n",
+    "            \n",
+    "        preds = model.predict(x_pred, verbose=0)[0]\n",
+    "        next_index = sample(preds)\n",
+    "        next_char = int_char[next_index]\n",
+    "        \n",
+    "        sentence = sentence[1:] + next_char\n",
+    "        \n",
+    "        sys.stdout.write(next_char)\n",
+    "        sys.stdout.flush()\n",
+    "    print()\n",
+    "\n",
+    "\n",
+    "print_callback = LambdaCallback(on_epoch_end=on_epoch_end)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 72,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Train on 115565 samples\n",
+      "Epoch 1/20\n",
+      "115552/115565 [============================>.] - ETA: 0s - loss: 2.4669\n",
+      "----- Generating text after Epoch: 0\n",
+      "----- Generating with seed: \"mio, command him away.    DROMIO OF EPHE\"\n",
+      "mio, command him away.    DROMIO OF EPHEUS. Mirovat     be.   MCLOON. With     De tblln.    Da’s suce wittue an fibllts.    Whorsen souk.  I CIMAIUS. ORANS. PMALEM. I sace mat is smass !    CECENI. Neler, the      poule lave therre ta douf mirmis.    HeRLDHN. Pawllound grwan cote anest ther                     No to deey udverst  To Ipae?    MEamr qurun;      Co     CESEN.NPAINN SGARIMN. SOosr'la’os yin rfan ould an       Woro feed     \n",
+      "115565/115565 [==============================] - 63s 546us/sample - loss: 2.4669\n",
+      "Epoch 2/20\n",
+      "115488/115565 [============================>.] - ETA: 0s - loss: 2.0416- ETA\n",
+      "----- Generating text after Epoch: 1\n",
+      "----- Generating with seed: \"men,      The common file- a plague! tri\"\n",
+      "men,      The common file- a plague! tringen and,  The d'des serad thiks.    Ardyor.   HCHEA.  Jow loke      Onatu’d, pithee.    “OEFINUS. [ICist, thbuld’d thit and the soab’d; not I  Srads hou wim    With syou the coun ' wo putind ans and corie; tho  Sreisie, mry beantfer all, he thithimanotr till.                F. AWL'k AF S  Thaw me’tfurd om,      Shuk with him wart 'gratirtur’d youn' be,            But As EFteRIANA.  Aro shiuld    \n",
+      "115565/115565 [==============================] - 60s 517us/sample - loss: 2.0415\n",
+      "Epoch 3/20\n",
+      "115552/115565 [============================>.] - ETA: 0s - loss: 1.8973\n",
+      "----- Generating text after Epoch: 2\n",
+      "----- Generating with seed: \"him when he was less furnish’d than now \"\n",
+      "him when he was less furnish’d than now cons,, Withy that my sezer  Des; sores dofish then, and in      ind; I ho, go's, on wother werd ther an yes gaotiy. I whane hill sthee maXil.                     II, sup is now con u'd!    SLUIDIO. I you ceOnots aman; jeety, I thoy swore prain hirghode.  Het ingelatbtw-yo; heouge chack finglly, witt forens fall vithe!    F IALIT.  FReSACVUS. I, bust ame Epour sjees an GEROMONO. Gay, will pranty wi\n",
+      "115565/115565 [==============================] - 60s 517us/sample - loss: 1.8973\n",
+      "Epoch 4/20\n",
+      "115488/115565 [============================>.] - ETA: 0s - loss: 1.7967\n",
+      "----- Generating text after Epoch: 3\n",
+      "----- Generating with seed: \" I will please you what you will demand.\"\n",
+      " I will please you what you will demand.    CELIRIUS.  You dillislong friwing to the upher! whith duTice  And gint, I lave      Whockith soer; ame Ro is day mither at I his fanderoby rufies,     Leat tgosp'd. Nod hourh, songh you so ly weed gell muy knot Mim,      The pupitend ane speck be in the redsos us I would love, bether boll you      To tostilaque that the bepyous. But ming, hor bost misnde and her Pris:  The pors.    LTICELUS. I\n",
+      "115565/115565 [==============================] - 58s 498us/sample - loss: 1.7964\n",
+      "Epoch 5/20\n",
+      "115456/115565 [============================>.] - ETA: 0s - loss: 1.7206\n",
+      "----- Generating text after Epoch: 4\n",
+      "----- Generating with seed: \" I am glad of your departure; adieu, goo\"\n",
+      " I am glad of your departure; adieu, good him,  Bith.    ANTIPHOLUS in tes'r me; 'it the ragh. coming shauke to that footh,      Yough Surizers, shelingagulion. I you.  That in now my come; meor hy some.    BRUTHS LWhie._]    DROMINS. Angir supe? We bus them ith, berish, and whath a with hour.  Faet then goot now you avore deatord.      What, shather tril's compest be lotm:  To you anows'd and not but woom, chursel the pratus,      Mome\n",
+      "115565/115565 [==============================] - 59s 513us/sample - loss: 1.7207\n",
+      "Epoch 6/20\n",
+      "115488/115565 [============================>.] - ETA: 0s - loss: 1.6634\n",
+      "----- Generating text after Epoch: 5\n",
+      "----- Generating with seed: \"NA.  Come, go. I will fall prostrate at \"\n",
+      "NA.  Come, go. I will fall prostrate at this be the simbsess stroth  Them are go diet bet of sibungy this of corne,      Ad will Duppius? I ho scigh, an were not even-slivind’s one, for.    DOUFELEN. Cupan, so not lead and would of lick. Farch and yot Paplay, Romancaius.                                        ExeAnt.                                      [_Exit Cite.  ExeEN] fitle man!      Thun, not veresed in oppain my cave' grean:  Th\n",
+      "115565/115565 [==============================] - 58s 501us/sample - loss: 1.6633\n",
+      "Epoch 7/20\n",
+      "115552/115565 [============================>.] - ETA: 0s - loss: 1.6174\n",
+      "----- Generating text after Epoch: 6\n",
+      "----- Generating with seed: \"No, take more.      What may be sworn by\"\n",
+      "No, take more.      What may be sworn by converouls hear on his it bely      Come os pring of clouk can I moureds if the ear thou swow with, go mode.    MENENIUS. The owry's you bely death tuserrably farten,  She has you but cortly? shord withint. They sack.    LUCIOS.  He so wome sir,     Shat ye. Lemy with thou know have seatt infthous      Thou dove up, is nere hooks shall that dead nom      That hour crieting you, morent; amp, bot  \n",
+      "115565/115565 [==============================] - 59s 507us/sample - loss: 1.6174\n",
+      "Epoch 8/20\n",
+      "115488/115565 [============================>.] - ETA: 0s - loss: 1.5781\n",
+      "----- Generating text after Epoch: 7\n",
+      "----- Generating with seed: \"the points o' th' compass.    SECOND CIT\"\n",
+      "the points o' th' compass.    SECOND CITIZEN. Ky, so to th' thouth will mis,  Fotter hos life, lide mer to fixhor theiss to preas      as have quight for thys. If with I as I amteet have not      sather framy wath mell be macr-litther.    FIRST OMORESt. And o' ratain ob gold,  A'tiniss]    ANTICLONE. I parius rey not of peely have wouchour you gencher  As not dill ms in hos there.  Thou shall with ows that egend they I this houl;      I\n",
+      "115565/115565 [==============================] - 58s 498us/sample - loss: 1.5780\n",
+      "Epoch 9/20\n",
+      "115488/115565 [============================>.] - ETA: 0s - loss: 1.5453\n",
+      "----- Generating text after Epoch: 8\n",
+      "----- Generating with seed: \" has eyes in’s head, then; I have not se\"\n",
+      " has eyes in’s head, then; I have not send  Distnent un ether Juponess, I consel, of llade to the disbates stild hard. He word home cinchay;      ofound then shem homeiler out Romakus      Mortide; the bean twomen, this wife hak, more, and troes,      And up nome. you will ney and all,  And wish'd nersic, nothing not but mep. Re-poor you an my stred      Mastwergad, furly-sanies than wash. Duppaslancue, go drack’d? CaLlain.    ANTALIO, \n",
+      "115565/115565 [==============================] - 59s 514us/sample - loss: 1.5450\n",
+      "Epoch 10/20\n",
+      "115488/115565 [============================>.] - ETA: 0s - loss: 1.5144\n",
+      "----- Generating text after Epoch: 9\n",
+      "----- Generating with seed: \"LINE.  Nobly doom’d!  We’ll learn our fr\"\n",
+      "LINE.  Nobly doom’d!  We’ll learn our frofless protter, and prode timans.    ROSALIMO.  Them is aid no miservounils.      Whuch my most two fust mor to cazelfite.           But tr’reetoness to these but ments not more it.    DROMIO OF EPHESUS.  Stnterit, shoke their asso's live of,    Tot to greatiented grotues ingreaches,      And to kindon they have and and tries,      You are thing in are aponged, wherest in.    MENELIRA.  Thou stele\n",
+      "115565/115565 [==============================] - 57s 495us/sample - loss: 1.5142\n",
+      "Epoch 11/20\n",
+      "115520/115565 [============================>.] - ETA: 0s - loss: 1.4874\n",
+      "----- Generating text after Epoch: 10\n",
+      "----- Generating with seed: \", mine own self’s better part,  Mine eye\"\n",
+      ", mine own self’s better part,  Mine eyeince, houtwer again.    This good fow you dore notule may row the cations's,      Trare. They your will plays. Youn wither, and such you; If that with of mess parsiont, sir f am the sworl. Lesure.    VOSSENGER. The L? BE SERVIUS, MCLANE, aplicar,  And lost me vouring and with Coriallar on Miscom,  Youre's like freechimus is that I am trucks      Unstrescrain was your touch.    MENENIUS. Whuld, do?\n",
+      "115565/115565 [==============================] - 58s 505us/sample - loss: 1.4875\n",
+      "Epoch 12/20\n",
+      "115488/115565 [============================>.] - ETA: 0s - loss: 1.4631\n",
+      "----- Generating text after Epoch: 11\n",
+      "----- Generating with seed: \"(’Tis in my cloak-bag) doublet, hat, hos\"\n",
+      "(’Tis in my cloak-bag) doublet, hat, hos. at di'll.    VIRSMONGE. Txis three natuse braich.    BELARIUS.  With yourde, Cartilis    An time true comion,     Agare wife wroke him not heragh'd that I that;      Rome. 'To Fele?    MENENIUS. Beso thil’ge forow on the Romans by odeeds of can to Bubled her it our like  (to live herbking and the word O fastain it fions.    OLIVER. What thou they houss, sirs the thid; for the choatl; yet      Ca\n",
+      "115565/115565 [==============================] - 58s 500us/sample - loss: 1.4631\n",
+      "Epoch 13/20\n",
+      "115488/115565 [============================>.] - ETA: 0s - loss: 1.4405\n",
+      "----- Generating text after Epoch: 12\n",
+      "----- Generating with seed: \"et us shame him with our knees.      To \"\n",
+      "et us shame him with our knees.      To herift sing, in my flays’d from Pesceediling my mowde,      To thou fair be bettees'd O, I could dove for with that alak ack'd      whocharding he a disfin that swomed aboud      make aghied gone, witnseal; but be hush I deill powand!  You gore, covine we shall, and by a made breads,      And petaied scount that poor of in      Were ming's gale, hos readonlen. If chough  Whoss is so now k’llow’st \n",
+      "115565/115565 [==============================] - 59s 510us/sample - loss: 1.4406\n",
+      "Epoch 14/20\n",
+      "115488/115565 [============================>.] - ETA: 0s - loss: 1.4183\n",
+      "----- Generating text after Epoch: 13\n",
+      "----- Generating with seed: \"ll not ask again. Close villain,  I’ll h\"\n",
+      "ll not ask again. Close villain,  I’ll have greng of his corsantale a mut a chhip.    AnDICES.  But you me told:                      Enter the Roman cucholy      Mithers, and, the wird you hearanne of      apmoraver-so me to’s froolesba decomme  ’lounds on my fastancas!    BELARIUS.  That bory whither; live to soil mornels      Ade first. We Roolas’d and sun''y prays—    VORUMIO OF SYRACUSE.  Good had hore. If hast aloness sincl elters\n",
+      "115565/115565 [==============================] - 58s 503us/sample - loss: 1.4183\n",
+      "Epoch 15/20\n",
+      "115552/115565 [============================>.] - ETA: 0s - loss: 1.3993\n",
+      "----- Generating text after Epoch: 14\n",
+      "----- Generating with seed: \"it must bear? Well, I will do't;      Ye\"\n",
+      "it must bear? Well, I will do't;      Yet with to th' looks, mesul'd is no bought      Drom uger'd that bots so for drough here      th' entend, and debprid us I was suntry.      Allopon,  For have nere benout op all would in good swear  s my well.      The finlys oad. Ye hourd atfording of than love forles      but thenss, prifoand to this mame’s mother, wound you remites; and lord             JAQUES. Which As sevich you have depones a\n",
+      "115565/115565 [==============================] - 59s 512us/sample - loss: 1.3993\n",
+      "Epoch 16/20\n",
+      "115456/115565 [============================>.] - ETA: 0s - loss: 1.3794\n",
+      "----- Generating text after Epoch: 15\n",
+      "----- Generating with seed: \"es honest she makes very      ill-favour\"\n",
+      "es honest she makes very      ill-favoure pesilains that the stough  But thee.    ANTIPHOLUS OF SYRACUSE.  Thes hast at her but the exemes righted  Wishapes be mo to those foot bearues siQ.  Fors sweet with upher's hooles betitime now,  Lue of their Lay no more noble be buin  FIredied of home. The shall tempoment.    LUCIUS.  To Hagl us yee?    SECOND POTHI. How run'd bine  Your daate no bere is but he worlibed as as      is awt me, sor\n",
+      "115565/115565 [==============================] - 60s 521us/sample - loss: 1.3793\n",
+      "Epoch 17/20\n",
+      "115552/115565 [============================>.] - ETA: 0s - loss: 1.3609\n",
+      "----- Generating text after Epoch: 16\n",
+      "----- Generating with seed: \" me after; you should not have mock'd me\"\n",
+      " me after; you should not have mock'd menter,      Inly be flound forshom; thus a rafknerour? In hull sme not that not her with to \" she      Censy fool-peels’st than the  The da'ch intiblest spord of everyly      With all pable, quit, any hows a baty,      In the godseaknondly are forture know that      Marcimug, holl, and in his pray you art not would you hip,  Do man ablestowr, the wornta her souk and ond.        Exit a BTENE.  Come,\n",
+      "115565/115565 [==============================] - 59s 507us/sample - loss: 1.3609\n",
+      "Epoch 18/20\n",
+      "115488/115565 [============================>.] - ETA: 0s - loss: 1.3429\n",
+      "----- Generating text after Epoch: 17\n",
+      "----- Generating with seed: \"art?'        Did you ever hear such rail\"\n",
+      "art?'        Did you ever hear such railite sty el’s      Till I iethate of it oole, one?    ROSALIND. What knop you live fadierty!     Enter down in the all Percayel—at of intoous,      to shadst of a safred.    ANTIPHOLUS OF EPHESUS.  No, so, have you are nopits Godd her;  Dutss; out fith doth in I could born, can in is suider of      And up not comes in the blood roth) and      for ferler to un litter hort; and our himsence,  (To in \n",
+      "115565/115565 [==============================] - 60s 517us/sample - loss: 1.3431\n",
+      "Epoch 19/20\n",
+      "115456/115565 [============================>.] - ETA: 0s - loss: 1.3262\n",
+      "----- Generating text after Epoch: 18\n",
+      "----- Generating with seed: \"u art a gallant youth;      I would thou\"\n",
+      "u art a gallant youth;      I would thougut now be more sterrith      by have dapperiuge and litt, which thle carn  (T BELAD. I am sooked Marcius.    IMOSSENG. To for what have my lie deep is no much eate henser-  Why beong but hurbor nerur on thy writwers;  Which in the the art, exchim is I think in Selent    ORLANDO. You thank I mornom the spate, and your lovistarners.      Save our no to the seture of my true,  Simply, thou lost was \n",
+      "115565/115565 [==============================] - 59s 512us/sample - loss: 1.3264\n",
+      "Epoch 20/20\n",
+      "115552/115565 [============================>.] - ETA: 0s - loss: 1.3100\n",
+      "----- Generating text after Epoch: 19\n",
+      "----- Generating with seed: \"he haver. If it be,      The man I speak\"\n",
+      "he haver. If it be,      The man I speak good 'tis coure.  I to ie confein'd honk a coult; they gentryonds  The cattingonats was I livish come do friends      The trubt tookings him eve us tle potience.      Get us th' colarailts 'Umsh'd they gain’d get your fate, sie,  FIrltice humblectias vary dindation      The should die day he beat then seer hinking roult be so.    CEOTEN.  I war that your kill,  And the Rome any Postion of noble p\n",
+      "115565/115565 [==============================] - 61s 526us/sample - loss: 1.3101\n"
+     ]
     },
     {
-      "cell_type": "markdown",
-      "source": [
-        "## Stretch goals:\n",
-        "- Refine the training and generation of text to be able to ask for different genres/styles of Shakespearean text (e.g. plays versus sonnets)\n",
-        "- Train a classification model that takes text and returns which work of Shakespeare it is most likely to be from\n",
-        "- Make it more performant! Many possible routes here - lean on Keras, optimize the code, and/or use more resources (AWS, etc.)\n",
-        "- Revisit the news example from class, and improve it - use categories or tags to refine the model/generation, or train a news classifier\n",
-        "- Run on bigger, better data\n",
-        "\n",
-        "## Resources:\n",
-        "- [The Unreasonable Effectiveness of Recurrent Neural Networks](https://karpathy.github.io/2015/05/21/rnn-effectiveness/) - a seminal writeup demonstrating a simple but effective character-level NLP RNN\n",
-        "- [Simple NumPy implementation of RNN](https://github.com/JY-Yoon/RNN-Implementation-using-NumPy/blob/master/RNN%20Implementation%20using%20NumPy.ipynb) - Python 3 version of the code from \"Unreasonable Effectiveness\"\n",
-        "- [TensorFlow RNN Tutorial](https://github.com/tensorflow/models/tree/master/tutorials/rnn) - code for training a RNN on the Penn Tree Bank language dataset\n",
-        "- [4 part tutorial on RNN](http://www.wildml.com/2015/09/recurrent-neural-networks-tutorial-part-1-introduction-to-rnns/) - relates RNN to the vanishing gradient problem, and provides example implementation\n",
-        "- [RNN training tips and tricks](https://github.com/karpathy/char-rnn#tips-and-tricks) - some rules of thumb for parameterizing and training your RNN"
-      ],
-      "metadata": {
-        "colab_type": "text",
-        "id": "uT3UV3gap9H6"
-      }
+     "data": {
+      "text/plain": [
+       "<tensorflow.python.keras.callbacks.History at 0x16dca1a4c50>"
+      ]
+     },
+     "execution_count": 72,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# fit the model\n",
+    "\n",
+    "model.fit(x, y,\n",
+    "          batch_size=32,\n",
+    "          epochs=20,\n",
+    "          callbacks=[print_callback])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 111,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "(42,)"
+      ]
+     },
+     "execution_count": 111,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "## okay, small data done, time for the big boy\n",
+    "\n",
+    "data.shape"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 141,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Encoding\n",
+    "\n",
+    "#Gathering All Text\n",
+    "text = \" \".join(data)\n",
+    "\n",
+    "#Character count\n",
+    "chars = list(set(text))\n",
+    "\n",
+    "# Lookup tables - mapping chars to nums for numeric representation \n",
+    "char_int = {c:i for i, c in enumerate(chars)} \n",
+    "int_char = {i:c for i, c in enumerate(chars)} "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 142,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "103"
+      ]
+     },
+     "execution_count": 142,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "len(chars)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 143,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "sequences:  2808472\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Create the sequence data\n",
+    "maxlen = 40\n",
+    "step = 5\n",
+    "\n",
+    "encoded = [char_int[c] for c in text]\n",
+    "\n",
+    "sequences = [] # Each element is 40 chars long\n",
+    "next_char = [] # One element for each sequence\n",
+    "\n",
+    "for i in range(0, len(encoded) - maxlen, step):\n",
+    "    sequences.append(encoded[i : i + maxlen])\n",
+    "    next_char.append(encoded[i + maxlen])\n",
+    "    \n",
+    "print('sequences: ', len(sequences))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 144,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Create x & y\n",
+    "x = np.zeros((len(sequences), maxlen, len(chars)), dtype=np.bool)\n",
+    "y = np.zeros((len(sequences),len(chars)), dtype=np.bool)\n",
+    "\n",
+    "for i, sequence in enumerate(sequences):\n",
+    "    for t, char in enumerate(sequence):\n",
+    "        x[i,t,char] = 1\n",
+    "    y[i, next_char[i]] = 1"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 145,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "(2808472, 40, 103)"
+      ]
+     },
+     "execution_count": 145,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "x.shape"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 146,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "(2808472, 103)"
+      ]
+     },
+     "execution_count": 146,
+     "metadata": {},
+     "output_type": "execute_result"
     }
-  ],
-  "metadata": {
-    "kernelspec": {
-      "display_name": "U4-S3-MNA-DS11",
-      "language": "python",
-      "name": "u4-s3-mna-ds11"
+   ],
+   "source": [
+    "y.shape"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 147,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# build the model: a single LSTM\n",
+    "model = Sequential()\n",
+    "model.add(LSTM(128, input_shape=(maxlen, len(chars))))\n",
+    "model.add(Dense(len(chars), activation='softmax'))\n",
+    "\n",
+    "model.compile(loss='categorical_crossentropy', optimizer='adam')"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 148,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def sample(preds):\n",
+    "    # helper function to sample an index from a probability array\n",
+    "    preds = np.asarray(preds).astype('float64')\n",
+    "    preds = np.log(preds) / 1\n",
+    "    exp_preds = np.exp(preds)\n",
+    "    preds = exp_preds / np.sum(exp_preds)\n",
+    "    probas = np.random.multinomial(1, preds, 1)\n",
+    "    return np.argmax(probas)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 149,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def on_epoch_end(epoch, _):\n",
+    "    # Function invoked at end of each epoch. Prints generated text.\n",
+    "    print()\n",
+    "    print('----- Generating text after Epoch: %d' % epoch)\n",
+    "\n",
+    "    start_index = random.randint(0, len(text) - maxlen - 1)\n",
+    "    \n",
+    "    generated = ''\n",
+    "    \n",
+    "    sentence = text[start_index: start_index + maxlen]\n",
+    "    generated += sentence\n",
+    "    \n",
+    "    print('----- Generating with seed: \"' + sentence + '\"')\n",
+    "    sys.stdout.write(generated)\n",
+    "    \n",
+    "    for i in range(400):\n",
+    "        x_pred = np.zeros((1, maxlen, len(chars)))\n",
+    "        for t, char in enumerate(sentence):\n",
+    "            x_pred[0, t, char_int[char]] = 1\n",
+    "            \n",
+    "        preds = model.predict(x_pred, verbose=0)[0]\n",
+    "        next_index = sample(preds)\n",
+    "        next_char = int_char[next_index]\n",
+    "        \n",
+    "        sentence = sentence[1:] + next_char\n",
+    "        \n",
+    "        sys.stdout.write(next_char)\n",
+    "        sys.stdout.flush()\n",
+    "    print()\n",
+    "\n",
+    "\n",
+    "print_callback = LambdaCallback(on_epoch_end=on_epoch_end)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 151,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Train on 2808472 samples\n",
+      "Epoch 1/10\n",
+      "2808448/2808472 [============================>.] - ETA: 0s - loss: 1.7560\n",
+      "----- Generating text after Epoch: 0\n",
+      "----- Generating with seed: \"es doom men to death.    [Rises] But whe\"\n",
+      "es doom men to death.    [Rises] But when with prarishdes a crax of this; Where sweet sit precilliillace, whoming fones frignd. [A jottus? The cures fire Thon I guen, strangeres friends. I shall are the parences or give the sidddy sands But white. Your say, I know with thee suns report of enemanteds. Somedero. SECOND GEWSTEM. He came natus, a faith? DUCTOR. By myself Speak our love Pattoudia, SERVANTIA. S Attenousus: To end the’s like s\n",
+      "2808472/2808472 [==============================] - 1211s 431us/sample - loss: 1.7560\n",
+      "Epoch 2/10\n",
+      "2808384/2808472 [============================>.] - ETA: 0s - loss: 1.5877\n",
+      "----- Generating text after Epoch: 1\n",
+      "----- Generating with seed: \"oo little for your thief, your true man \"\n",
+      "oo little for your thief, your true man wisely Addess? No, sir, hearts natury three, wouldst same, Or good Our indocess’ His mind me with hears childreth a gonder, To see such and known this part with a may your house of moneming. VALENTINE. Neilwits,” eops drow the wind a Gide. MACBETH. Since with a turn in triaming Helen? When I'll eyean andee this where to the most Enter [TECRSCELES. Hath stand full with here mavish! His expression l\n",
+      "2808472/2808472 [==============================] - 1209s 431us/sample - loss: 1.5877\n",
+      "Epoch 3/10\n",
+      "2808448/2808472 [============================>.] - ETA: 0s - loss: 1.5354\n",
+      "----- Generating text after Epoch: 2\n",
+      "----- Generating with seed: \"LO. Hold, for your lives! IAGO. Hold, ho\"\n",
+      "LO. Hold, for your lives! IAGO. Hold, holen, To be satience are come your pawring to my enar'd Hand their no masters of the time a read That it claim Caloudes a freen in foot a meant and Roma foods. What shall pit’t a serrneral wine? Enter or brought, I younghy canste, I had if these fonded to remenct trust eyes, out , heremon on a fool.' Who may, and his ere a Eases forceld, To be my footes is will honour effect to of be worth. [_Exeun\n",
+      "2808472/2808472 [==============================] - 1318s 469us/sample - loss: 1.5354\n",
+      "Epoch 4/10\n",
+      "2808384/2808472 [============================>.] - ETA: 0s - loss: 1.5060\n",
+      "----- Generating text after Epoch: 3\n",
+      "----- Generating with seed: \"l our pageants of delight were play'd, O\"\n",
+      "l our pageants of delight were play'd, O bit in time as a place, Vectape of the ploud Achillencefit bats England; I have and underate to their isso? This demaid? buriodin. My Priare Lady Sifizal with darders.] Hectors the blos’—’ child, the Kench and favour of eyes For can hath not ye day to have service.OTEM. I have in my heart fell be demands Sir, And fellow she’s faires,. KING ELIMERDE. It withing scarce, it ishoming madam. A will we\n",
+      "2808472/2808472 [==============================] - 1245s 443us/sample - loss: 1.5060\n",
+      "Epoch 5/10\n",
+      "2808352/2808472 [============================>.] - ETA: 0s - loss: 1.4867\n",
+      "----- Generating text after Epoch: 4\n",
+      "----- Generating with seed: \"so tyrannous: O my soft harted Sister, w\"\n",
+      "so tyrannous: O my soft harted Sister, with my realimalf; The guilty, my poidin, chara"
+     ]
     },
-    "language_info": {
-      "name": "python",
-      "version": "3.7.0",
-      "mimetype": "text/x-python",
-      "codemirror_mode": {
-        "name": "ipython",
-        "version": 3
-      },
-      "pygments_lexer": "ipython3",
-      "nbconvert_exporter": "python",
-      "file_extension": ".py"
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "C:\\Users\\timro\\anaconda3\\envs\\U4-S2-Neural-Networks\\lib\\site-packages\\ipykernel_launcher.py:4: RuntimeWarning: divide by zero encountered in log\n",
+      "  after removing the cwd from sys.path.\n"
+     ]
     },
-    "nteract": {
-      "version": "0.23.3"
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "ity, to her field yet change, Unto husband thy time againer you shidps daskety. COSSIOM. If you, heirlic, he sin; So embrave and man, is not me upon, advised! To me. Nay, under our nailot in heren? CATESBY. No wills wintectly eyes wife. Well, there wast new haste The Sounder that thou like thee They need—that devil gone! Repess great themselves’s othal\n",
+      "2808472/2808472 [==============================] - 1233s 439us/sample - loss: 1.4867\n",
+      "Epoch 6/10\n",
+      "2808352/2808472 [============================>.] - ETA: 0s - loss: 1.4724\n",
+      "----- Generating text after Epoch: 5\n",
+      "----- Generating with seed: \"ll. To-morrow, sir, I wrestle for my cre\"\n",
+      "ll. To-morrow, sir, I wrestle for my creech of died to deign That I would meet evoling behall’d but Agan? SToken at very sweet? I’ll now suint, Alack stood the kingdom But being cure Ratcin as so's child. A mark love it, what an one of and eye, Look my lives, and this lives me secuse, There's a mark honest more than talk, and curtain find emolleg? Hang itsy is, if I speak to said thee it, Who prais’s myself. EMILIA. Why think you, ghite\n",
+      "2808472/2808472 [==============================] - 1233s 439us/sample - loss: 1.4724\n",
+      "Epoch 7/10\n",
+      "2808384/2808472 [============================>.] - ETA: 0s - loss: 1.4609\n",
+      "----- Generating text after Epoch: 6\n",
+      "----- Generating with seed: \"ng, boy? JULIA. Here 'tis; this is it. P\"\n",
+      "ng, boy? JULIA. Here 'tis; this is it. POLIXENES. What, I’ll call a gold and natchest, As he Duke! Esca. “What is the tongress bright joy, and turn thou dost? A rirt nay. KING EDWARD. With a proud with the body shall thy threedfeens!  EMILIA. If the arms and ride bulled air. The retire! I'll be supper, ye after persone! To Ne itters him call of the wind. You another winds, I’ll calame inglighness. He not, a citwixt still to the stirr'd \n",
+      "2808472/2808472 [==============================] - 1388s 494us/sample - loss: 1.4609\n",
+      "Epoch 8/10\n",
+      "2808384/2808472 [============================>.] - ETA: 0s - loss: 1.4517\n",
+      "----- Generating text after Epoch: 7\n",
+      "----- Generating with seed: \"To Lucius] Speak, Rome's dear friend, as\"\n",
+      "To Lucius] Speak, Rome's dear friend, as they nothing-housed bed approvisedy in-gife To who shouts the our familis! The great his gequor'd, ho's dead; I think the else into secret of infonste. Pruisc tast; in the voise in Voucin got the most of wome? COSTARD. O he in it your are the Call indander? What is till she putslen in his brains, By my mountaul days out, For the happinst, Yeatianes of to sees. COUNTESLUS. This chargely dare with \n",
+      "2808472/2808472 [==============================] - 1274s 454us/sample - loss: 1.4517\n",
+      "Epoch 9/10\n",
+      "2808416/2808472 [============================>.] - ETA: 0s - loss: 1.4447\n",
+      "----- Generating text after Epoch: 8\n",
+      "----- Generating with seed: \"hamefully bereft of life. COMMONS. [With\"\n",
+      "hamefully bereft of life. COMMONS. [Within Silordes’ stenge as a lady whoues to earth yet. Let remied sometitule lovim'd 'em, The praise will bring'd itto't this letter. It from the brees of warth I can not welcome; And that’s noble condemn’d we like nequicts, Drown hand be all rade I have other young. Fit’s, Having this men, thy overth nete I have doning curse. BUCKINGHAM. Their drown hath delay some Cun all this merrocldic deed it her\n",
+      "2808472/2808472 [==============================] - 1280s 456us/sample - loss: 1.4447\n",
+      "Epoch 10/10\n",
+      "2808448/2808472 [============================>.] - ETA: 0s - loss: 1.4385\n",
+      "----- Generating text after Epoch: 9\n",
+      "----- Generating with seed: \"the world but I, and I am sunburnt. I ma\"\n",
+      "the world but I, and I am sunburnt. I may nourly trishbar. cover him the true, be command worse death. THESEUS. With the watent the old lord? BOLTIMITUS. [_Swert, Enterm As, that sleep'd? POSTHUS.  Enter SIMPLE YORK, a spitions frey A last gone. Being to her too of prehechbart Sir Armanting in autlow’y, weak to woo? MONTANO. You carrye to with the more, as plutting untimen? LEONATO. Take the admotifutrour, move jety, in so base! Bhy, sw\n",
+      "2808472/2808472 [==============================] - 1308s 466us/sample - loss: 1.4385\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "<tensorflow.python.keras.callbacks.History at 0x16de6def3c8>"
+      ]
+     },
+     "execution_count": 151,
+     "metadata": {},
+     "output_type": "execute_result"
     }
+   ],
+   "source": [
+    "# fit the model\n",
+    "\n",
+    "model.fit(x, y,\n",
+    "          batch_size=32,\n",
+    "          epochs=10,\n",
+    "          callbacks=[print_callback])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "colab_type": "text",
+    "id": "zE4a4O7Bp5x1"
+   },
+   "source": [
+    "# Resources and Stretch Goals"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "colab_type": "text",
+    "id": "uT3UV3gap9H6"
+   },
+   "source": [
+    "## Stretch goals:\n",
+    "- Refine the training and generation of text to be able to ask for different genres/styles of Shakespearean text (e.g. plays versus sonnets)\n",
+    "- Train a classification model that takes text and returns which work of Shakespeare it is most likely to be from\n",
+    "- Make it more performant! Many possible routes here - lean on Keras, optimize the code, and/or use more resources (AWS, etc.)\n",
+    "- Revisit the news example from class, and improve it - use categories or tags to refine the model/generation, or train a news classifier\n",
+    "- Run on bigger, better data\n",
+    "\n",
+    "## Resources:\n",
+    "- [The Unreasonable Effectiveness of Recurrent Neural Networks](https://karpathy.github.io/2015/05/21/rnn-effectiveness/) - a seminal writeup demonstrating a simple but effective character-level NLP RNN\n",
+    "- [Simple NumPy implementation of RNN](https://github.com/JY-Yoon/RNN-Implementation-using-NumPy/blob/master/RNN%20Implementation%20using%20NumPy.ipynb) - Python 3 version of the code from \"Unreasonable Effectiveness\"\n",
+    "- [TensorFlow RNN Tutorial](https://github.com/tensorflow/models/tree/master/tutorials/rnn) - code for training a RNN on the Penn Tree Bank language dataset\n",
+    "- [4 part tutorial on RNN](http://www.wildml.com/2015/09/recurrent-neural-networks-tutorial-part-1-introduction-to-rnns/) - relates RNN to the vanishing gradient problem, and provides example implementation\n",
+    "- [RNN training tips and tricks](https://github.com/karpathy/char-rnn#tips-and-tricks) - some rules of thumb for parameterizing and training your RNN"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "U4-S2-Neural-Networks (Python3)",
+   "language": "python",
+   "name": "u4-s2-neural-networks"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.0"
   },
-  "nbformat": 4,
-  "nbformat_minor": 4
+  "nteract": {
+   "version": "0.23.3"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
 }