deep_learning/04-transfer-learning.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**[Deep Learning Course Home Page](https://www.kaggle.com/learn/deep-learning)**\n",
    "\n",
    "---\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercise Introduction\n",
    "\n",
    "The cameraman who shot our deep learning videos mentioned a problem that we can solve with deep learning.  \n",
    "\n",
    "He offers a service that scans photographs to store them digitally.  He uses a machine that quickly scans many photos. But depending on the orientation of the original photo, many images are digitized sideways.  He fixes these manually, looking at each photo to determine which ones to rotate.\n",
    "\n",
    "In this exercise, you will build a model that distinguishes which photos are sideways and which are upright, so an app could automatically rotate each image if necessary.\n",
    "\n",
    "If you were going to sell this service commercially, you might use a large dataset to train the model. But you'll have great success with even a small dataset.  You'll work with a small dataset of dog pictures, half of which are rotated sideways.\n",
    "\n",
    "Specifying and compiling the model look the same as in the example you've seen. But you'll need to make some changes to fit the model.\n",
    "\n",
    "**Run the following cell to set up automatic feedback.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Setup Complete\n"
     ]
    }
   ],
   "source": [
    "# Set up code checking\n",
    "from learntools.core import binder\n",
    "binder.bind(globals())\n",
    "from learntools.deep_learning.exercise_4 import *\n",
    "print(\"Setup Complete\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1) Specify the Model\n",
    "\n",
    "Since this is your first time, we'll provide some starter code for you to modify. You will probably copy and modify code the first few times you work on your own projects.\n",
    "\n",
    "There are some important parts left blank in the following code.\n",
    "\n",
    "Fill in the blanks (marked with `____`) and run the cell\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "tags": [
     "raises-exception"
    ]
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint8 = np.dtype([(\"qint8\", np.int8, 1)])\n",
      "/opt/conda/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_quint8 = np.dtype([(\"quint8\", np.uint8, 1)])\n",
      "/opt/conda/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint16 = np.dtype([(\"qint16\", np.int16, 1)])\n",
      "/opt/conda/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_quint16 = np.dtype([(\"quint16\", np.uint16, 1)])\n",
      "/opt/conda/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint32 = np.dtype([(\"qint32\", np.int32, 1)])\n",
      "/opt/conda/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  np_resource = np.dtype([(\"resource\", np.ubyte, 1)])\n",
      "/opt/conda/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint8 = np.dtype([(\"qint8\", np.int8, 1)])\n",
      "/opt/conda/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_quint8 = np.dtype([(\"quint8\", np.uint8, 1)])\n",
      "/opt/conda/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint16 = np.dtype([(\"qint16\", np.int16, 1)])\n",
      "/opt/conda/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_quint16 = np.dtype([(\"quint16\", np.uint16, 1)])\n",
      "/opt/conda/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint32 = np.dtype([(\"qint32\", np.int32, 1)])\n",
      "/opt/conda/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  np_resource = np.dtype([(\"resource\", np.ubyte, 1)])\n"
     ]
    },
    {
     "data": {
      "application/javascript": [
       "parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"outcomeType\": 1, \"valueTowardsCompletion\": 0.5, \"interactionType\": 1, \"questionType\": 2, \"learnTutorialId\": 76, \"questionId\": \"1_SetTraininableAndNumClasses\", \"learnToolsVersion\": \"0.3.1\", \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\"}}, \"*\")"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<span style=\"color:#33cc33\">Correct</span>"
      ],
      "text/plain": [
       "Correct"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from tensorflow.keras.applications import ResNet50\n",
    "from tensorflow.keras.models import Sequential\n",
    "from tensorflow.keras.layers import Dense, Flatten, GlobalAveragePooling2D\n",
    "\n",
    "num_classes = 2\n",
    "resnet_weights_path = '../input/resnet50/resnet50_weights_tf_dim_ordering_tf_kernels_notop.h5'\n",
    "\n",
    "my_new_model = Sequential()\n",
    "my_new_model.add(ResNet50(include_top=False, pooling='avg', weights=resnet_weights_path))\n",
    "my_new_model.add(Dense(num_classes, activation='softmax'))\n",
    "\n",
    "# Indicate whether the first layer should be trained/changed or not.\n",
    "my_new_model.layers[0].trainable = False\n",
    "\n",
    "step_1.check()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# step_1.hint()\n",
    "# step_1.solution()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2) Compile the Model\n",
    "\n",
    "You now compile the model with the following line.  Run this cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_new_model.compile(optimizer='sgd', \n",
    "                     loss='categorical_crossentropy', \n",
    "                     metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That ran nearly instantaneously.  Deep learning models have a reputation for being computationally demanding.  Why did that run so quickly?\n",
    "\n",
    "After thinking about this, check your answer by uncommenting the cell below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"interactionType\": 3, \"questionType\": 4, \"learnTutorialId\": 76, \"questionId\": \"2_CompileTransferLearningModel\", \"learnToolsVersion\": \"0.3.1\", \"valueTowardsCompletion\": 0.0, \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\", \"outcomeType\": 4}}, \"*\")"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<span style=\"color:#33cc99\">Solution:</span> \n",
       "The compile model doesn't change the values in any convolutions.  In fact, your model has not even\n",
       "received an argument with data yet.  Compile specifies how your model will make updates a later\n",
       "`fit` step where it receives data.  That is the part that will take longer.\n"
      ],
      "text/plain": [
       "Solution: \n",
       "The compile model doesn't change the values in any convolutions.  In fact, your model has not even\n",
       "received an argument with data yet.  Compile specifies how your model will make updates a later\n",
       "`fit` step where it receives data.  That is the part that will take longer."
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "step_2.solution()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3) Review the Compile Step\n",
    "You provided three arguments in the compile step.  \n",
    "- optimizer\n",
    "- loss\n",
    "- metrics\n",
    "\n",
    "Which arguments could affect the accuracy of the predictions that come out of the model?  After you have your answer, run the cell below to see the solution."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"interactionType\": 3, \"questionType\": 4, \"learnTutorialId\": 76, \"questionId\": \"3_WhatCompileArgsAffectModel\", \"learnToolsVersion\": \"0.3.1\", \"valueTowardsCompletion\": 0.0, \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\", \"outcomeType\": 4}}, \"*\")"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<span style=\"color:#33cc99\">Solution:</span> \n",
       "- **optimizer** determines how we determine the numerical values that make up the model. So it can affect the resulting model and predictions\n",
       "- **loss** determines what goal we optimize when determining numerical values in the model. So it can affect the resulting model and predictions\n",
       "- **metrics** determines only what we print out while the model is being built, but it doesn't affect the model itself.\n",
       "\n",
       "You may not understand all of this yet. That's totally fine for now.  It will become\n",
       "clearer in an upcoming lesson (called A Deeper Understanding of Deep Learning).\n"
      ],
      "text/plain": [
       "Solution: \n",
       "- **optimizer** determines how we determine the numerical values that make up the model. So it can affect the resulting model and predictions\n",
       "- **loss** determines what goal we optimize when determining numerical values in the model. So it can affect the resulting model and predictions\n",
       "- **metrics** determines only what we print out while the model is being built, but it doesn't affect the model itself.\n",
       "\n",
       "You may not understand all of this yet. That's totally fine for now.  It will become\n",
       "clearer in an upcoming lesson (called A Deeper Understanding of Deep Learning)."
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "step_3.solution()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4) Fit Model\n",
    "\n",
    "**Your training data is in the directory `../input/dogs-gone-sideways/images/train`. The validation data is in `../input/dogs-gone-sideways/images/val`**. Use that information when setting up `train_generator` and `validation_generator`.\n",
    "\n",
    "You have 220 images of training data and 217 of validation data.  For the training generator, we set a batch size of 10. Figure out the appropriate value of `steps_per_epoch` in your `fit_generator` call.\n",
    "\n",
    "Fill in all the blanks (again marked as `____`).  Then run the cell of code.  Watch as your model trains the weights and the accuracy improves."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "tags": [
     "raises-exception"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found 220 images belonging to 2 classes.\n",
      "Found 217 images belonging to 2 classes.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/lib/python3.6/site-packages/keras_preprocessing/image/image_data_generator.py:716: UserWarning: This ImageDataGenerator specifies `featurewise_center`, but it hasn't been fit on any training data. Fit it first by calling `.fit(numpy_data)`.\n",
      "  warnings.warn('This ImageDataGenerator specifies '\n",
      "/opt/conda/lib/python3.6/site-packages/keras_preprocessing/image/image_data_generator.py:716: UserWarning: This ImageDataGenerator specifies `featurewise_center`, but it hasn't been fit on any training data. Fit it first by calling `.fit(numpy_data)`.\n",
      "  warnings.warn('This ImageDataGenerator specifies '\n",
      "/opt/conda/lib/python3.6/site-packages/keras_preprocessing/image/image_data_generator.py:716: UserWarning: This ImageDataGenerator specifies `featurewise_center`, but it hasn't been fit on any training data. Fit it first by calling `.fit(numpy_data)`.\n",
      "  warnings.warn('This ImageDataGenerator specifies '\n",
      "/opt/conda/lib/python3.6/site-packages/keras_preprocessing/image/image_data_generator.py:716: UserWarning: This ImageDataGenerator specifies `featurewise_center`, but it hasn't been fit on any training data. Fit it first by calling `.fit(numpy_data)`.\n",
      "  warnings.warn('This ImageDataGenerator specifies '\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "22/22 [==============================] - 8s 368ms/step - loss: 0.5156 - acc: 0.7500 - val_loss: 0.2013 - val_acc: 0.9062\n"
     ]
    },
    {
     "data": {
      "application/javascript": [
       "parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"outcomeType\": 1, \"valueTowardsCompletion\": 0.5, \"interactionType\": 1, \"questionType\": 2, \"learnTutorialId\": 76, \"questionId\": \"4_FitTransferModel\", \"learnToolsVersion\": \"0.3.1\", \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\"}}, \"*\")"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<span style=\"color:#33cc33\">Correct</span>"
      ],
      "text/plain": [
       "Correct"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from tensorflow.keras.applications.resnet50 import preprocess_input\n",
    "from tensorflow.keras.preprocessing.image import ImageDataGenerator\n",
    "\n",
    "image_size = 224\n",
    "data_generator = ImageDataGenerator(preprocess_input)\n",
    "\n",
    "train_generator = data_generator.flow_from_directory(\n",
    "                                        directory='../input/dogs-gone-sideways/images/train',\n",
    "                                        target_size=(image_size, image_size),\n",
    "                                        batch_size=10,\n",
    "                                        class_mode='categorical')\n",
    "\n",
    "validation_generator = data_generator.flow_from_directory(\n",
    "                                        directory='../input/dogs-gone-sideways/images/val',\n",
    "                                        target_size=(image_size, image_size),\n",
    "                                        class_mode='categorical')\n",
    "\n",
    "# fit_stats below saves some statistics describing how model fitting went\n",
    "# the key role of the following line is how it changes my_new_model by fitting to data\n",
    "fit_stats = my_new_model.fit_generator(train_generator,\n",
    "                                       steps_per_epoch=22,\n",
    "                                       validation_data=validation_generator,\n",
    "                                       validation_steps=1)\n",
    "\n",
    "step_4.check()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# step_4.solution()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Can you tell from the results what fraction of the time your model was correct in the validation data? \n",
    "\n",
    "In the next step, we'll see if we can improve on that.\n",
    "\n",
    "# Keep Going\n",
    "Move on to learn about **[data augmentation](https://www.kaggle.com/dansbecker/data-augmentation/)**.  It is a clever and easy way to improve your models. Then you'll apply data augmentation to this automatic image rotation problem.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "**[Deep Learning Course Home Page](https://www.kaggle.com/learn/deep-learning)**\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}