yolov3-keras-image-object-detection.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# YOLOv3 Keras API Image Object Detection\n",
    "\n",
    "In this notebook I want to implement simple object detection with Keras of some JPEG images in our training set. I use a pretrained (MSCOCO dataset) YOLOv3 model.\n",
    "\n",
    "This kernel is mainly based on https://machinelearningmastery.com/how-to-perform-object-detection-with-yolov3-in-keras and adapted for Lyft 3D Object Detection for Autonomous Driving challenge. I highly advise you to read to throught the blog for deeper understanding, and giving the original blogger the props.\n",
    "\n",
    "Please note that the model hasn't been fine tuned, or trained myself. The following implementation should serve the purpose of facilitating to get started with the current challenge.\n",
    "I don't know how relevant object detection with images is, in a 3D object detection challenge, as the LiDAR data is not regarded.\n",
    "\n",
    "If this Kernel helped you in any way, give it an upvote (such that I can increasy my online self-esteem...)$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['drone-dataset-with-steering-angle-2', '3d-object-detection-for-autonomous-vehicles', 'lyft-3d-recognition']\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "print(os.listdir('/kaggle/input'))\n",
    "DATA_PATH = \"/kaggle/input/drone-dataset-with-steering-angle-2/center\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## YOLOv3 NET\n",
    "Don't get freaked out by the following lines of code, I definitely didn't write them myself. They are used to create the YOLOv3 Neural Network in Keras.\n",
    "\n",
    "based on: https://machinelearningmastery.com/how-to-perform-object-detection-with-yolov3-in-keras/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n",
      "/opt/conda/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint8 = np.dtype([(\"qint8\", np.int8, 1)])\n",
      "/opt/conda/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_quint8 = np.dtype([(\"quint8\", np.uint8, 1)])\n",
      "/opt/conda/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint16 = np.dtype([(\"qint16\", np.int16, 1)])\n",
      "/opt/conda/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_quint16 = np.dtype([(\"quint16\", np.uint16, 1)])\n",
      "/opt/conda/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint32 = np.dtype([(\"qint32\", np.int32, 1)])\n",
      "/opt/conda/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  np_resource = np.dtype([(\"resource\", np.ubyte, 1)])\n",
      "/opt/conda/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint8 = np.dtype([(\"qint8\", np.int8, 1)])\n",
      "/opt/conda/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_quint8 = np.dtype([(\"quint8\", np.uint8, 1)])\n",
      "/opt/conda/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint16 = np.dtype([(\"qint16\", np.int16, 1)])\n",
      "/opt/conda/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_quint16 = np.dtype([(\"quint16\", np.uint16, 1)])\n",
      "/opt/conda/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint32 = np.dtype([(\"qint32\", np.int32, 1)])\n",
      "/opt/conda/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  np_resource = np.dtype([(\"resource\", np.ubyte, 1)])\n"
     ]
    }
   ],
   "source": [
    "# create a YOLOv3 Keras model and save it to file\n",
    "# based on https://github.com/experiencor/keras-yolo3\n",
    "import struct\n",
    "import numpy as np\n",
    "from keras.layers import Conv2D\n",
    "from keras.layers import Input\n",
    "from keras.layers import BatchNormalization\n",
    "from keras.layers import LeakyReLU\n",
    "from keras.layers import ZeroPadding2D\n",
    "from keras.layers import UpSampling2D\n",
    "from keras.layers.merge import add, concatenate\n",
    "from keras.models import Model\n",
    " \n",
    "def _conv_block(inp, convs, skip=True):\n",
    "    x = inp\n",
    "    count = 0\n",
    "    for conv in convs:\n",
    "        if count == (len(convs) - 2) and skip:\n",
    "            skip_connection = x\n",
    "        count += 1\n",
    "        if conv['stride'] > 1: x = ZeroPadding2D(((1,0),(1,0)))(x) # peculiar padding as darknet prefer left and top\n",
    "        x = Conv2D(conv['filter'],\n",
    "                   conv['kernel'],\n",
    "                   strides=conv['stride'],\n",
    "                   padding='valid' if conv['stride'] > 1 else 'same', # peculiar padding as darknet prefer left and top\n",
    "                   name='conv_' + str(conv['layer_idx']),\n",
    "                   use_bias=False if conv['bnorm'] else True)(x)\n",
    "        if conv['bnorm']: x = BatchNormalization(epsilon=0.001, name='bnorm_' + str(conv['layer_idx']))(x)\n",
    "        if conv['leaky']: x = LeakyReLU(alpha=0.1, name='leaky_' + str(conv['layer_idx']))(x)\n",
    "    return add([skip_connection, x]) if skip else x\n",
    "\n",
    "def make_yolov3_model():\n",
    "    input_image = Input(shape=(None, None, 3))\n",
    "    # Layer  0 => 4\n",
    "    x = _conv_block(input_image, [{'filter': 32, 'kernel': 3, 'stride': 1, 'bnorm': True, 'leaky': True, 'layer_idx': 0},\n",
    "                                  {'filter': 64, 'kernel': 3, 'stride': 2, 'bnorm': True, 'leaky': True, 'layer_idx': 1},\n",
    "                                  {'filter': 32, 'kernel': 1, 'stride': 1, 'bnorm': True, 'leaky': True, 'layer_idx': 2},\n",
    "                                  {'filter': 64, 'kernel': 3, 'stride': 1, 'bnorm': True, 'leaky': True, 'layer_idx': 3}])\n",
    "    # Layer  5 => 8\n",
    "    x = _conv_block(x, [{'filter': 128, 'kernel': 3, 'stride': 2, 'bnorm': True, 'leaky': True, 'layer_idx': 5},\n",
    "                        {'filter':  64, 'kernel': 1, 'stride': 1, 'bnorm': True, 'leaky': True, 'layer_idx': 6},\n",
    "                        {'filter': 128, 'kernel': 3, 'stride': 1, 'bnorm': True, 'leaky': True, 'layer_idx': 7}])\n",
    "    # Layer  9 => 11\n",
    "    x = _conv_block(x, [{'filter':  64, 'kernel': 1, 'stride': 1, 'bnorm': True, 'leaky': True, 'layer_idx': 9},\n",
    "                        {'filter': 128, 'kernel': 3, 'stride': 1, 'bnorm': True, 'leaky': True, 'layer_idx': 10}])\n",
    "    # Layer 12 => 15\n",
    "    x = _conv_block(x, [{'filter': 256, 'kernel': 3, 'stride': 2, 'bnorm': True, 'leaky': True, 'layer_idx': 12},\n",
    "                        {'filter': 128, 'kernel': 1, 'stride': 1, 'bnorm': True, 'leaky': True, 'layer_idx': 13},\n",
    "                        {'filter': 256, 'kernel': 3, 'stride': 1, 'bnorm': True, 'leaky': True, 'layer_idx': 14}])\n",
    "    # Layer 16 => 36\n",
    "    for i in range(7):\n",
    "        x = _conv_block(x, [{'filter': 128, 'kernel': 1, 'stride': 1, 'bnorm': True, 'leaky': True, 'layer_idx': 16+i*3},\n",
    "                            {'filter': 256, 'kernel': 3, 'stride': 1, 'bnorm': True, 'leaky': True, 'layer_idx': 17+i*3}])\n",
    "    skip_36 = x\n",
    "    # Layer 37 => 40\n",
    "    x = _conv_block(x, [{'filter': 512, 'kernel': 3, 'stride': 2, 'bnorm': True, 'leaky': True, 'layer_idx': 37},\n",
    "                        {'filter': 256, 'kernel': 1, 'stride': 1, 'bnorm': True, 'leaky': True, 'layer_idx': 38},\n",
    "                        {'filter': 512, 'kernel': 3, 'stride': 1, 'bnorm': True, 'leaky': True, 'layer_idx': 39}])\n",
    "    # Layer 41 => 61\n",
    "    for i in range(7):\n",
    "        x = _conv_block(x, [{'filter': 256, 'kernel': 1, 'stride': 1, 'bnorm': True, 'leaky': True, 'layer_idx': 41+i*3},\n",
    "                            {'filter': 512, 'kernel': 3, 'stride': 1, 'bnorm': True, 'leaky': True, 'layer_idx': 42+i*3}])\n",
    "    skip_61 = x\n",
    "    # Layer 62 => 65\n",
    "    x = _conv_block(x, [{'filter': 1024, 'kernel': 3, 'stride': 2, 'bnorm': True, 'leaky': True, 'layer_idx': 62},\n",
    "                        {'filter':  512, 'kernel': 1, 'stride': 1, 'bnorm': True, 'leaky': True, 'layer_idx': 63},\n",
    "                        {'filter': 1024, 'kernel': 3, 'stride': 1, 'bnorm': True, 'leaky': True, 'layer_idx': 64}])\n",
    "    # Layer 66 => 74\n",
    "    for i in range(3):\n",
    "        x = _conv_block(x, [{'filter':  512, 'kernel': 1, 'stride': 1, 'bnorm': True, 'leaky': True, 'layer_idx': 66+i*3},\n",
    "                            {'filter': 1024, 'kernel': 3, 'stride': 1, 'bnorm': True, 'leaky': True, 'layer_idx': 67+i*3}])\n",
    "    # Layer 75 => 79\n",
    "    x = _conv_block(x, [{'filter':  512, 'kernel': 1, 'stride': 1, 'bnorm': True, 'leaky': True, 'layer_idx': 75},\n",
    "                        {'filter': 1024, 'kernel': 3, 'stride': 1, 'bnorm': True, 'leaky': True, 'layer_idx': 76},\n",
    "                        {'filter':  512, 'kernel': 1, 'stride': 1, 'bnorm': True, 'leaky': True, 'layer_idx': 77},\n",
    "                        {'filter': 1024, 'kernel': 3, 'stride': 1, 'bnorm': True, 'leaky': True, 'layer_idx': 78},\n",
    "                        {'filter':  512, 'kernel': 1, 'stride': 1, 'bnorm': True, 'leaky': True, 'layer_idx': 79}], skip=False)\n",
    "    # Layer 80 => 82\n",
    "    yolo_82 = _conv_block(x, [{'filter': 1024, 'kernel': 3, 'stride': 1, 'bnorm': True,  'leaky': True,  'layer_idx': 80},\n",
    "                              {'filter':  255, 'kernel': 1, 'stride': 1, 'bnorm': False, 'leaky': False, 'layer_idx': 81}], skip=False)\n",
    "    # Layer 83 => 86\n",
    "    x = _conv_block(x, [{'filter': 256, 'kernel': 1, 'stride': 1, 'bnorm': True, 'leaky': True, 'layer_idx': 84}], skip=False)\n",
    "    x = UpSampling2D(2)(x)\n",
    "    x = concatenate([x, skip_61])\n",
    "    # Layer 87 => 91\n",
    "    x = _conv_block(x, [{'filter': 256, 'kernel': 1, 'stride': 1, 'bnorm': True, 'leaky': True, 'layer_idx': 87},\n",
    "                        {'filter': 512, 'kernel': 3, 'stride': 1, 'bnorm': True, 'leaky': True, 'layer_idx': 88},\n",
    "                        {'filter': 256, 'kernel': 1, 'stride': 1, 'bnorm': True, 'leaky': True, 'layer_idx': 89},\n",
    "                        {'filter': 512, 'kernel': 3, 'stride': 1, 'bnorm': True, 'leaky': True, 'layer_idx': 90},\n",
    "                        {'filter': 256, 'kernel': 1, 'stride': 1, 'bnorm': True, 'leaky': True, 'layer_idx': 91}], skip=False)\n",
    "    # Layer 92 => 94\n",
    "    yolo_94 = _conv_block(x, [{'filter': 512, 'kernel': 3, 'stride': 1, 'bnorm': True,  'leaky': True,  'layer_idx': 92},\n",
    "                              {'filter': 255, 'kernel': 1, 'stride': 1, 'bnorm': False, 'leaky': False, 'layer_idx': 93}], skip=False)\n",
    "    # Layer 95 => 98\n",
    "    x = _conv_block(x, [{'filter': 128, 'kernel': 1, 'stride': 1, 'bnorm': True, 'leaky': True,   'layer_idx': 96}], skip=False)\n",
    "    x = UpSampling2D(2)(x)\n",
    "    x = concatenate([x, skip_36])\n",
    "    # Layer 99 => 106\n",
    "    yolo_106 = _conv_block(x, [{'filter': 128, 'kernel': 1, 'stride': 1, 'bnorm': True,  'leaky': True,  'layer_idx': 99},\n",
    "                               {'filter': 256, 'kernel': 3, 'stride': 1, 'bnorm': True,  'leaky': True,  'layer_idx': 100},\n",
    "                               {'filter': 128, 'kernel': 1, 'stride': 1, 'bnorm': True,  'leaky': True,  'layer_idx': 101},\n",
    "                               {'filter': 256, 'kernel': 3, 'stride': 1, 'bnorm': True,  'leaky': True,  'layer_idx': 102},\n",
    "                               {'filter': 128, 'kernel': 1, 'stride': 1, 'bnorm': True,  'leaky': True,  'layer_idx': 103},\n",
    "                               {'filter': 256, 'kernel': 3, 'stride': 1, 'bnorm': True,  'leaky': True,  'layer_idx': 104},\n",
    "                               {'filter': 255, 'kernel': 1, 'stride': 1, 'bnorm': False, 'leaky': False, 'layer_idx': 105}], skip=False)\n",
    "    model = Model(input_image, [yolo_82, yolo_94, yolo_106])\n",
    "    return model\n",
    "\n",
    "class WeightReader:\n",
    "    def __init__(self, weight_file):\n",
    "        with open(weight_file, 'rb') as w_f:\n",
    "            major,\t= struct.unpack('i', w_f.read(4))\n",
    "            minor,\t= struct.unpack('i', w_f.read(4))\n",
    "            revision, = struct.unpack('i', w_f.read(4))\n",
    "            if (major*10 + minor) >= 2 and major < 1000 and minor < 1000:\n",
    "                w_f.read(8)\n",
    "            else:\n",
    "                w_f.read(4)\n",
    "            transpose = (major > 1000) or (minor > 1000)\n",
    "            binary = w_f.read()\n",
    "        self.offset = 0\n",
    "        self.all_weights = np.frombuffer(binary, dtype='float32')\n",
    " \n",
    "    def read_bytes(self, size):\n",
    "        self.offset = self.offset + size\n",
    "        return self.all_weights[self.offset-size:self.offset]\n",
    "\n",
    "    def load_weights(self, model):\n",
    "        for i in range(106):\n",
    "            try:\n",
    "                conv_layer = model.get_layer('conv_' + str(i))\n",
    "                print(\"loading weights of convolution #\" + str(i))\n",
    "                if i not in [81, 93, 105]:\n",
    "                    norm_layer = model.get_layer('bnorm_' + str(i))\n",
    "                    size = np.prod(norm_layer.get_weights()[0].shape)\n",
    "                    beta  = self.read_bytes(size) # bias\n",
    "                    gamma = self.read_bytes(size) # scale\n",
    "                    mean  = self.read_bytes(size) # mean\n",
    "                    var   = self.read_bytes(size) # variance\n",
    "                    weights = norm_layer.set_weights([gamma, beta, mean, var])\n",
    "                if len(conv_layer.get_weights()) > 1:\n",
    "                    bias   = self.read_bytes(np.prod(conv_layer.get_weights()[1].shape))\n",
    "                    kernel = self.read_bytes(np.prod(conv_layer.get_weights()[0].shape))\n",
    "                    kernel = kernel.reshape(list(reversed(conv_layer.get_weights()[0].shape)))\n",
    "                    kernel = kernel.transpose([2,3,1,0])\n",
    "                    conv_layer.set_weights([kernel, bias])\n",
    "                else:\n",
    "                    kernel = self.read_bytes(np.prod(conv_layer.get_weights()[0].shape))\n",
    "                    kernel = kernel.reshape(list(reversed(conv_layer.get_weights()[0].shape)))\n",
    "                    kernel = kernel.transpose([2,3,1,0])\n",
    "                    conv_layer.set_weights([kernel])\n",
    "            except ValueError:\n",
    "                print(\"no convolution #\" + str(i))\n",
    "\n",
    "    def reset(self):\n",
    "        self.offset = 0\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visualization Functions\n",
    "The underlying class and functions are used to show the predictions of the detected objects in the original picture."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from matplotlib import pyplot\n",
    "from matplotlib.patches import Rectangle\n",
    " \n",
    "class BoundBox:\n",
    "    def __init__(self, xmin, ymin, xmax, ymax, objness = None, classes = None):\n",
    "        self.xmin = xmin\n",
    "        self.ymin = ymin\n",
    "        self.xmax = xmax\n",
    "        self.ymax = ymax\n",
    "        self.objness = objness\n",
    "        self.classes = classes\n",
    "        self.label = -1\n",
    "        self.score = -1\n",
    "\n",
    "    def get_label(self):\n",
    "        if self.label == -1:\n",
    "            self.label = np.argmax(self.classes)\n",
    "\n",
    "        return self.label\n",
    "\n",
    "    def get_score(self):\n",
    "        if self.score == -1:\n",
    "            self.score = self.classes[self.get_label()]\n",
    " \n",
    "        return self.score\n",
    " \n",
    "def _sigmoid(x):\n",
    "    return 1. / (1. + np.exp(-x))\n",
    " \n",
    "def decode_netout(netout, anchors, obj_thresh, net_h, net_w):\n",
    "    grid_h, grid_w = netout.shape[:2]\n",
    "    nb_box = 3\n",
    "    netout = netout.reshape((grid_h, grid_w, nb_box, -1))\n",
    "    nb_class = netout.shape[-1] - 5\n",
    "    boxes = []\n",
    "    netout[..., :2]  = _sigmoid(netout[..., :2])\n",
    "    netout[..., 4:]  = _sigmoid(netout[..., 4:])\n",
    "    netout[..., 5:]  = netout[..., 4][..., np.newaxis] * netout[..., 5:]\n",
    "    netout[..., 5:] *= netout[..., 5:] > obj_thresh\n",
    " \n",
    "    for i in range(grid_h*grid_w):\n",
    "        row = i / grid_w\n",
    "        col = i % grid_w\n",
    "        for b in range(nb_box):\n",
    "            # 4th element is objectness score\n",
    "            objectness = netout[int(row)][int(col)][b][4]\n",
    "            if(objectness.all() <= obj_thresh): continue\n",
    "            # first 4 elements are x, y, w, and h\n",
    "            x, y, w, h = netout[int(row)][int(col)][b][:4]\n",
    "            x = (col + x) / grid_w # center position, unit: image width\n",
    "            y = (row + y) / grid_h # center position, unit: image height\n",
    "            w = anchors[2 * b + 0] * np.exp(w) / net_w # unit: image width\n",
    "            h = anchors[2 * b + 1] * np.exp(h) / net_h # unit: image height\n",
    "            # last elements are class probabilities\n",
    "            classes = netout[int(row)][col][b][5:]\n",
    "            box = BoundBox(x-w/2, y-h/2, x+w/2, y+h/2, objectness, classes)\n",
    "            boxes.append(box)\n",
    "    return boxes\n",
    " \n",
    "def correct_yolo_boxes(boxes, image_h, image_w, net_h, net_w):\n",
    "    new_w, new_h = net_w, net_h\n",
    "    for i in range(len(boxes)):\n",
    "        x_offset, x_scale = (net_w - new_w)/2./net_w, float(new_w)/net_w\n",
    "        y_offset, y_scale = (net_h - new_h)/2./net_h, float(new_h)/net_h\n",
    "        boxes[i].xmin = int((boxes[i].xmin - x_offset) / x_scale * image_w)\n",
    "        boxes[i].xmax = int((boxes[i].xmax - x_offset) / x_scale * image_w)\n",
    "        boxes[i].ymin = int((boxes[i].ymin - y_offset) / y_scale * image_h)\n",
    "        boxes[i].ymax = int((boxes[i].ymax - y_offset) / y_scale * image_h)\n",
    "\n",
    "def _interval_overlap(interval_a, interval_b):\n",
    "    x1, x2 = interval_a\n",
    "    x3, x4 = interval_b\n",
    "    if x3 < x1:\n",
    "        if x4 < x1:\n",
    "            return 0\n",
    "        else:\n",
    "            return min(x2,x4) - x1\n",
    "    else:\n",
    "        if x2 < x3:\n",
    "            return 0\n",
    "        else:\n",
    "            return min(x2,x4) - x3\n",
    "\n",
    "def bbox_iou(box1, box2):\n",
    "    intersect_w = _interval_overlap([box1.xmin, box1.xmax], [box2.xmin, box2.xmax])\n",
    "    intersect_h = _interval_overlap([box1.ymin, box1.ymax], [box2.ymin, box2.ymax])\n",
    "    intersect = intersect_w * intersect_h\n",
    "    w1, h1 = box1.xmax-box1.xmin, box1.ymax-box1.ymin\n",
    "    w2, h2 = box2.xmax-box2.xmin, box2.ymax-box2.ymin\n",
    "    union = w1*h1 + w2*h2 - intersect\n",
    "    return float(intersect) / union\n",
    " \n",
    "def do_nms(boxes, nms_thresh):\n",
    "    if len(boxes) > 0:\n",
    "        nb_class = len(boxes[0].classes)\n",
    "    else:\n",
    "        return\n",
    "    for c in range(nb_class):\n",
    "        sorted_indices = np.argsort([-box.classes[c] for box in boxes])\n",
    "        for i in range(len(sorted_indices)):\n",
    "            index_i = sorted_indices[i]\n",
    "            if boxes[index_i].classes[c] == 0: continue\n",
    "            for j in range(i+1, len(sorted_indices)):\n",
    "                index_j = sorted_indices[j]\n",
    "                if bbox_iou(boxes[index_i], boxes[index_j]) >= nms_thresh:\n",
    "                    boxes[index_j].classes[c] = 0\n",
    "\n",
    "# load and prepare an image\n",
    "def load_image_pixels(filename, shape):\n",
    "    # load the image to get its shape\n",
    "    image = load_img(filename)\n",
    "    width, height = image.size\n",
    "    # load the image with the required size\n",
    "    image = load_img(filename, target_size=shape)\n",
    "    # convert to numpy array\n",
    "    image = img_to_array(image)\n",
    "    # scale pixel values to [0, 1]\n",
    "    image = image.astype('float32')\n",
    "    image /= 255.0\n",
    "    # add a dimension so that we have one sample\n",
    "    image = expand_dims(image, 0)\n",
    "    return image, width, height\n",
    " \n",
    "# get all of the results above a threshold\n",
    "def get_boxes(boxes, labels, thresh):\n",
    "    v_boxes, v_labels, v_scores = list(), list(), list()\n",
    "    # enumerate all boxes\n",
    "    for box in boxes:\n",
    "        # enumerate all possible labels\n",
    "        for i in range(len(labels)):\n",
    "            # check if the threshold for this label is high enough\n",
    "            if box.classes[i] > thresh:\n",
    "                v_boxes.append(box)\n",
    "                v_labels.append(labels[i])\n",
    "                v_scores.append(box.classes[i]*100)\n",
    "                # don't break, many labels may trigger for one box\n",
    "    return v_boxes, v_labels, v_scores\n",
    " \n",
    "# draw all results\n",
    "def draw_boxes(filename, v_boxes, v_labels, v_scores):\n",
    "    # load the image\n",
    "    data = pyplot.imread(filename)\n",
    "    # plot the image\n",
    "    pyplot.imshow(data)\n",
    "    # get the context for drawing boxes\n",
    "    ax = pyplot.gca()\n",
    "    # plot each box\n",
    "    for i in range(len(v_boxes)):\n",
    "        box = v_boxes[i]\n",
    "        # get coordinates\n",
    "        y1, x1, y2, x2 = box.ymin, box.xmin, box.ymax, box.xmax\n",
    "        # calculate width and height of the box\n",
    "        width, height = x2 - x1, y2 - y1\n",
    "        # create the shape\n",
    "        rect = Rectangle((x1, y1), width, height, fill=False, color='white')\n",
    "        # draw the box\n",
    "        ax.add_patch(rect)\n",
    "        # draw text and score in top left corner\n",
    "        label = \"%s (%.3f)\" % (v_labels[i], v_scores[i])\n",
    "        pyplot.text(x1, y1, label, color='white')\n",
    "    # show the plot\n",
    "    pyplot.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create YOLOv3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "loading weights of convolution #0\n",
      "loading weights of convolution #1\n",
      "loading weights of convolution #2\n",
      "loading weights of convolution #3\n",
      "no convolution #4\n",
      "loading weights of convolution #5\n",
      "loading weights of convolution #6\n",
      "loading weights of convolution #7\n",
      "no convolution #8\n",
      "loading weights of convolution #9\n",
      "loading weights of convolution #10\n",
      "no convolution #11\n",
      "loading weights of convolution #12\n",
      "loading weights of convolution #13\n",
      "loading weights of convolution #14\n",
      "no convolution #15\n",
      "loading weights of convolution #16\n",
      "loading weights of convolution #17\n",
      "no convolution #18\n",
      "loading weights of convolution #19\n",
      "loading weights of convolution #20\n",
      "no convolution #21\n",
      "loading weights of convolution #22\n",
      "loading weights of convolution #23\n",
      "no convolution #24\n",
      "loading weights of convolution #25\n",
      "loading weights of convolution #26\n",
      "no convolution #27\n",
      "loading weights of convolution #28\n",
      "loading weights of convolution #29\n",
      "no convolution #30\n",
      "loading weights of convolution #31\n",
      "loading weights of convolution #32\n",
      "no convolution #33\n",
      "loading weights of convolution #34\n",
      "loading weights of convolution #35\n",
      "no convolution #36\n",
      "loading weights of convolution #37\n",
      "loading weights of convolution #38\n",
      "loading weights of convolution #39\n",
      "no convolution #40\n",
      "loading weights of convolution #41\n",
      "loading weights of convolution #42\n",
      "no convolution #43\n",
      "loading weights of convolution #44\n",
      "loading weights of convolution #45\n",
      "no convolution #46\n",
      "loading weights of convolution #47\n",
      "loading weights of convolution #48\n",
      "no convolution #49\n",
      "loading weights of convolution #50\n",
      "loading weights of convolution #51\n",
      "no convolution #52\n",
      "loading weights of convolution #53\n",
      "loading weights of convolution #54\n",
      "no convolution #55\n",
      "loading weights of convolution #56\n",
      "loading weights of convolution #57\n",
      "no convolution #58\n",
      "loading weights of convolution #59\n",
      "loading weights of convolution #60\n",
      "no convolution #61\n",
      "loading weights of convolution #62\n",
      "loading weights of convolution #63\n",
      "loading weights of convolution #64\n",
      "no convolution #65\n",
      "loading weights of convolution #66\n",
      "loading weights of convolution #67\n",
      "no convolution #68\n",
      "loading weights of convolution #69\n",
      "loading weights of convolution #70\n",
      "no convolution #71\n",
      "loading weights of convolution #72\n",
      "loading weights of convolution #73\n",
      "no convolution #74\n",
      "loading weights of convolution #75\n",
      "loading weights of convolution #76\n",
      "loading weights of convolution #77\n",
      "loading weights of convolution #78\n",
      "loading weights of convolution #79\n",
      "loading weights of convolution #80\n",
      "loading weights of convolution #81\n",
      "no convolution #82\n",
      "no convolution #83\n",
      "loading weights of convolution #84\n",
      "no convolution #85\n",
      "no convolution #86\n",
      "loading weights of convolution #87\n",
      "loading weights of convolution #88\n",
      "loading weights of convolution #89\n",
      "loading weights of convolution #90\n",
      "loading weights of convolution #91\n",
      "loading weights of convolution #92\n",
      "loading weights of convolution #93\n",
      "no convolution #94\n",
      "no convolution #95\n",
      "loading weights of convolution #96\n",
      "no convolution #97\n",
      "no convolution #98\n",
      "loading weights of convolution #99\n",
      "loading weights of convolution #100\n",
      "loading weights of convolution #101\n",
      "loading weights of convolution #102\n",
      "loading weights of convolution #103\n",
      "loading weights of convolution #104\n",
      "loading weights of convolution #105\n"
     ]
    }
   ],
   "source": [
    "# define the model\n",
    "model = make_yolov3_model()\n",
    "\n",
    "# load the model weights\n",
    "# I have loaded the pretrained weights in a separate dataset\n",
    "weight_reader = WeightReader('../input/lyft-3d-recognition/yolov3.weights')\n",
    "\n",
    "# set the model weights into the model\n",
    "weight_reader.load_weights(model)\n",
    "\n",
    "# save the model to file\n",
    "model.save('model.h5')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load YOLOv3 Model\n",
    "As the weights have now been saved in an h5 file (which can be found in the output files), we load it into the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/lib/python3.6/site-packages/keras/engine/saving.py:292: UserWarning: No training configuration found in save file: the model was *not* compiled. Compile it manually.\n",
      "  warnings.warn('No training configuration found in save file: '\n"
     ]
    }
   ],
   "source": [
    "# load yolov3 model\n",
    "from keras.models import load_model\n",
    "model = load_model('model.h5')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "__________________________________________________________________________________________________\n",
      "Layer (type)                    Output Shape         Param #     Connected to                     \n",
      "==================================================================================================\n",
      "input_1 (InputLayer)            (None, None, None, 3 0                                            \n",
      "__________________________________________________________________________________________________\n",
      "conv_0 (Conv2D)                 (None, None, None, 3 864         input_1[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_0 (BatchNormalization)    (None, None, None, 3 128         conv_0[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "leaky_0 (LeakyReLU)             (None, None, None, 3 0           bnorm_0[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "zero_padding2d_1 (ZeroPadding2D (None, None, None, 3 0           leaky_0[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "conv_1 (Conv2D)                 (None, None, None, 6 18432       zero_padding2d_1[0][0]           \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_1 (BatchNormalization)    (None, None, None, 6 256         conv_1[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "leaky_1 (LeakyReLU)             (None, None, None, 6 0           bnorm_1[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "conv_2 (Conv2D)                 (None, None, None, 3 2048        leaky_1[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_2 (BatchNormalization)    (None, None, None, 3 128         conv_2[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "leaky_2 (LeakyReLU)             (None, None, None, 3 0           bnorm_2[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "conv_3 (Conv2D)                 (None, None, None, 6 18432       leaky_2[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_3 (BatchNormalization)    (None, None, None, 6 256         conv_3[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "leaky_3 (LeakyReLU)             (None, None, None, 6 0           bnorm_3[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "add_1 (Add)                     (None, None, None, 6 0           leaky_1[0][0]                    \n",
      "                                                                 leaky_3[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "zero_padding2d_2 (ZeroPadding2D (None, None, None, 6 0           add_1[0][0]                      \n",
      "__________________________________________________________________________________________________\n",
      "conv_5 (Conv2D)                 (None, None, None, 1 73728       zero_padding2d_2[0][0]           \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_5 (BatchNormalization)    (None, None, None, 1 512         conv_5[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "leaky_5 (LeakyReLU)             (None, None, None, 1 0           bnorm_5[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "conv_6 (Conv2D)                 (None, None, None, 6 8192        leaky_5[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_6 (BatchNormalization)    (None, None, None, 6 256         conv_6[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "leaky_6 (LeakyReLU)             (None, None, None, 6 0           bnorm_6[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "conv_7 (Conv2D)                 (None, None, None, 1 73728       leaky_6[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_7 (BatchNormalization)    (None, None, None, 1 512         conv_7[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "leaky_7 (LeakyReLU)             (None, None, None, 1 0           bnorm_7[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "add_2 (Add)                     (None, None, None, 1 0           leaky_5[0][0]                    \n",
      "                                                                 leaky_7[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "conv_9 (Conv2D)                 (None, None, None, 6 8192        add_2[0][0]                      \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_9 (BatchNormalization)    (None, None, None, 6 256         conv_9[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "leaky_9 (LeakyReLU)             (None, None, None, 6 0           bnorm_9[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "conv_10 (Conv2D)                (None, None, None, 1 73728       leaky_9[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_10 (BatchNormalization)   (None, None, None, 1 512         conv_10[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_10 (LeakyReLU)            (None, None, None, 1 0           bnorm_10[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "add_3 (Add)                     (None, None, None, 1 0           add_2[0][0]                      \n",
      "                                                                 leaky_10[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "zero_padding2d_3 (ZeroPadding2D (None, None, None, 1 0           add_3[0][0]                      \n",
      "__________________________________________________________________________________________________\n",
      "conv_12 (Conv2D)                (None, None, None, 2 294912      zero_padding2d_3[0][0]           \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_12 (BatchNormalization)   (None, None, None, 2 1024        conv_12[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_12 (LeakyReLU)            (None, None, None, 2 0           bnorm_12[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_13 (Conv2D)                (None, None, None, 1 32768       leaky_12[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_13 (BatchNormalization)   (None, None, None, 1 512         conv_13[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_13 (LeakyReLU)            (None, None, None, 1 0           bnorm_13[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_14 (Conv2D)                (None, None, None, 2 294912      leaky_13[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_14 (BatchNormalization)   (None, None, None, 2 1024        conv_14[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_14 (LeakyReLU)            (None, None, None, 2 0           bnorm_14[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "add_4 (Add)                     (None, None, None, 2 0           leaky_12[0][0]                   \n",
      "                                                                 leaky_14[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_16 (Conv2D)                (None, None, None, 1 32768       add_4[0][0]                      \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_16 (BatchNormalization)   (None, None, None, 1 512         conv_16[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_16 (LeakyReLU)            (None, None, None, 1 0           bnorm_16[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_17 (Conv2D)                (None, None, None, 2 294912      leaky_16[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_17 (BatchNormalization)   (None, None, None, 2 1024        conv_17[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_17 (LeakyReLU)            (None, None, None, 2 0           bnorm_17[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "add_5 (Add)                     (None, None, None, 2 0           add_4[0][0]                      \n",
      "                                                                 leaky_17[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_19 (Conv2D)                (None, None, None, 1 32768       add_5[0][0]                      \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_19 (BatchNormalization)   (None, None, None, 1 512         conv_19[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_19 (LeakyReLU)            (None, None, None, 1 0           bnorm_19[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_20 (Conv2D)                (None, None, None, 2 294912      leaky_19[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_20 (BatchNormalization)   (None, None, None, 2 1024        conv_20[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_20 (LeakyReLU)            (None, None, None, 2 0           bnorm_20[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "add_6 (Add)                     (None, None, None, 2 0           add_5[0][0]                      \n",
      "                                                                 leaky_20[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_22 (Conv2D)                (None, None, None, 1 32768       add_6[0][0]                      \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_22 (BatchNormalization)   (None, None, None, 1 512         conv_22[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_22 (LeakyReLU)            (None, None, None, 1 0           bnorm_22[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_23 (Conv2D)                (None, None, None, 2 294912      leaky_22[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_23 (BatchNormalization)   (None, None, None, 2 1024        conv_23[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_23 (LeakyReLU)            (None, None, None, 2 0           bnorm_23[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "add_7 (Add)                     (None, None, None, 2 0           add_6[0][0]                      \n",
      "                                                                 leaky_23[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_25 (Conv2D)                (None, None, None, 1 32768       add_7[0][0]                      \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_25 (BatchNormalization)   (None, None, None, 1 512         conv_25[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_25 (LeakyReLU)            (None, None, None, 1 0           bnorm_25[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_26 (Conv2D)                (None, None, None, 2 294912      leaky_25[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_26 (BatchNormalization)   (None, None, None, 2 1024        conv_26[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_26 (LeakyReLU)            (None, None, None, 2 0           bnorm_26[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "add_8 (Add)                     (None, None, None, 2 0           add_7[0][0]                      \n",
      "                                                                 leaky_26[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_28 (Conv2D)                (None, None, None, 1 32768       add_8[0][0]                      \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_28 (BatchNormalization)   (None, None, None, 1 512         conv_28[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_28 (LeakyReLU)            (None, None, None, 1 0           bnorm_28[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_29 (Conv2D)                (None, None, None, 2 294912      leaky_28[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_29 (BatchNormalization)   (None, None, None, 2 1024        conv_29[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_29 (LeakyReLU)            (None, None, None, 2 0           bnorm_29[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "add_9 (Add)                     (None, None, None, 2 0           add_8[0][0]                      \n",
      "                                                                 leaky_29[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_31 (Conv2D)                (None, None, None, 1 32768       add_9[0][0]                      \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_31 (BatchNormalization)   (None, None, None, 1 512         conv_31[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_31 (LeakyReLU)            (None, None, None, 1 0           bnorm_31[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_32 (Conv2D)                (None, None, None, 2 294912      leaky_31[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_32 (BatchNormalization)   (None, None, None, 2 1024        conv_32[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_32 (LeakyReLU)            (None, None, None, 2 0           bnorm_32[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "add_10 (Add)                    (None, None, None, 2 0           add_9[0][0]                      \n",
      "                                                                 leaky_32[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_34 (Conv2D)                (None, None, None, 1 32768       add_10[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_34 (BatchNormalization)   (None, None, None, 1 512         conv_34[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_34 (LeakyReLU)            (None, None, None, 1 0           bnorm_34[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_35 (Conv2D)                (None, None, None, 2 294912      leaky_34[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_35 (BatchNormalization)   (None, None, None, 2 1024        conv_35[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_35 (LeakyReLU)            (None, None, None, 2 0           bnorm_35[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "add_11 (Add)                    (None, None, None, 2 0           add_10[0][0]                     \n",
      "                                                                 leaky_35[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "zero_padding2d_4 (ZeroPadding2D (None, None, None, 2 0           add_11[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "conv_37 (Conv2D)                (None, None, None, 5 1179648     zero_padding2d_4[0][0]           \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_37 (BatchNormalization)   (None, None, None, 5 2048        conv_37[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_37 (LeakyReLU)            (None, None, None, 5 0           bnorm_37[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_38 (Conv2D)                (None, None, None, 2 131072      leaky_37[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_38 (BatchNormalization)   (None, None, None, 2 1024        conv_38[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_38 (LeakyReLU)            (None, None, None, 2 0           bnorm_38[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_39 (Conv2D)                (None, None, None, 5 1179648     leaky_38[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_39 (BatchNormalization)   (None, None, None, 5 2048        conv_39[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_39 (LeakyReLU)            (None, None, None, 5 0           bnorm_39[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "add_12 (Add)                    (None, None, None, 5 0           leaky_37[0][0]                   \n",
      "                                                                 leaky_39[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_41 (Conv2D)                (None, None, None, 2 131072      add_12[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_41 (BatchNormalization)   (None, None, None, 2 1024        conv_41[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_41 (LeakyReLU)            (None, None, None, 2 0           bnorm_41[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_42 (Conv2D)                (None, None, None, 5 1179648     leaky_41[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_42 (BatchNormalization)   (None, None, None, 5 2048        conv_42[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_42 (LeakyReLU)            (None, None, None, 5 0           bnorm_42[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "add_13 (Add)                    (None, None, None, 5 0           add_12[0][0]                     \n",
      "                                                                 leaky_42[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_44 (Conv2D)                (None, None, None, 2 131072      add_13[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_44 (BatchNormalization)   (None, None, None, 2 1024        conv_44[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_44 (LeakyReLU)            (None, None, None, 2 0           bnorm_44[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_45 (Conv2D)                (None, None, None, 5 1179648     leaky_44[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_45 (BatchNormalization)   (None, None, None, 5 2048        conv_45[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_45 (LeakyReLU)            (None, None, None, 5 0           bnorm_45[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "add_14 (Add)                    (None, None, None, 5 0           add_13[0][0]                     \n",
      "                                                                 leaky_45[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_47 (Conv2D)                (None, None, None, 2 131072      add_14[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_47 (BatchNormalization)   (None, None, None, 2 1024        conv_47[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_47 (LeakyReLU)            (None, None, None, 2 0           bnorm_47[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_48 (Conv2D)                (None, None, None, 5 1179648     leaky_47[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_48 (BatchNormalization)   (None, None, None, 5 2048        conv_48[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_48 (LeakyReLU)            (None, None, None, 5 0           bnorm_48[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "add_15 (Add)                    (None, None, None, 5 0           add_14[0][0]                     \n",
      "                                                                 leaky_48[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_50 (Conv2D)                (None, None, None, 2 131072      add_15[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_50 (BatchNormalization)   (None, None, None, 2 1024        conv_50[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_50 (LeakyReLU)            (None, None, None, 2 0           bnorm_50[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_51 (Conv2D)                (None, None, None, 5 1179648     leaky_50[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_51 (BatchNormalization)   (None, None, None, 5 2048        conv_51[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_51 (LeakyReLU)            (None, None, None, 5 0           bnorm_51[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "add_16 (Add)                    (None, None, None, 5 0           add_15[0][0]                     \n",
      "                                                                 leaky_51[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_53 (Conv2D)                (None, None, None, 2 131072      add_16[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_53 (BatchNormalization)   (None, None, None, 2 1024        conv_53[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_53 (LeakyReLU)            (None, None, None, 2 0           bnorm_53[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_54 (Conv2D)                (None, None, None, 5 1179648     leaky_53[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_54 (BatchNormalization)   (None, None, None, 5 2048        conv_54[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_54 (LeakyReLU)            (None, None, None, 5 0           bnorm_54[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "add_17 (Add)                    (None, None, None, 5 0           add_16[0][0]                     \n",
      "                                                                 leaky_54[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_56 (Conv2D)                (None, None, None, 2 131072      add_17[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_56 (BatchNormalization)   (None, None, None, 2 1024        conv_56[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_56 (LeakyReLU)            (None, None, None, 2 0           bnorm_56[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_57 (Conv2D)                (None, None, None, 5 1179648     leaky_56[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_57 (BatchNormalization)   (None, None, None, 5 2048        conv_57[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_57 (LeakyReLU)            (None, None, None, 5 0           bnorm_57[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "add_18 (Add)                    (None, None, None, 5 0           add_17[0][0]                     \n",
      "                                                                 leaky_57[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_59 (Conv2D)                (None, None, None, 2 131072      add_18[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_59 (BatchNormalization)   (None, None, None, 2 1024        conv_59[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_59 (LeakyReLU)            (None, None, None, 2 0           bnorm_59[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_60 (Conv2D)                (None, None, None, 5 1179648     leaky_59[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_60 (BatchNormalization)   (None, None, None, 5 2048        conv_60[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_60 (LeakyReLU)            (None, None, None, 5 0           bnorm_60[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "add_19 (Add)                    (None, None, None, 5 0           add_18[0][0]                     \n",
      "                                                                 leaky_60[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "zero_padding2d_5 (ZeroPadding2D (None, None, None, 5 0           add_19[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "conv_62 (Conv2D)                (None, None, None, 1 4718592     zero_padding2d_5[0][0]           \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_62 (BatchNormalization)   (None, None, None, 1 4096        conv_62[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_62 (LeakyReLU)            (None, None, None, 1 0           bnorm_62[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_63 (Conv2D)                (None, None, None, 5 524288      leaky_62[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_63 (BatchNormalization)   (None, None, None, 5 2048        conv_63[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_63 (LeakyReLU)            (None, None, None, 5 0           bnorm_63[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_64 (Conv2D)                (None, None, None, 1 4718592     leaky_63[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_64 (BatchNormalization)   (None, None, None, 1 4096        conv_64[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_64 (LeakyReLU)            (None, None, None, 1 0           bnorm_64[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "add_20 (Add)                    (None, None, None, 1 0           leaky_62[0][0]                   \n",
      "                                                                 leaky_64[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_66 (Conv2D)                (None, None, None, 5 524288      add_20[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_66 (BatchNormalization)   (None, None, None, 5 2048        conv_66[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_66 (LeakyReLU)            (None, None, None, 5 0           bnorm_66[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_67 (Conv2D)                (None, None, None, 1 4718592     leaky_66[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_67 (BatchNormalization)   (None, None, None, 1 4096        conv_67[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_67 (LeakyReLU)            (None, None, None, 1 0           bnorm_67[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "add_21 (Add)                    (None, None, None, 1 0           add_20[0][0]                     \n",
      "                                                                 leaky_67[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_69 (Conv2D)                (None, None, None, 5 524288      add_21[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_69 (BatchNormalization)   (None, None, None, 5 2048        conv_69[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_69 (LeakyReLU)            (None, None, None, 5 0           bnorm_69[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_70 (Conv2D)                (None, None, None, 1 4718592     leaky_69[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_70 (BatchNormalization)   (None, None, None, 1 4096        conv_70[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_70 (LeakyReLU)            (None, None, None, 1 0           bnorm_70[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "add_22 (Add)                    (None, None, None, 1 0           add_21[0][0]                     \n",
      "                                                                 leaky_70[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_72 (Conv2D)                (None, None, None, 5 524288      add_22[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_72 (BatchNormalization)   (None, None, None, 5 2048        conv_72[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_72 (LeakyReLU)            (None, None, None, 5 0           bnorm_72[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_73 (Conv2D)                (None, None, None, 1 4718592     leaky_72[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_73 (BatchNormalization)   (None, None, None, 1 4096        conv_73[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_73 (LeakyReLU)            (None, None, None, 1 0           bnorm_73[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "add_23 (Add)                    (None, None, None, 1 0           add_22[0][0]                     \n",
      "                                                                 leaky_73[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_75 (Conv2D)                (None, None, None, 5 524288      add_23[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_75 (BatchNormalization)   (None, None, None, 5 2048        conv_75[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_75 (LeakyReLU)            (None, None, None, 5 0           bnorm_75[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_76 (Conv2D)                (None, None, None, 1 4718592     leaky_75[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_76 (BatchNormalization)   (None, None, None, 1 4096        conv_76[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_76 (LeakyReLU)            (None, None, None, 1 0           bnorm_76[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_77 (Conv2D)                (None, None, None, 5 524288      leaky_76[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_77 (BatchNormalization)   (None, None, None, 5 2048        conv_77[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_77 (LeakyReLU)            (None, None, None, 5 0           bnorm_77[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_78 (Conv2D)                (None, None, None, 1 4718592     leaky_77[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_78 (BatchNormalization)   (None, None, None, 1 4096        conv_78[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_78 (LeakyReLU)            (None, None, None, 1 0           bnorm_78[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_79 (Conv2D)                (None, None, None, 5 524288      leaky_78[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_79 (BatchNormalization)   (None, None, None, 5 2048        conv_79[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_79 (LeakyReLU)            (None, None, None, 5 0           bnorm_79[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_84 (Conv2D)                (None, None, None, 2 131072      leaky_79[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_84 (BatchNormalization)   (None, None, None, 2 1024        conv_84[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_84 (LeakyReLU)            (None, None, None, 2 0           bnorm_84[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "up_sampling2d_1 (UpSampling2D)  (None, None, None, 2 0           leaky_84[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "concatenate_1 (Concatenate)     (None, None, None, 7 0           up_sampling2d_1[0][0]            \n",
      "                                                                 add_19[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "conv_87 (Conv2D)                (None, None, None, 2 196608      concatenate_1[0][0]              \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_87 (BatchNormalization)   (None, None, None, 2 1024        conv_87[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_87 (LeakyReLU)            (None, None, None, 2 0           bnorm_87[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_88 (Conv2D)                (None, None, None, 5 1179648     leaky_87[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_88 (BatchNormalization)   (None, None, None, 5 2048        conv_88[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_88 (LeakyReLU)            (None, None, None, 5 0           bnorm_88[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_89 (Conv2D)                (None, None, None, 2 131072      leaky_88[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_89 (BatchNormalization)   (None, None, None, 2 1024        conv_89[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_89 (LeakyReLU)            (None, None, None, 2 0           bnorm_89[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_90 (Conv2D)                (None, None, None, 5 1179648     leaky_89[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_90 (BatchNormalization)   (None, None, None, 5 2048        conv_90[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_90 (LeakyReLU)            (None, None, None, 5 0           bnorm_90[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_91 (Conv2D)                (None, None, None, 2 131072      leaky_90[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_91 (BatchNormalization)   (None, None, None, 2 1024        conv_91[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_91 (LeakyReLU)            (None, None, None, 2 0           bnorm_91[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_96 (Conv2D)                (None, None, None, 1 32768       leaky_91[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_96 (BatchNormalization)   (None, None, None, 1 512         conv_96[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_96 (LeakyReLU)            (None, None, None, 1 0           bnorm_96[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "up_sampling2d_2 (UpSampling2D)  (None, None, None, 1 0           leaky_96[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "concatenate_2 (Concatenate)     (None, None, None, 3 0           up_sampling2d_2[0][0]            \n",
      "                                                                 add_11[0][0]                     \n",
      "__________________________________________________________________________________________________\n",
      "conv_99 (Conv2D)                (None, None, None, 1 49152       concatenate_2[0][0]              \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_99 (BatchNormalization)   (None, None, None, 1 512         conv_99[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "leaky_99 (LeakyReLU)            (None, None, None, 1 0           bnorm_99[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_100 (Conv2D)               (None, None, None, 2 294912      leaky_99[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_100 (BatchNormalization)  (None, None, None, 2 1024        conv_100[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "leaky_100 (LeakyReLU)           (None, None, None, 2 0           bnorm_100[0][0]                  \n",
      "__________________________________________________________________________________________________\n",
      "conv_101 (Conv2D)               (None, None, None, 1 32768       leaky_100[0][0]                  \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_101 (BatchNormalization)  (None, None, None, 1 512         conv_101[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "leaky_101 (LeakyReLU)           (None, None, None, 1 0           bnorm_101[0][0]                  \n",
      "__________________________________________________________________________________________________\n",
      "conv_102 (Conv2D)               (None, None, None, 2 294912      leaky_101[0][0]                  \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_102 (BatchNormalization)  (None, None, None, 2 1024        conv_102[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "leaky_102 (LeakyReLU)           (None, None, None, 2 0           bnorm_102[0][0]                  \n",
      "__________________________________________________________________________________________________\n",
      "conv_103 (Conv2D)               (None, None, None, 1 32768       leaky_102[0][0]                  \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_103 (BatchNormalization)  (None, None, None, 1 512         conv_103[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "leaky_103 (LeakyReLU)           (None, None, None, 1 0           bnorm_103[0][0]                  \n",
      "__________________________________________________________________________________________________\n",
      "conv_80 (Conv2D)                (None, None, None, 1 4718592     leaky_79[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_92 (Conv2D)                (None, None, None, 5 1179648     leaky_91[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_104 (Conv2D)               (None, None, None, 2 294912      leaky_103[0][0]                  \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_80 (BatchNormalization)   (None, None, None, 1 4096        conv_80[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_92 (BatchNormalization)   (None, None, None, 5 2048        conv_92[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "bnorm_104 (BatchNormalization)  (None, None, None, 2 1024        conv_104[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "leaky_80 (LeakyReLU)            (None, None, None, 1 0           bnorm_80[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "leaky_92 (LeakyReLU)            (None, None, None, 5 0           bnorm_92[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "leaky_104 (LeakyReLU)           (None, None, None, 2 0           bnorm_104[0][0]                  \n",
      "__________________________________________________________________________________________________\n",
      "conv_81 (Conv2D)                (None, None, None, 2 261375      leaky_80[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_93 (Conv2D)                (None, None, None, 2 130815      leaky_92[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv_105 (Conv2D)               (None, None, None, 2 65535       leaky_104[0][0]                  \n",
      "==================================================================================================\n",
      "Total params: 62,001,757\n",
      "Trainable params: 61,949,149\n",
      "Non-trainable params: 52,608\n",
      "__________________________________________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Predict Objects in first 10 training images\n",
    "We now try to check if our implementation works, by predicting the objects in the first ten images of the training set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Parameters used in the Dataset, on which YOLOv3 was pretrained\n",
    "anchors = [[116,90, 156,198, 373,326], [30,61, 62,45, 59,119], [10,13, 16,30, 33,23]]\n",
    "\n",
    "# define the expected input shape for the model\n",
    "WIDTH, HEIGHT = 416, 416\n",
    "\n",
    "# define the probability threshold for detected objects\n",
    "class_threshold = 0.7"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "from matplotlib import pyplot as plt\n",
    "images = os.listdir('../input/3d-object-detection-for-autonomous-vehicles/train_images')[:10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "from numpy import expand_dims\n",
    "from keras.preprocessing.image import load_img, img_to_array\n",
    "\n",
    "# load and prepare an image\n",
    "def load_image_pixels(filename, shape):\n",
    "    '''\n",
    "    Function preprocess the images to 416x416, which is the standard input shape for YOLOv3, \n",
    "    and also keeps track of the originl shape, which is later used to draw the boxes.\n",
    "    \n",
    "    paramters:\n",
    "    filename {String}: path to the image\n",
    "    shape {tuple}: shape of the input dimensions of the network\n",
    "    \n",
    "    returns:\n",
    "    image {PIL}: image of shape 'shape'\n",
    "    width {int}: original width of the picture\n",
    "    height {int}: original height of the picture\n",
    "    '''\n",
    "    # load the image to get its shape\n",
    "    image = load_img(filename)\n",
    "    width, height = image.size\n",
    "    \n",
    "    # load the image with the required size\n",
    "    image = load_img(filename, target_size=shape)\n",
    "    \n",
    "    # convert to numpy array\n",
    "    image = img_to_array(image)\n",
    "    \n",
    "    # scale pixel values to [0, 1]\n",
    "    image = image.astype('float32')\n",
    "    image /= 255.0\n",
    "    \n",
    "    # add a dimension so that we have one sample\n",
    "    image = expand_dims(image, 0)\n",
    "    return image, width, height"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Iterate through the pictures and detect the objects."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "images = os.listdir(\"/kaggle/input/drone-dataset-with-steering-angle-2/center\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "for file in images:\n",
    "    photo_filename = DATA_PATH + \"/\" + file\n",
    "    \n",
    "    # load picture with old dimensions\n",
    "    image, image_w, image_h = load_image_pixels(photo_filename, (WIDTH, HEIGHT))\n",
    "    \n",
    "    # Predict image\n",
    "    yhat = model.predict(image)\n",
    "    \n",
    "    # Create boxes\n",
    "    boxes = list()\n",
    "    for i in range(3):\n",
    "        # decode the output of the network\n",
    "        boxes += decode_netout(yhat[i][0], anchors[i], class_threshold, HEIGHT, WIDTH)\n",
    "\n",
    "    # correct the sizes of the bounding boxes for the shape of the image\n",
    "    correct_yolo_boxes(boxes, image_h, image_w, HEIGHT, WIDTH)\n",
    "\n",
    "    # suppress non-maximal boxes\n",
    "    do_nms(boxes, 0.5)\n",
    "\n",
    "    # define the labels (Filtered only the ones relevant for this task, which were used in pretraining the YOLOv3 model)\n",
    "    labels = [\"person\", \"bicycle\", \"car\", \"motorbike\", \"aeroplane\", \"bus\", \"train\", \"truck\",\"boat\"]\n",
    "\n",
    "    # get the details of the detected objects\n",
    "    v_boxes, v_labels, v_scores = get_boxes(boxes, labels, class_threshold)\n",
    "\n",
    "    # summarize what we found\n",
    "    for i in range(len(v_boxes)):\n",
    "\n",
    "        print(v_labels[i], v_scores[i])\n",
    "\n",
    "    # draw what we found\n",
    "    draw_boxes(photo_filename, v_boxes, v_labels, v_scores)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}