3_cv_basics/6_blob_detection.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# BLOB DETECTION\n",
    "\n",
    "BLOB stands for **Binary Large Object**\n",
    "\n",
    "Informally a blob is a region of an image in which some properties like intensity or color are approximately constant.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Importing cv2 and numpy libary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import cv2\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Preproccessing\n",
    "## 1. Choosing the color of blob\n",
    "* loading the live video through webcam using cv2.VideoCapture()\n",
    "* Reading the frame using video.read()\n",
    "* select a region in a frame of your color choice using cv2.selectROI()\n",
    "![](./images/roi_blob.png)\n",
    "\n",
    "## 2. Converting to HSV and finding bounding values for mask\n",
    "* Converting the frame to HSV format using cv2.cvtColor(_,cv2.COLOR_BGR2HSV)\n",
    "    * Why converting to HSV?  \n",
    "        Since we are using the web cam the intensity and illumination of consecutive frame does not remain same.  \n",
    "        Hence to find the color in range instead of particular color. HSV format is useful as H value denotes specific color and S, V can be used for illumination and intensity.  \n",
    "        ![](./images/hsv_blob.png)\n",
    "* Extracting the select region from roi using index slicing\n",
    "* Calculating the median H,S,V values from roi\n",
    "* Initializing the lower and upper bound for mask"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (616) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[171. 113. 107.] [181. 213. 207.]\n"
     ]
    }
   ],
   "source": [
    "#Start the webcam in your device\n",
    "#'0' is a special argument to open the webcam\n",
    "video = cv2.VideoCapture(0)\n",
    "\n",
    "#An infinite loop to get each frame of the video\n",
    "while(video.isOpened()):\n",
    "    _, frame = video.read()     #Read the video frame\n",
    "    cv2.imshow(\"Image\",frame)   #Show each frame of the video\n",
    "\n",
    "    if cv2.waitKey(10) == ord('q'):   #If in interval of 10ms 'q' is pressed then execute below\n",
    "        bbox = cv2.selectROI(frame)   #Select the ROI(Region Of Interest)\n",
    "        # selectROI returns a tuple of x,y,h,w\n",
    "        x = bbox[0] # x cordinate of leftmost point in ROI\n",
    "        y = bbox[1] # y cordinate of leftmost point in ROI\n",
    "        w = bbox[2] # width of ROI\n",
    "        h = bbox[3] # height of ROI \n",
    "        \n",
    "        hsv = cv2.cvtColor(frame,cv2.COLOR_BGR2HSV)   #Convert the BGR format to HSV format\n",
    "\n",
    "        obj_img = hsv[ y:y+h , x:x+w ]    #slice the ROI from image\n",
    "\n",
    "        h, s, v = np.median(obj_img[:,:,0]), np.median(obj_img[:,:,1]), np.median(obj_img[:,:,2])     #Taking median of the hsv values in the ROI\n",
    "\n",
    "        lower = np.array([h-5, max(0,s-50),max(0,v-50)])        #The lowerbound values for hsv\n",
    "        upper = np.array([h+5, min(s+50,255),min(v+50,255)])    #The upperbound values for hsv\n",
    "        print(lower,upper)\n",
    "        cv2.destroyAllWindows()\n",
    "        video.release()\n",
    "        \n",
    "        break"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Detecting the blob\n",
    "## Constructing mask for detecting blob\n",
    "* Read the frame from video\n",
    "* Convert from BGR to HSV\n",
    "* Make a mask using cv2.inRange() by passing lower and upper bounds calculated earlier\n",
    "    * What is a mask?  \n",
    "    A mask is a binary image consisting of zero and non-zero values. If a mask is applied to another image of the same size, all pixels which are zero in the mask are set to zero in the output image. All others remain unchanged. \n",
    "   \n",
    "  ![](./images/not_blurredmask.png)\n",
    "* Blur the mask to remove the noise using cv2.medianBlur()  \n",
    "\n",
    "![](./images/blurred_mask.png)\n",
    "* Placing mask over frame to finded colored mask using cv2.bitwise_and()\n",
    "\n",
    "![](./images/colored_mask.png)\n",
    "\n",
    "\n",
    "## Drawing the blob\n",
    "* Find the contour from the generated mask using cv2.findContours()\n",
    "    * What is contour?  \n",
    "    Contours can be explained simply as a curve joining all the continuous points (along the boundary), having same color or intensity. The contours are a useful tool for shape analysis and object detection and recognition\n",
    "\n",
    "    ![](./images/multiple_contours.png) \n",
    "    ![](./images/blob.png)\n",
    "    \n",
    "* Find the contour having the maximum area using cv2.contourArea()\n",
    "* Draw the contour on the frame using cv2.drawContours()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Select a ROI and then press SPACE or ENTER button!\n",
      "Cancel the selection process by pressing c button!\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (616) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created\n"
     ]
    }
   ],
   "source": [
    "video = cv2.VideoCapture(0)\n",
    "while(video.isOpened()):\n",
    "    _, frame = video.read()\n",
    "\n",
    "    hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV) \n",
    "\n",
    "    masked = cv2.inRange(hsv, lower, upper) #Masking the frames in the video for the given range\n",
    "\n",
    "    blur = cv2.medianBlur(masked, 5)        #Applying meadin Blur on the frame\n",
    "\n",
    "    blob_mask = cv2.bitwise_and(frame,frame,mask=blur)  #Using bitwise-and to show only the ROI object\n",
    "\n",
    "    cv2.imshow(\"blob_mask\",blob_mask)                   #Showing the output of bitwise_and\n",
    "\n",
    "    contours, _ = cv2.findContours(blur, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)    #Detect the contours and store them in a list\n",
    "\n",
    "    idx, current_max, counter = 0, 0, 0     #idx = index of contour with maximum area, current_max = Contour with max ares, counter = temporary variable\n",
    "\n",
    "    for n in contours:                      #To find the index of the contour with maximum area\n",
    "        area = cv2.contourArea(n)\n",
    "        if area > current_max:\n",
    "            current_max = area\n",
    "            idx = counter\n",
    "        counter += 1\n",
    "\n",
    "    cv2.drawContours(frame, contours, idx, (0,0,255),2)     #Draw the contours on the region\n",
    "    cv2.imshow(\"Output\",frame)                              #Show the output with the contours\n",
    "\n",
    "    if cv2.waitKey(10) == ord('x'):                         #When 'x' is pressed at an interval of 10 ms, then close all windows\n",
    "        cv2.destroyAllWindows()\n",
    "        video.release()\n",
    "        break"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#complete code\n",
    "import numpy as np \n",
    "import cv2\n",
    "\n",
    "video = cv2.VideoCapture(0)\n",
    "\n",
    "while(video.isOpened()):\n",
    "    _, frame = video.read()\n",
    "    cv2.imshow(\"Image\",frame)\n",
    "\n",
    "    if cv2.waitKey(10) == ord('q'):\n",
    "        bbox = cv2.selectROI(frame)\n",
    "        \n",
    "        hsv = cv2.cvtColor(frame,cv2.COLOR_BGR2HSV)\n",
    "\n",
    "        obj_img = hsv[bbox[1]:bbox[1]+bbox[3],bbox[0]:bbox[0]+bbox[2]]\n",
    "\n",
    "        h, s, v = np.median(obj_img[:,:,0]), np.median(obj_img[:,:,1]), np.median(obj_img[:,:,2])\n",
    "\n",
    "        lower = np.array([h-5, max(0,s-50),max(0,v-50)])\n",
    "        upper = np.array([h+5, min(s+50,255),min(v+50,255)])\n",
    "        print(lower,upper)\n",
    "        break\n",
    "\n",
    "\n",
    "while(video.isOpened()):\n",
    "    _, frame = video.read()\n",
    "\n",
    "    hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)\n",
    "\n",
    "    masked = cv2.inRange(hsv, lower, upper)\n",
    "\n",
    "    blur = cv2.medianBlur(masked, 5)\n",
    "\n",
    "    blob_mask = cv2.bitwise_and(frame,frame,mask=blur)\n",
    "\n",
    "    cv2.imshow(\"blob_mask\",blob_mask)\n",
    "\n",
    "    contours, _ = cv2.findContours(blur, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)\n",
    "\n",
    "    idx, current_max, counter = 0, 0, 0\n",
    "\n",
    "    for n in contours:\n",
    "        area = cv2.contourArea(n)\n",
    "        if area > current_max:\n",
    "            current_max = area\n",
    "            idx = counter\n",
    "        counter += 1\n",
    "\n",
    "    cv2.drawContours(frame, contours, idx, (0,255,255),2)\n",
    "    cv2.imshow(\"Output\",frame)\n",
    "\n",
    "    if cv2.waitKey(10) == ord('x'):\n",
    "        cv2.destroyAllWindows()\n",
    "        video.release()\n",
    "        break\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#complete code for not having webcam\n",
    "import numpy as np \n",
    "import cv2\n",
    "\n",
    "video = cv2.VideoCapture('./video/Blob_detection_video.mp4')\n",
    "\n",
    "\n",
    "_, frame = video.read()\n",
    "cv2.imshow(\"Image\",frame)\n",
    "\n",
    "bbox = cv2.selectROI(frame)\n",
    "\n",
    "hsv = cv2.cvtColor(frame,cv2.COLOR_BGR2HSV)\n",
    "\n",
    "obj_img = hsv[bbox[1]:bbox[1]+bbox[3],bbox[0]:bbox[0]+bbox[2]]\n",
    "\n",
    "h, s, v = np.median(obj_img[:,:,0]), np.median(obj_img[:,:,1]), np.median(obj_img[:,:,2])\n",
    "\n",
    "lower = np.array([h-5, max(0,s-50),max(0,v-50)])\n",
    "upper = np.array([h+5, min(s+50,255),min(v+50,255)])\n",
    "print(lower,upper)\n",
    "\n",
    "\n",
    "\n",
    "while(video.isOpened()):\n",
    "    _, frame = video.read()\n",
    "\n",
    "    hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)\n",
    "\n",
    "    masked = cv2.inRange(hsv, lower, upper)\n",
    "\n",
    "    blur = cv2.medianBlur(masked, 5)\n",
    "\n",
    "    blob_mask = cv2.bitwise_and(frame,frame,mask=blur)\n",
    "\n",
    "    cv2.imshow(\"blob_mask\",blob_mask)\n",
    "\n",
    "    contours, _ = cv2.findContours(blur, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)\n",
    "\n",
    "    idx, current_max, counter = 0, 0, 0\n",
    "\n",
    "    for n in contours:\n",
    "        area = cv2.contourArea(n)\n",
    "        if area > current_max:\n",
    "            current_max = area\n",
    "            idx = counter\n",
    "        counter += 1\n",
    "\n",
    "    cv2.drawContours(frame, contours, idx, (0,255,255),2)\n",
    "    cv2.imshow(\"Output\",frame)\n",
    "\n",
    "    if cv2.waitKey(10) == ord('x'):\n",
    "        cv2.destroyAllWindows()\n",
    "        video.release()\n",
    "        break"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To learn more interesting stuffs of computer vision using opencv [click here](https://docs.opencv.org/master/d9/df8/tutorial_root.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "916dbcbb3f70747c44a77c7bcd40155683ae19c65e1c03b4aa3499c5328201f1"
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.4"
  },
  "metadata": {
   "interpreter": {
    "hash": "1687991aefb5a25a4be79cee9303ce7e804bfd60be2e2a478909a2d6f1c6fd34"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}