Object-Detections-and-Text-Recognitions by Using YoloV3 and XGBoost

Objects Detection and Text Recognitions

In this projects, I am trying to use yolov3 model to detect the targets(text) and recognition it.
Training of yolov3 , https://www.youtube.com/watch?v=_FNfRtXEbr4&t=1421s as my references.

Steps of processing :

1. YoloV3 : Find out the bounding boxes of target and characters
   
2. HOG : To get features of Characters
   
3. XGBoost : Model used to classify
   
4. Join the results
   

Problems during processing :

1. Some of original images are vertical flip.
   1.1 We have to develop a methods to detect.
   1.2 By observation,we found that most of the images are in format 'XXXXX XXX XX' or 'XXXXX XXXX' or 'XXXXXXXXX' where X are Arabic numerals [0-9] or captital letter [A-Z but not included O and I].
   1.3 Detect the sides(left or right) which contains most of X.
   If num(left characters) > num(right characters) : Normal case
   Else : Vertical Flip
   

2.There are some characters with less training dataset.
2.1 Create more training data for those characters by adding Gaussian Noises / Random Chopping / Random Brightness Adjustment
2.2 Total Training Images : ~ 139k with 34 classes

• Results of Detections:
  

• Results of Cropping :
  

• Accuracy of Classifications of Chars :
  acc = 0.9602 - 0.984
  

• Confusion Matirx (Because of diagonal elements are large,I've changed them into 99 for easier understanding results ) :
  
  
  

• Results of Detections and Recognitions :
  

Numbers of unable to be detected by YoloV3 (with total numbers : 9915):

UnFlipped Case :

Flipped Case :

Failed Case: