No significant reduction in loss.

[piedcoderr]

Last part of my tiny-yolo-voc-1c.cfg file:

[convolutional]
batch_normalize=1
size=3
stride=1
pad=1
filters=125
activation=leaky

[convolutional]
size=1
stride=1
pad=1
filters=30
activation=linear

[region]
anchors = 1.08,1.19,  3.42,4.41,  6.63,11.38,  9.42,5.11,  16.62,10.52
bias_match=1
classes=1
coords=4
num=5
softmax=1
jitter=.2
rescore=1

object_scale=5
noobject_scale=1
class_scale=1
coord_scale=1

absolute=1
thresh = .5
random=1

Even after i trained for 6 hours on gpu the loss seems to reduce very slowly and it was above 90.
Here is a small sample after epoch 3 and epoch 16:

Right now the training goes like this:

Epoch 3:

step 100 - loss 105.87599182128906 - moving ave loss 105.92378855073041 step 101 - loss 105.90802001953125 - moving ave loss 105.92221169761049 step 102 - loss 105.96070861816406 - moving ave loss 105.92606138966585 step 103 - loss 105.91077423095703 - moving ave loss 105.92453267379497 step 104 - loss 105.89263916015625 - moving ave loss 105.9213433224311 step 105 - loss 105.79179382324219 - moving ave loss 105.9083883725122 step 106 - loss 105.97239685058594 - moving ave loss 105.91478922031959 step 107 - loss 105.93550109863281 - moving ave loss 105.91686040815092 step 108 - loss 105.82689666748047 - moving ave loss 105.90786403408389 step 109 - loss 105.89955139160156 - moving ave loss 105.90703276983565 step 110 - loss 105.97517395019531 - moving ave loss 105.91384688787163 step 111 - loss 105.94683074951172 - moving ave loss 105.91714527403565 Finish 3 epoch(es) step 112 - loss 105.88888549804688 - moving ave loss 105.91431929643677 step 113 - loss 105.77619934082031 - moving ave loss 105.90050730087513 step 114 - loss 106.05404663085938 - moving ave loss 105.91586123387356 step 115 - loss 105.96916198730469 - moving ave loss 105.92119130921667 step 116 - loss 105.88710021972656 - moving ave loss 105.91778220026767 step 117 - loss 105.81925201416016 - moving ave loss 105.90792918165691 step 118 - loss 105.8878173828125 - moving ave loss 105.90591800177248 step 119 - loss 105.89876556396484 - moving ave loss 105.90520275799172 step 120 - loss 105.86831665039062 - moving ave loss 105.90151414723161 step 121 - loss 105.82453918457031 - moving ave loss 105.89381665096549 step 122 - loss 105.72496795654297 - moving ave loss 105.87693178152324 step 123 - loss 105.88580322265625 - moving ave loss 105.87781892563655

Epoch 16:

step 592 - loss 104.63841247558594 - moving ave loss 104.63666866604511 Finish 16 epoch(es) step 593 - loss 104.696533203125 - moving ave loss 104.6426551197531 step 594 - loss 104.6519775390625 - moving ave loss 104.64358736168404 step 595 - loss 104.64488220214844 - moving ave loss 104.64371684573048 step 596 - loss 104.63302612304688 - moving ave loss 104.64264777346212 step 597 - loss 104.54460144042969 - moving ave loss 104.63284314015887 step 598 - loss 104.62289428710938 - moving ave loss 104.63184825485394 step 599 - loss 104.47380065917969 - moving ave loss 104.61604349528652 step 600 - loss 104.67867279052734 - moving ave loss 104.6223064248106 step 601 - loss 104.47022247314453 - moving ave loss 104.60709802964399 step 602 - loss 104.56214904785156 - moving ave loss 104.60260313146475 step 603 - loss 104.53447723388672 - moving ave loss 104.59579054170695 step 604 - loss 104.71623229980469 - moving ave loss 104.60783471751672 step 605 - loss 104.48993682861328 - moving ave loss 104.59604492862638 step 606 - loss 104.55252838134766 - moving ave loss 104.59169327389851 step 607 - loss 104.4090576171875 - moving ave loss 104.57342970822742 step 608 - loss 104.76617431640625 - moving ave loss 104.5927041690453 step 609 - loss 104.65669250488281 - moving ave loss 104.59910300262905 step 610 - loss 104.4439697265625 - moving ave loss 104.5835896750224 step 611 - loss 104.50250244140625 - moving ave loss 104.57548095166078 step 612 - loss 104.66641235351562 - moving ave loss 104.58457409184626 step 613 - loss 104.51042175292969 - moving ave loss 104.57715885795461 step 614 - loss 104.57950592041016 - moving ave loss 104.57739356420016 step 615 - loss 104.4698257446289 - moving ave loss 104.56663678224304 step 616 - loss 104.62855529785156 - moving ave loss 104.5728286338039 step 617 - loss 104.53482818603516 - moving ave loss 104.56902858902703 step 618 - loss 104.41508483886719 - moving ave loss 104.55363421401105 step 619 - loss 104.50767517089844 - moving ave loss 104.5490383096998

So As you can see its taking too much time and their is no significant progress.
I have also posted in stackoverflow #57143339