Training instance segmentation

[marekjaszuk]

Hi,
I'm trying to reproduce Your results in instance segmentation, using the scripts that you delivered (train_maskyolo_step1.sh and 2). I did everything according to the instruction. The scripts work, and produce some loss values. The first phase of training gives me the roi rectangles, that identify the persons, but after the second phase of training I get no result (neither roi or masks). What do you think I could do wrong? Did you get the result using the same scripts?

[leon-liangwu]

@marekjaszuk
Hi. Can I see your log, please?

[marekjaszuk]

Here are the logs:
train_step1.log
train_step2.log

[lucasjinreal]

BTW, the original prototxt 1 and 2 has different num_class:

1：

layer {
  name: "mask_score"
  type: "Convolution"
  bottom: "pool5_2_conv6_relu" #
  top: "mask_score"
  param { lr_mult: 1.0 decay_mult: 1.0 }
  param { lr_mult: 2.0 decay_mult: 0 }
  convolution_param {
    num_output: 18  # 9 affordance classes + 1 background
    kernel_size: 1 pad: 0 
    weight_filler {type: "gaussian" std: 0.01 } #weight_filler { type: "xavier" }
    bias_filler { type: "constant" value: 0 }
  }
}

2：

layer {
  name: "mask_score"
  type: "Convolution"
  bottom: "pool5_2_conv6_relu" #
  top: "mask_score"
  param { lr_mult: 1.0 decay_mult: 1.0 }
  param { lr_mult: 2.0 decay_mult: 0 }
  convolution_param {
    num_output: 2  # 9 affordance classes + 1 background
    kernel_size: 1 pad: 0 
    weight_filler {type: "gaussian" std: 0.01 } #weight_filler { type: "xavier" }
    bias_filler { type: "constant" value: 0 }
  }
}

How do u able to resume weights? (I got error when try to resume step1 trained weights to step2)

[lucasjinreal]

What's the first prototxt 18 stands for? Isn't that only 1 class? What if wanna training on 2 or more classes?

[leon-liangwu]

@marekjaszuk @jinfagang
Yes, thanks. @jinfagang You are right. There is a mistake in step1.prototxt. As we only segment human bodies, the num_output should be set to 2.
@marekjaszuk I have updated the model.tgz, please try agrain.

[marekjaszuk]

The step1 of training went fine, but you refer to pva_solver.prototxt file in the train_maskyolo_step2.sh
Where can I find this .prototxt file?

[lucasjinreal]

@leon-liangwu Thanks for replying.... I have one more question still. You set class to 1, but, in our data, we should have 80 classes stored? How can do specific the only person class as training?
Have u tried training on more than 1 class? What the result would be?

[leon-liangwu]

@jinfagang You can refer to script/createdata_xxx.py. I select all the person targets to create the lmdb.

[leon-liangwu]

@marekjaszuk Yes, just use the solver_step2.prototxt to replace pva_solver.prototxt. That is a mistake. I have updated the model.tgz.
Thanks.

[lucasjinreal]

If change create_dataxxx.py to select multi classes such as car and person, and also edit classes num in prototxt, would that able to work?

[leon-liangwu]

@jinfagang
Yes, sure.

[lucasjinreal]

@leon-liangwu But if using classes more than person, then it should not have keypoints, only box and mask, does it compatible to using kps_data_layer to load data?

[leon-liangwu]

@jinfagang You can use scripts/createdata_mask_only.py this file to generate lmdb with boxes and masks.

[lucasjinreal]

@leon-liangwu I have tried mask training, I can not reproduce your effect. 40000 iterations for step1 and 40000 iterations for step2:

[leon-liangwu]

@jinfagang Actually, you need to change batch_size: 1 in KpsBoxData layer and prop_num: 1 in DecodeRois layer.
Here 'prop_num' is the mounts of proposals. It is usually set to 1batchsize to 2batchsize, depending on the amounts of targets according to your dataset.
I suggest that if you set these two values both as 64, you will get good results.
Also, multi-gpu training is suggested, so you can use bigger batchsize.