Reproduce 2D miou on scannet dataset

[OPradelle]

Hello, thanks for sharing your code.

I wanted to reproduce the 2D miou score given in your article (61.5 miou for the 2D only) but i did not manage to go over 56.2 miou.

I used your preprocessing script for 2d datas, a resnet34 similar to yours (using conv2dTranspose over F.interpolate) and when i tried using the same data augmentation (random crop on image with gaussian blur on training set), the network miou score worsen.

Did i miss something ?
Thanks for your answer

[xuxiaoxxxx]

Hi, do you solve this problem? Can you share the code of 2D only with me? I try to reproduce the miou but the result is worsen.

[OPradelle]

Hi, i did not manage to reproduce the result given in the paper (61.5 mIoU) for the validation set on 2D only based methods.
I used the resnet34 given in models/resnet.py and write a custom dataset using dataset/scanNetCross.py by removing the link creator and 3D load from the getitem method. I updated similarly the train.py file to only create a resnet and apply loss on the 2D part.
I don't have this code anymore but i can help you if you have any trouble.

[xuxiaoxxxx]

Thank you very much ! I have solve this problem.

[xuxiaoxxxx]

Hi, i did not manage to reproduce the result given in the paper (61.5 mIoU) for the validation set on 2D only based methods. I used the resnet34 given in models/resnet.py and write a custom dataset using dataset/scanNetCross.py by removing the link creator and 3D load from the getitem method. I updated similarly the train.py file to only create a resnet and apply loss on the 2D part. I don't have this code anymore but i can help you if you have any trouble.

Hi, I have an another question. After reading the code, I found that the camera intrinsics matrix had not changed.

Can you tell me why the camera intrinsics matrix had not changed?

[OPradelle]

Hi, when i dealt with both modalities (3d and 2d), I did not change anything on the authors code.
I do not know why they did it like this, especially for the ScanNet dataset where there is multiple camera...

[yhyang-myron]

@OPradelle @xuxiaoxxxx Hi, do you use the 2d label from _2d-label-filt.zip file? Why did I only get 26 miou for 2D with the pretrained model ? But I got 70.6 miou for 3D just like the performance in the paper.