Home

Welcome to the ManojKolpeThesis wiki!

Project Timeline

• How to do research

Potential paper topics

• Literature review on temporal fusion
• Efficient semantic segmentation with temporal fusion

Literature papers

• Deep multimodal fusion for semantic image segmentation: A survey - https://www.sciencedirect.com/science/article/abs/pii/S0262885620301748

Semantic segmentation dataset

• Cityscapes Dataset - https://github.com/mcordts/cityscapesScripts
• http://www.scan-net.org/
• Mammography dataset
• NYUv2

The dataset should contain the world, vehicle, and camera coordinates in case of data related to road. But in most cases, the world coordinate is not given. https://www.mathworks.com/help/driving/ug/coordinate-systems.html

Semantic segmentation dataset with camera poses

• https://rll.berkeley.edu/bigbird/access.html
• Create your own labelled dataset https://towardsdatascience.com/custom-instance-segmentation-training-with-7-lines-of-code-ff340851e99b
• https://www.apeer.com/home/
• https://neptune.ai/blog/image-segmentation

Semantic segmentation dataset with sequence data

• Cityscapes Dataset - https://github.com/mcordts/cityscapesScripts
• ScanNet - https://github.com/ScanNet/ScanNet
• Stanford-2D-3D-Semantic dataset - http://buildingparser.stanford.edu/dataset.html
• Create your own dataset

Semantic segmentation dataset RGBD benchmark

• https://github.com/Yangzhangcst/RGBD-semantic-segmentation

Encoder decoder based semantic segmentation model with pretrained weights

• https://github.com/orsic/swiftnet

Accuracy metric for semantic segmentation

• Class accuracy
• Pixel Accuracy (PA)
• Mean Pixel Accuracy (MPA)
• Mean Intersection over Union (MIoU)
• Frequency Weighted Intersection over Union (FWIoU)

Paper+implementation - https://github.com/DeepSceneSeg/SSMA

Gaussian process

• https://www.cs.ubc.ca/~nando/540-2013/lectures.html
• https://www.youtube.com/watch?v=4vGiHC35j9s&list=PLE6Wd9FR--EdyJ5lbFl8UuGjecvVw66F6&index=8

Test output $k^*$is nothing but the input itself where we want to get the updated values

As per the original equation, the mean of the noisy Gaussian regression is given by

where k star transpose is the test point kernel and x star is the test point, however, in our case, the test point is the input point itself. So we multiply the kernel K by the below equation

The above equation can be solved by framing it as AX=B where B = y and A = (K+sigma2*I)

High-Resolution Image Synthesis with Latent Diffusion Models

1Ludwig Maximilian University of Munich & IWR, Heidelberg University, Germany Runway ML https://github.com/CompVis/latent-diffusion