GNR-652-Project

Contains implementation of various papers that were read as part of the preparation for the problem statement, "Adversarial Inpainting".

Content

Consists of implementation of some GANs. Namely, GAN, DCGAN and WGAN. The implementations are in their respective jupyter notebooks. The GAN is trained on the MNIST dataset, whereas the DCGAN and WGAN are trained on the CelebA dataset.

It also contains the implementation of the EdgeConnect paper as discussed below:

Edge-Connect

The code for training as well as to perform predictions is attached above. I have used a few images from the CelebA dataset for training. The pre-trained weights are also present(after training for about 350 epochs). Only regular masks were used as part of the training. The gram matrix computation function has been taken from the official implementation. The hyperparameter values were also set according to the official implementation. I used a 80:20 split for the training:testing dataset size.

Model Architecture

Above shown is the architecture as well as the working of the model in brief. The model consists of two GANs. The concatenation of the mask, grayscale image(with mask applied) and edge map(with mask applied) is fed into the first GAN, which outputs the completed edge map(which fills in the missing edge in the mask region). This is then concatenated with the original colour image(with mask) and fed into the second GAN which fills in the colour of the image and gives us the completed image.

Results

Since the epochs as well as the number of training images were less, the results are not as good as the official implementation. An example of a batch of predictions is as follows:

Above are the input images with regular masks. Masks are depicted in white.

Next we trained the model on irregular masks(salt and pepper noise) as well. We trained it for 10 epochs on this noise ranging from 10% of the input masked to 60% of it masked(with increments of 10%). Below are the results while testing

At 10%

At 20%

At 30%

At 40%

At 50%

At 60%

At 70%

At 80%

At 90%

As we can see, the de-noising gets better as the percent of pixels masked increases, with the best results at around 50% of the input masked. The accuracy again decreases as the amount of pixels masked increases.