CapsNet

CapsNet[https://hackernoon.com/what-is-a-capsnet-or-capsule-network-2bfbe48769cc]

Convolutional Neural Networks are Doomed:

1. Due to Subsampling or pooling - spatial correlation between images and pixels is lost.
2. Convolution networks are bad if the image is in any different position, rotated or upside down -- thus need for data augmentation as proposed by AlexNet
3. invariance vs equivariance
4. The goal of subsampling or pooling was to make the network invariant to small changes in the spatial locality.
5. better to aim for equivariance - if we rotate the image, the network should learn the change

Capsule Network

1. Idea: Human brain should achieve translation invariance in a better way i.e. better than pooling
2. Brain has different modules - called as Capsules which can handle a different kind of stimulus
3. In CNN, routing through the network is done through pooling - we have Convolution, Non-Linearity and the routing through pooling
4. Better way to route data -
5. Basic Idea: In place of adding another layer, lets nest another layer within a layer. That nested layer is called Capsule - which is a group of neurons
6. Thus instead of making network deeper, make the network deep in terms of nesting or inner structure.
7. The Capsule based model is more robust to translation and rotation.

Features of Capsule Network

• Layer based Squashing:

1. In typical NN, only output is squashed using ReLU or non-linear layer.
2. In place applying Non-Linearity to a single neuron we can group the neurons into a capsule and apply Non-Linearity on a capsule

• Dynamic Routing

1. Replace scalar output feature detectors with vector output capsules
2. Replaces MaxPooling by Routing by Agreement