ISSUE!!! connection table 1st layer

[culurciello]

Hi Clement, Yann,
I was going through the tutorials [WELL DONE, GREAT ONES!!!!]

I have a few questions I will email you, if you do not mind:

Q1: on https://github.com/clementfarabet/ipam-tutorials/tree/master/th_tutorials/1_supervised

Why do you have a random connection table on layer 1 here:
-- stage 1 : filter bank -> squashing -> L2 pooling -> normalization
model:add(nn.SpatialConvolutionMap(nn.tables.random(nfeats, nstates[1], fanin[1]), filtsize, filtsize))

I do not understand why not fully connect here? That way you can use all filters for all input planes and combine them into the outputs.
The way it is coded above only forces 2 filters to operate on random input planes to create the 2nd layer

We would get a lot more powerful networks if the fan-in is 4, not 1 as in this case. Also you are now using only 2 filters per plane... is that not TOO LOW?

I understand this way code is faster to execute: less convolution on input planes.... but....

Am I missing something here?

[clementfarabet]

Hey Eugenio,

The random connection on the first layer is not the best thing to do, instead, you might want to define a table of connections by hand. But I like avoiding having full connections from the first layer, it breaks symmetry rapidly. The information is really redundant across input channelsl, so it seems redundant to have fully connectivity...

Clement.