Spatial Pooler: separate Inhibition, Topology, Boosting into standalone classes

[breznak]

• Boosting
• Topology
• Inhibition

After #108

[ctrl-z-9000-times]

I think that all of these changes would break API compatibility. That doesnt mean it can't be done, just that it wont be easy.

Another way is subclasses which over ride certain methods. Its not as flexible of a solution but it seems simpler and less disruptive.

[breznak]

Another way is subclasses

Yes, those classes would not be visible to the public API and SP would call them.

AbstractInhibition
InhibitionLocal
InhibitionGlobal
SP { Inhibition = InhibitionLocal }

something like this.

[ctrl-z-9000-times]

If Boosting was refactored into its own class, I would add this boosting function:

• boost-factor = log(activation-frequency) / log(target-sparsity)

I like this function because it has a zero-crossing at activation-frequency = 1 and an asymptote to infinity at activation-frequency = 0.

[breznak]

That sounds even like a good default, eventhough we'd have to do some tests.

Another motivation for this modularity: while testing performance impact of PR #119 , I figured there are no tests for using local inhibition, at all! The separate class would make that more obvious.

I was thinking about your comment how to make this less intrusive, API-wise.
SpatialPooler(..., inhibition*=nullptr)
would allow us to create instance of Inhibition and provide it to override the default.

[breznak]

@ctrl-z-9000-times now that David is working on pybind, do you want to join me here and then to #93?
I figured we should proceed here first as it separates the SP into smaller logical blocks, where none of Boosting,Inhibition,Topology are used in TM, therefore shielding them will make the port simpler.

[ctrl-z-9000-times]

I would rather work on #93 first. It seems more straightforward than this one.

[ctrl-z-9000-times]

Here are my thoughts on each of these topics:

Boosting: I would like to use the logarithmic boosting function as a default. Maybe we could sneak this in by turning it on when boosting-strength == -1 ?

Topology: Conceptually, this is the function topology( mini-column-location ) -> potential-pool. The SP could keep a pointer to this function, with setter/getter methods, to allow other ppl to hook in their own topology. This allows the defaults to remain, and for new functions to be added. Python function callbacks can be added.

I would add the topology function:

Topology( X )
    RandomSample( num = potential_pool_size
    data = for Input in InputSpace:
                   Normal(loc = Input, mean = X, Std = Radius)

This is the simpler 1-D case. The N-Dimensional extension of this function samples from the product of the normal-distributions in all dimensions. This has the feature of rotational symmetry, which is nice.

Update: I tried this topology function and it didn't perform as well as the one that's already in the spatial pooler. I tested it on the MNIST dataset.

Inhibition: This is a core part of the SP, and I don't think its going to be easy to change them without modifying the whole class. Local inhibition currently is a lot of the code in the SP, and my solution also changes the API.

[ctrl-z-9000-times]

@breznak ,
I have a proposal for how to implement topology.

We should change the private method:
vector<UInt> SpatialPooler::initMapPotential_(UInt column, bool wrapAround);

Into protected method:
virtual SpatialPooler::initMapPotential( SDR &column, SDR &potentialPool );

Subclasses of SpatialPooler can then override the default topology.

• Attributes: potentialRadius_, potentialPct_, and wrapAround_ will all be made protected so that subclasses can use them if they want to.
• WrapAround is not an argument any more, since it's specific to this implementation of topology.
• Both the mini-column index and the input index are changed into SDRs for convenience.
• No API changes, only additions.