Spiking-Deep-Belief-Network

Abstract: Deep learning as a set of algorithms in Machine Learning, has produced state-of-the-art results and their efficiency in various type of applications has been demonstrated. In a biological neural network, both of memory and computational elements are integrated in the body of each neuron. Because of the physical distance between the memory elements and the arithmetic modules in Von-Neumann architecture, implementing deep architectures in general-purpose hardware, despite its accuracy, ignores this biological aspect of neurons. Therefore implementing deep learning algorithms in a specific-purpose hardware with biological inspired neurons not only can utilizes the robustness of deep learning models, but also because of using local analog computation can demonstrate speedup and power efficiency advantages relative to digital systems. Memristor as a new nano-device provides big opportunity for implementing low power and dense circuits. Implementing biological inspired model of neurons and synaptic connections using Memristor, has proven the efficiency of Crossbars of Memristors in hardware implementation of Spiking Neural Networks. In this dissertation considering the characteristics of Leaky Integrated-and-Fire (LIF) model as the most popular model of spiking neurons, we have tried to find a method for utilizing Contrastive Divergence (CD) in a network of LIF neurons. There are some problems with using Machine Learning algorithms in spiking model of neural networks. The most challenge with implementing Machine Learning methods using biological inspired model of neurons, is because of their different approaches for updating the synaptic weights. STDP is the most popular method for Spiking Neural Networks (SNN) which uses the spike timing aspect of neural coding, but in this work we provide a framework to use the proposed rate based version of Contrastive Divergence updating weight rule. Respecting to the rate aspect of neural coding, we developed a Spiking Restricted Boltzmann Machine (Spiking RBM) and stacking several RBMs we proposed a Spike-Based Deep Belief Network (S-DBN) with Leaky Integrate-and-Fire neurons. The proposed framework was evaluated in handwritten digit (MNIST) recognition application task (94.9%). Spike-Based Deep Belief Network (S-DBN), has smoothed the way for utilizing Memristor in a specific hardware implementation of a Deep Spiking Neural Network and has provided a suitable framework to utilizing in deep architectures in a desired Neuromorphic Hardware Accelerator.