Meta-Dynamic Neurons improved Spatio-temporal Generalization of Spiking Neural Networks

Spiking Neural Networks (SNNs) have incorporated more biologically-plausible structures and learning principles, hence are playing critical roles in bridging the gap between artificial and natural neural networks. The spikes are the sparse signals described the above-threshold event-based firing and under-threshold dynamic computation of membrane potentials, which give us an alternative uniformed and efficient way on both information representation and computation. Inspired from the biological network, where a finite number of meta neurons integrated together for various of cognitive functions, we proposed and constructed Meta-Dynamic Neurons (MDN) to improve SNNs for the better network generalization on both spatial and temporal tasks. The MDNs are designed with basic neuronal dynamics containing 1st-order and 2nd-order dynamics of membrane potentials, including the spatial and temporal meta types supported by some hyper-parameters. The MDNs generated from a spatial (MNIST) and a temporal (TIDigits) datasets first, and then extended to various other different spatio-temporal tasks (including Fashion-MNIST, NETtalk, Cifar-10, TIMIT and N-MNIST). The comparable accuracy was reached compared to other SOTA SNN algorithms, and a better generalization was also achieved by SNNs using MDNs than that without using MDNs.