Biologically-informed excitatory and inhibitory ratio for robust spiking neural network training

Abstract

To further the understanding of training SNNs with progressively more biological realism, this work looks directly into the ratios of excitatory and inhibitory neurons and corresponding activity within a network being trained. In various regions of the mammalian brain, the ratio between excitatory and inhibitory neurons varies between approximately 10:90 to 90:10 with 80:20 being the approximate excitatory:inhibitory (E:I) ratio across the adult mouse brain as a whole. Our work investigates networks within this range relevant to rodent brain regions and analyzes their spiking activity. Additionally, a variety of network initial conditions were trained on two AI-relevant datasets, namely the Fashion-MNIST (Modified National Institute of Standards and Technology) image dataset and Spiking Heidelberg Digits (SHD) audio dataset. Overall, the analysis here provides a deeper look at the relationship between spiking behavior and network training performance. While previous work has relied on a mathematical derivation for optimal initialization, in this work we look for a more general activity-based solution that is potentially applicable beyond leaky-integrate-and-fire (LIF) models. This solution provides some guidelines for increased training robustness and efficiency of relevance to spiking neural networks towards biologically realistic implementations.

Results

Non-noisy networks using the range of initial conditions, i.e. weight distributions and random seeds, trained on Fashion-MNIST were able to train to over 80% accuracy, which is similar to networks of the same size with unbounded weights (Fig. 1c). Accuracy is shown as peak accuracy measured across the 30 epochs of training and we observed no overfitting in the trials we have measured. For networks with E:I ratios 80:20, 95:5, and 100:0, the best performing networks were initialized with weights distributions generating initial firing rates within the biologically-realistic bounds between 0.01 Hz and 25.6 Hz, leading to an energy-efficient implementation while still retaining a high accuracy performance. Networks with a high percentage of inhibitory neurons, e.g. 50:50, were only able to train at the lowest end of the tested activity range, but the accuracy performance was not robust across the 8 repeat trials with different random seeds for the same distributions. In contrast, 100:0 networks exhibited high accuracy (over 80%) at initial firing rates over 50Hz. However, these upper activity levels would not be energy efficient due to unnecessarily high spiking activity levels, nor do they confer additional accuracy benefits. Therefore, we focused on the lower activity spectrum for further analysis.

source: https://www.nature.com/articles/s41598-025-03408-7

#SNNs #spikingNetwork #inhibitory #ratio #informedModels #spikingNeuralNetwork