Can artificial intelligence learn to think in pulses like the human brain?
In short: Modern AI is incredibly power-hungry, demanding massive electricity and data to function. To solve this, researchers are looking toward Spiking Neural Networks (SNNs). By mimicking the brain's use of discrete electrical pulses rather than continuous numbers, these networks aim to achieve high-speed intelligence with a fraction of the energy.
Traditional deep neural networks (DNNs) are the backbone of modern AI, but they are fundamentally different from biological systems. While the brain uses spikes—discrete electrical signals—to communicate, standard ANNs exchange real-valued signals in discrete time. This makes ANNs highly resource-intensive, requiring high-performance GPUs and vast amounts of annotated data. To combat this, researchers have explored techniques like quantization and pruning, but SNNs represent a third generation of neural networks designed to bridge the gap between neuroscience and machine learning.
The core mechanism of an SNN is the spiking neuron, which integrates inputs over time to increase its membrane potential. A neuron only fires a signal when this potential reaches a specific threshold, similar to how biological neurons propagate action potentials via ions like Na, K, and Cl. One prominent model is the 'leaky integrate-and-fire' model, where the neuron's state decays over time unless pushed higher by incoming spikes. This allows SNNs to embrace 'sparsity'—the biological reality that not every neuron is active at once—making them highly compatible with temporal codes.
A major breakthrough in SNN efficiency involves 'time-to-first-spike' (TTFS) coding. In sensory brain areas, more salient information often leads to earlier spikes, allowing for rapid responses to stimuli. By ensuring each neuron fires at most a single spike, TTFS networks can perform energy-efficient inference with less than one spike per neuron. This is a significant departure from 'rate coding,' where information is conveyed by the frequency of spikes, which can be too slow for tasks requiring millisecond-level processing.
Despite their promise, SNNs are notoriously difficult to train because spike operations are non-differentiable, making standard backpropagation complex to apply. However, new methods use surrogate gradients or translate ANN activations into spike counts to train these models. SNNs are also being utilised to investigate intricate neurological puzzles like 'binding by polychrony.' This concept posits that neurons discharge in exact, synchronized, time-aligned patterns to create the unified objects of our perception, a process that might ultimately bolster the reliability of computer vision technologies.
Opens the Intellect+ feed with this card first. Swipe up for the next one.
Sources
- Summary by Intellect · source: Spiking Neural Networks and Their Applications: A Review
- Also: High-performance deep spiking neural networks with 0.3 spikes per neuron - Nature Communications
- Also: Spiking neural network
To cite this card, link to https://intellectplus.co.uk/app/health-spiking-neural-network.
How we source, check and correct cards: Editorial standards.
General information, not professional advice. Spotted an error? Flag the card or see corrections and complaints.
Intellect Plus is a visual educational discovery platform for exploring fascinating facts and ideas. Brain Food, Not Brain Rot. What is Intellect Plus?
Related
More in Memory · More in Health · Latest · A–Z · All topics
