Can a machine learn like a brain by mimicking physics?
In short: Standard AI learns by calculating gradients through layers of neurons, like a chain of gears. But a different approach suggests the brain might learn using local information and physical principles. By treating neural connections like energy states in a physical system, Boltzmann machines offer a glimpse into a more biological model of intelligence.
Boltzmann machines are a type of stochastic neural network named after Ludwig Boltzmann. Unlike standard networks that use backpropagation to calculate performance gradients, these models are inspired by statistical mechanics and spin-glass models. They treat neurons as units that can be active or inactive, with weighted couplings between them. The goal is to arrange these weights so that training data—such as images—become local minima of an energy function. However, simple pairwise correlations are often insufficient to capture complex patterns, leading to the inclusion of 'hidden' neurons to represent higher-order correlations.
The mechanism relies on the Boltzmann distribution, where the probability of a neuron being active is tied to the system's energy state. In this framework, neurons are asynchronous and stochastic; they adopt states based on the difference in energy between being active and inactive. This allowed for a 'proof of principle' that neural networks with hidden neurons could learn using only locally available information. This was a significant departure from the prevailing belief at the time, which suggested that global information was necessary for such learning.
While theoretically intriguing due to their parallelism and resemblance to physical processes, unconstrained Boltzmann machines have not been proven useful for practical problems. Efficiency in machine learning only occurs when the connectivity is properly constrained. These models are often categorized as energy-based models (EBMs), where the Hamiltonian of a spin glass serves as the starting point for defining the learning task. They represent a bridge between cognitive science and statistical physics, exploring how local rules can give rise to complex global representations.
Recent research has even explored using quantum annealing to sample from these distributions. Studies comparing D-Wave quantum annealers to classical Gibbs sampling found that while both techniques find different local minima, they often overlap on high-probability states. The results suggest that while quantum approaches don't always provide a significant leap in sampling quality, there remains potential for combined classical-quantum approaches to improve how these machines learn.
Opens the Intellect+ feed with this card first. Swipe up for the next one.
Sources
- Summary by Intellect · source: Professor Geoffrey Hinton Distinguished Lecture on Boltzmann Machines | School of Informatics | School of Informatics
- Also: Boltzmann machine
To cite this card, link to https://intellectplus.co.uk/app/health-boltzmann-machine.
How we source, check and correct cards: Editorial standards.
General information, not professional advice. Spotted an error? Flag the card or see corrections and complaints.
Intellect Plus is a visual educational discovery platform for exploring fascinating facts and ideas. Brain Food, Not Brain Rot. What is Intellect Plus?
