Before you see the data, what do you already believe about the world?
In short: In Bayesian statistics, a prior probability is your starting assumption before new evidence arrives. Whether based on expert opinion, past experiments, or pure indifference, these distributions act as the foundation for your final conclusions. Understanding how we weight these initial beliefs is the key to mastering statistical inference.
A prior probability distribution represents the assumed state of an uncertain quantity before incorporating new data. In Bayesian analysis, Bayes' rule allows us to update this prior into a posterior distribution once evidence is observed. While historically researchers were limited to conjugate families of functions for mathematical convenience, modern computational methods like Markov chain Monte Carlo have removed these constraints, allowing for more flexible modeling.
Priors are categorized by how much information they contain. Informative priors, such as using today's temperature to predict tomorrow's, rely on specific, definite evidence. Weakly informative priors, like setting a broad range for a temperature in St. Louis, provide regularization to keep estimates within reasonable bounds. Conversely, uninformative or diffuse priors express vague information. The principle of indifference, which assigns equal probability to all possibilities, is the oldest method for constructing such priors, though it remains a subject of philosophical debate.
The field is divided between subjective Bayesians, who view priors as personal judgements, and objective Bayesians, who seek priors logically required by a state of uncertainty. Figures like Edwin T. Jaynes championed objective approaches based on symmetry and maximum entropy. Jaynes argued that if a problem remains unchanged by swapping labels—like a ball hidden under one of three cups—a uniform prior is the only logical choice. Other sophisticated techniques, such as reference priors introduced by José-Miguel Bernardo, aim to maximize the information gained from data by minimizing the prior's influence in the asymptotic limit.
Ultimately, the distinction between prior and posterior is relative to the specific observation. As evidence accumulates, the posterior from one problem often becomes the prior for the next. Provided the initial assumption allows for the possibility of the evidence, the final result will eventually be dominated by the data rather than the original starting point.
Opens the Intellect+ feed with this card first. Swipe up for the next one.
Sources
- Summary by Intellect · source: Prior probability
To cite this card, link to https://intellectplus.co.uk/app/science-prior-probability.
How we source, check and correct cards: Editorial standards.
General information, not professional advice. Spotted an error? Flag the card or see corrections and complaints.
Intellect Plus is a visual educational discovery platform for exploring fascinating facts and ideas. Brain Food, Not Brain Rot. What is Intellect Plus?
Related
More in Statistics and probability · More in Science · Latest · A–Z · All topics