Likelihood is not probability, and confusing them can be disastrous

Likelihood is not probability, and confusing them can be disastrous
Image: Intellect (generated)

In short: Toss a coin twice and get two heads. If the coin is fair, that outcome had a 0.25 chance, so the likelihood of fairness given those tosses is 0.25. It is tempting to conclude there is a 25% chance the coin is fair. That leap is a classic error, the same one behind the prosecutor's fallacy.

A likelihood function scores how well different versions of a statistical model explain the data actually observed. The models usually differ by a parameter, often written as theta, such as the probability that a coin lands heads. The same formula can be read two ways. Hold the parameter fixed and vary the data, and it is a probability distribution over outcomes. Hold the data fixed and vary the parameter, and it becomes the likelihood, a way of comparing candidate parameter values.

The coin shows the difference clearly. Let the heads probability range anywhere from 0 to 1. Two heads in a row has probability 0.25 if the coin is fair, but only 0.09 if the coin lands heads 30% of the time, so the data favour fairness over that bias. Plot the likelihood across every possible value and the area under the curve comes to one third, not one. Likelihoods are not probabilities of the parameter and need not add up to 1.

Turning a likelihood into the probability that a parameter is correct requires extra information: how plausible each value was before seeing the data. That is the job of Bayes' theorem, which produces what Bayesian statisticians call the posterior probability. Without that step, reading a likelihood as the chance a hypothesis is true can have serious consequences, as when a court mistakes the rarity of evidence for the probability of innocence.

In the rival frequentist approach, the parameter is treated as a fixed unknown rather than a random quantity. Its workhorse, maximum likelihood estimation, simply picks the parameter value that makes the observed data most probable, and uses a quantity called the Fisher information to judge how precise that estimate is.

Opens the Intellect+ feed with this card first. Swipe up for the next one.

Sources

To cite this card, link to https://intellectplus.co.uk/app/science-likelihood-function.

How we source, check and correct cards: Editorial standards.

General information, not professional advice. Spotted an error? Flag the card or see corrections and complaints.

Intellect Plus is a visual educational discovery platform for exploring fascinating facts and ideas. Brain Food, Not Brain Rot. What is Intellect Plus?

Related

Finding something worth knowing…