Drawing the best straight line through data explains regression toward the mean

Drawing the best straight line through data explains regression toward the mean
Image: Intellect (generated)

In short: Measure something once, then measure it again, and the second result will on average sit closer to the typical value than the first did. That pull, known as regression toward the mean, falls straight out of the humble task of fitting a line through a scatter of points with one predictor.

Simple linear regression looks for the straight line that best predicts one variable from a single other variable, say y from x. Simple refers to that single predictor. The line has two numbers to choose: a slope and an intercept where it crosses the vertical axis. Real data never fall exactly on a line, so each point sits some vertical distance above or below it, and those gaps are called residuals.

The usual rule for choosing the best line is ordinary least squares. Square every residual and add them up, then pick the slope and intercept that make that total as small as possible. Squaring treats misses above and below the line alike and punishes big misses heavily. The minimisation has a tidy closed-form answer, so the best line can be calculated directly rather than searched for.

The result has two elegant properties. The fitted line always passes through the centre of mass of the data, the point given by the average x and average y. And its slope equals the correlation between the two variables, rescaled by the ratio of their standard deviations. If both variables are first converted to standard units, the slope simply becomes the correlation coefficient itself, and the line runs through the origin. With a single predictor, the familiar R squared measure of fit is just that correlation squared.

Because a correlation is always less than one in size unless the relationship is perfect, the standardised prediction for y is always nearer zero than the standardised x. Applied to repeated measurements of the same thing, this means a follow-up reading is expected to land closer to the average than the first one. That is regression toward the mean, and it gave the method its name.

Opens the Intellect+ feed with this card first. Swipe up for the next one.

Sources

To cite this card, link to https://intellectplus.co.uk/app/science-simple-linear-regression.

How we source, check and correct cards: Editorial standards.

General information, not professional advice. Spotted an error? Flag the card or see corrections and complaints.

Intellect Plus is a visual educational discovery platform for exploring fascinating facts and ideas. Brain Food, Not Brain Rot. What is Intellect Plus?

Related

Finding something worth knowing…