Drawing the best straight line through data explains regression toward the mean
Measure something once, then measure it again, and the second result will on average sit closer to the typical value than the first did. That pull, known as regression toward the mean, falls straight out of the humble task of fitting a line through a scatter of points with one predictor.
Simple linear regression looks for the straight line that best predicts one variable from a single other variable, say y from x. Simple refers to that single predictor. The line has two numbers to choose: a slope and an intercept where it crosses the vertical axis. Real data never fall exactly on a line, so each point sits some vertical distance above or below it, and those gaps are called residuals.
The usual rule for choosing the best line is ordinary least squares. Square every residual and add them up, then pick the slope and intercept that make that total as small as possible. Squaring treats misses above and below the line alike and punishes big misses heavily. The minimisation has a tidy closed-form answer, so the best line can be calculated directly rather than searched for.
The result has two elegant properties. The fitted line always passes through the centre of mass of the data, the point given by the average x and average y. And its slope equals the correlation between the two variables, rescaled by the ratio of their standard deviations. If both variables are first converted to standard units, the slope simply becomes the correlation coefficient itself, and the line runs through the origin. With a single predictor, the familiar R squared measure of fit is just that correlation squared.
Because a correlation is always less than one in size unless the relationship is perfect, the standardised prediction for y is always nearer zero than the standardised x. Applied to repeated measurements of the same thing, this means a follow-up reading is expected to land closer to the average than the first one. That is regression toward the mean, and it gave the method its name.
Source: Simple linear regression
