Regression first meant children of tall parents shrink
Francis Galton coined "regression" for heights drifting toward the average. The math toolkit now fits lines and curves between outcomes and predictors. Least squares grew from tracking comets across the sky—yet a fitted link is not, by itself, a cause.
Regression analysis estimates how a dependent outcome relates to one or more independent predictors—also called covariates, features, or explanatory variables. Linear regression is the workhorse: ordinary least squares finds the unique line or hyperplane that minimizes squared gaps between data and fit, estimating the conditional average of the outcome given predictors. Other flavors chase quantiles, nonparametric curves, or different location parameters.
Researchers use regression mainly to forecast—overlapping machine learning—or, with extra justification, to infer causation. On a fixed dataset, regressions only reveal associations; claiming predictive power in a new setting or a causal story from observational data needs careful argument. Newton sketched embryonic linear averaging around 1700 while studying equinoxes. Legendre published least squares in 1805 and Gauss in 1809, both aiming at solar orbits of comets and later minor planets; Gauss developed the theory further in 1821.
Galton's nineteenth-century biological sense—tall ancestors' descendants tending toward a normal average—was generalized by Udny Yule and Karl Pearson under Gaussian joint assumptions. R.A. Fisher in 1922 and 1925 assumed only the conditional response was Gaussian. Mid-century economists ground regressions on electromechanical calculators; before 1970 a single fit could take up to twenty-four hours.
Modern practice still chooses a model form, then estimates unknown parameters amid observed predictors, outcomes, and unobserved errors. Active research covers robust methods, time series, missing data, measurement error, high-dimensional predictors, and causal inference. Spreadsheet and statistical software now finish in seconds what once filled a day—yet the word still whispers Galton's biological retreat to the mean.
Source: Regression analysis
