The Kolmogorov–Smirnov test judges data by its single biggest gap
How can you tell whether a pile of measurements really follows the curve you expected? One classic answer, devised by two Soviet mathematicians in the 1930s, ignores averages entirely. It lines up the running tally of your data against the expected tally and asks one question: where is the gap between them largest?
The Kolmogorov–Smirnov test, named after Andrey Kolmogorov and Nikolai Smirnov, compares cumulative distributions. For a sample, you build an empirical distribution function: at any value, the fraction of observations at or below it. The test statistic is simply the maximum vertical distance between that staircase and the smooth cumulative curve of a reference distribution. Kolmogorov published the form of the statistic and how it behaves in large samples, while Smirnov supplied a table of its values.
The method comes in two flavours. The one-sample version checks whether data plausibly came from a specified distribution. The two-sample version checks whether two samples came from the same, unspecified distribution, and it is regarded as one of the most useful general tools for comparing samples because it reacts to differences in both location and shape. It makes no assumptions about the form of the underlying distribution, which is what statisticians mean by calling it nonparametric.
Underneath lies some elegant theory. If the sample really comes from the reference distribution, the maximum gap shrinks towards zero as the sample grows, a result known as the Glivenko–Cantelli theorem. Kolmogorov went further by pinning down how fast it shrinks, and the limiting distribution of the scaled gap does not depend on which continuous distribution you started with.
The test has known weaknesses. It generally needs more data than rivals such as the Anderson–Darling test before it can reject a poor fit. If the reference distribution's parameters are estimated from the same data, the standard critical values no longer apply, and corrections such as the Lilliefors test are needed. Even corrected, several studies found it less powerful at detecting non-normal data than the Shapiro–Wilk or Anderson–Darling tests.
Source: Kolmogorov–Smirnov test
