One hot oven can wreck the average temperature of a room
Measure ten objects in a room: nine sit between 20 and 25 degrees Celsius, and one is an oven at 175. The median stays sensibly between 20 and 25, but the mean jumps to somewhere between 35.5 and 40, a temperature nothing in the room actually has.
That oven is an outlier, a data point far from the rest. It might reflect natural variability, a genuine discovery, or a mistake, and only the last is a good reason to throw it out. Causes include a momentary instrument fault, a transcription slip, fraud, human error, a change in how a system behaves, or plain natural variation. Outliers can also signal that a population has heavy tails, in which case extreme values are part of the story rather than noise.
Some unusual values are simply expected. With normally distributed data, roughly one observation in 22 lands two or more standard deviations from the mean, and about one in 370 lands three or more away. In a sample of 1000, finding up to five of those extreme points is no cause for alarm. The most extreme values in a sample are not automatically outliers either, if they sit close to their neighbours.
Awkwardly, there is no strict mathematical definition. Deciding whether a point counts is ultimately a judgement call. Detection methods range from graphical ones, such as probability plots, to model-based tests that flag improbable values under a normal distribution, with box plots somewhere in between. Regression modellers lean on measures such as Mahalanobis distance and leverage, while anomaly detection in finance, networking and data mining may rely on density or on distances to nearest neighbours.
One procedure, the modified Thompson Tau test, takes a data set's mean and standard deviation and computes a statistically based rejection zone. It removes one suspect at a time, recalculates the average and zone, and repeats until nothing falls outside. Because outliers can distort results so badly, statisticians value robust estimators that shrug them off; the median is one, the mean is not.
Source: Outlier
