Kernel density estimation draws a smooth curve through scattered data without assuming a shape

Kernel density estimation draws a smooth curve through scattered data without assuming a shape
Image: Intellect (generated)

In short: Hand a statistician a scatter of data points with no assumed shape, and kernel density estimation can still expose the pattern hidden inside them. It places a smooth bump on each point and adds the bumps together. The result is a smooth picture of the underlying distribution, and the single setting that matters most is the bandwidth, which controls how wide each bump is.

Kernel density estimation, often abbreviated KDE, is a non-parametric method for estimating the density of a random variable from a finite sample. Non-parametric means the method does not presume the data follow a particular family of curves, such as a normal bell shape. Instead, every observed point contributes a kernel, a non-negative weighting function, and the kernels are summed to give the estimate. In signal processing and econometrics the method is also known as the Parzen-Rosenblatt window method, named for Emanuel Parzen and Murray Rosenblatt, who are generally credited with developing it independently in its present form.

The method is closely related to the histogram. A histogram divides the data range into bins and stacks a box for each point that lands in a bin, producing a stepped picture. A kernel estimate instead centres a smooth curve on each point, so the result can be made smooth or continuous by picking a suitable kernel. Common choices include the uniform, triangular, Epanechnikov and normal kernels. The Epanechnikov kernel is best in a mean square error sense, though the other listed kernels lose only a little efficiency.

The normal kernel is often chosen for its convenient mathematical properties. The bandwidth, a smoothing parameter usually written h, is the decisive choice. A very small bandwidth gives a jagged estimate full of spurious features, known as undersmoothing, while a very large one hides real structure, known as oversmoothing. The best value balances bias against variance, and a common criterion is the mean integrated squared error. Because the ideal bandwidth depends on the unknown density, automatic selectors were developed, and review studies generally favour plug-in and cross-validation selectors across many datasets. A quick shortcut, Silverman's rule of thumb, derives a bandwidth from the sample size and the data's spread. It is reliable only when the density is close to normal and can be badly inaccurate for skewed, heavy-tailed or bimodal data, so it should be used with caution.

Opens the Intellect+ feed with this card first. Swipe up for the next one.

Sources

To cite this card, link to https://intellectplus.co.uk/app/science-kernel-density-estimation.

How we source, check and correct cards: Editorial standards.

General information, not professional advice. Spotted an error? Flag the card or see corrections and complaints.

Intellect Plus is a visual educational discovery platform for exploring fascinating facts and ideas. Brain Food, Not Brain Rot. What is Intellect Plus?

Related

Finding something worth knowing…