The Hessian matrix tells a peak from a valley or a saddle

The Hessian matrix tells a peak from a valley or a saddle
Image: Intellect (generated)

In short: At the top of a hill, the bottom of a bowl and the middle of a mountain pass, the ground is momentarily flat. Slope alone cannot tell these apart. The Hessian matrix, a grid of second derivatives, measures how the surface curves in every direction and so reveals which kind of flat spot you are standing on.

For a function of several variables, the Hessian collects all its second-order partial derivatives into a square array describing local curvature. The German mathematician Ludwig Otto Hesse developed it in the 19th century, calling it a functional determinant; the modern name honours him. When those second derivatives are continuous, the matrix is symmetric.

Its best-known job is the second-derivative test. At a critical point where the gradient vanishes, a positive-definite Hessian signals a local minimum, a negative-definite one a local maximum, and mixed signs a saddle. If it is only semidefinite, the test says nothing. With two variables the determinant does the work, since it is the product of the two eigenvalues: positive means both share a sign, negative means they differ. Geometrically, the eigenvalues are the principal curvatures, the eigenvectors point along the principal directions, and the determinant equals the Gaussian curvature. Morse theory and catastrophe theory rely on it to classify critical points.

It also surfaces in geometry of curves. On a plane projective curve, a smooth point is an inflection point precisely when the determinant of the Hessian there equals zero, and Bézout's theorem then limits a cubic curve to at most 9 inflection points.

In optimisation, Newton-type methods use the Hessian because it supplies the quadratic term of a function's local Taylor expansion. The catch is size: storing it becomes infeasible for functions with huge numbers of parameters, such as neural network loss functions. Truncated-Newton and quasi-Newton algorithms get around this by approximating it, and one of the most popular quasi-Newton methods is BFGS.

Opens the Intellect+ feed with this card first. Swipe up for the next one.

Sources

To cite this card, link to https://intellectplus.co.uk/app/science-hessian-matrix.

How we source, check and correct cards: Editorial standards.

General information, not professional advice. Spotted an error? Flag the card or see corrections and complaints.

Intellect Plus is a visual educational discovery platform for exploring fascinating facts and ideas. Brain Food, Not Brain Rot. What is Intellect Plus?

Related

Finding something worth knowing…