Gram–Schmidt straightens a tilted set of arrows into perpendicular ones
Hand the Gram–Schmidt process a set of arrows pointing in awkward, overlapping directions, and it returns a tidy set at right angles to each other, spanning exactly the same space. Its name honours the Danish mathematician Jørgen Pedersen Gram and Erhard Schmidt, though Pierre-Simon Laplace knew the method before either of them.
The idea is simple enough to do by hand. Keep the first arrow as it is. For the second, work out its shadow along the first arrow, the part that points the same way, and subtract it. What remains sticks out at a right angle. For the third, subtract its shadows along both earlier results, and so on. Each new arrow ends up perpendicular to every arrow already built.
If you also shrink or stretch each result to length one, you get what mathematicians call an orthonormal set, and the process is named orthonormalization instead of orthogonalization. A proof that every output really is perpendicular to the rest goes step by step, using mathematical induction.
Feed in arrows that are not truly independent, where one is already a combination of earlier ones, and at that step the process produces a zero arrow, since nothing is left after the shadows are removed. A zero arrow cannot be stretched to length one, so a careful implementation checks for these and throws them away. The number that survive then equals the true dimension of the space the inputs span.
The method reaches well beyond hand calculation. Applied to the columns of a matrix, it splits the matrix into an orthogonal part and a triangular part, known as the QR decomposition, a workhorse of numerical computing. It also extends to infinite sequences of vectors, and in the theory of Lie groups it is generalised by the Iwasawa decomposition.
Source: Gram–Schmidt process
