Why minimizing different norms changes the point you find on a line
In short: When solving linear equations with constraints, the choice of norm—L1, L2, or L-infinity—drastically shifts your result. By visualizing these constraints in two dimensions, we can see how geometric shapes like circles interact with lines to reveal the optimal point for each mathematical standard.
In the context of linear systems where Ax equals b, we often face the challenge of finding a solution that satisfies a specific constraint. Professor Gilbert Strang of MIT highlights that when we minimize different norms—such as the L1, L2, or L-infinity norms—we are essentially looking for the point on a constraint line that is closest to the origin under that specific metric. This process is fundamental to data analysis and machine learning, where selecting the right norm can change the nature of the solution found.
To visualize this, consider a constraint represented as a line in a 2D plane. For the standard L2 norm, the set of points with a constant norm forms a circle. To minimize the L2 norm for a point on the line, we expand or shrink this circle until it just touches the line. Geometrically, the optimal point occurs where the radius of the circle is perpendicular to the constraint line. Any other point on the line would necessarily fall outside this circle, meaning it would have a larger L2 norm.
This geometric approach illustrates why different norms yield different results. While the L2 norm relies on the properties of circles and perpendicularity, other norms like L1 or L-infinity create different shapes in the plane, such as diamonds or squares. These shapes touch the constraint line at different locations, leading to distinct optimal points. Understanding these interactions is essential for anyone working with matrix methods, as the choice of norm dictates which solution is prioritized in the presence of nearly dependent columns or large datasets.
Beyond the theory, the practical application of these methods often involves iterative refinement. Professor Strang notes that developing the underlying mathematical theory is a labor-intensive process, often taking years of drafting, scanning, and revising to ensure the concepts are clearly articulated. This iterative cycle of improvement mirrors the way we refine our mathematical models to better fit the constraints of real-world data.
Opens the Intellect+ feed with this card first. Swipe up for the next one.
Sources
- Summary by Intellect · source: Lecture 11: Minimizing ‖x‖ Subject to Ax = b
- Also: Lecture 11: Minimizing ‖x‖ Subject to Ax = b | Matrix Methods in Data Analysis, Signal Processing, and Machine Learning | Mathematics | MIT OpenCourseWare
To cite this card, link to https://intellectplus.co.uk/app/ideas-lecture-11-minimizing-x-subject-to-ax-b.
How we source, check and correct cards: Editorial standards.
General information, not professional advice. Spotted an error? Flag the card or see corrections and complaints.
Intellect Plus is a visual educational discovery platform for exploring fascinating facts and ideas. Brain Food, Not Brain Rot. What is Intellect Plus?
Related
More in Mathematics · More in Ideas & Philosophy · Latest · A–Z · All topics
