When the objective bends, linear solvers no longer suffice
Nonlinear programming hunts maxima or minima when the goal function or the constraints refuse to stay linear. Feasible, infeasible and unbounded cases sort the landscape; Karush–Kuhn–Tucker conditions then say when a candidate can be optimal—and when convexity makes that optimum global.
An optimization problem asks for extrema of a real objective over variables that must also satisfy equalities and inequalities. Nonlinear programming (NLP) is the branch that covers every case that is not purely linear: at least one of the objective or constraint functions curves. In standard form one minimizes f(x) subject to inequality constraints gi(x) ≤ 0, equalities hj(x) = 0, and x inside a set X, usually a box in Rn.
Feasibility language matters. A feasible problem has at least one point meeting every constraint; an infeasible one has contradictory constraints and an empty feasible set. An unbounded feasible problem can drive the objective past any finite bound, so no optimum exists. Modellers treat infeasible or unbounded outcomes as model failures; sometimes they minimize summed constraint violations instead. Special structure helps: convex objectives and convex constraint sets unlock convex-optimization methods; quadratic objectives with linear constraints use quadratic programming; certain concave-over-convex ratios yield to fractional programming. Non-convex costs appear when, for example, shipping petroleum by pipeline, rail, road, barge or coastal tanker mixes economies of scale and capacity limits, even with discontinuous batch costs.
When functions are smooth enough and regularity checks hold, the KKT stationarity rules of Karush, Kuhn, and Tucker become necessary for a local optimum; subdifferential versions handle non-smooth pieces. Convexity makes KKT sufficient for a global optimum; without it they certify only a local one. Analytic KKT solutions are rare, so iterative numerical methods step from an initial guess using zero-order function values, first-order gradients, or second-order Hessians—third-order updates exist in theory but are unused in practice. Branch-and-bound schemes carve the problem into convex or linear lower bounds and can stop at ε-optimal points within a tolerance. Open tools such as SciPy, IPOPT, NLopt and ALGLIB, plus proprietary SNOPT, implement these ideas; spectrum fitting in experimental science is a everyday nonlinear instance when peak heights are unknown.
Werner Fenchel laid foundational work for the field. Toy examples maximize x1 + x2 over an annular region between radii 1 and 2 in the first quadrant, or maximize x1x2 + x2x3 under quadratic ball-like constraints—simple pictures of a vast algorithmic toolkit.
Source: Nonlinear programming
