Newton's method doubles your correct digits with every step
Want the square root of 2 to twenty decimal places? Make a rough guess, draw the tangent line to the curve at that guess, and see where the line hits zero. That point is a far better guess. Repeat, and near the answer each round roughly doubles the number of correct digits.
The Newton–Raphson method hunts for the roots of a function, the inputs where it equals zero. Its trick is to replace the curve, near your current guess, with its tangent line, whose root is trivial to find. The new estimate is the old one minus the function's value divided by its slope. When the starting guess is close and the root is a simple one, convergence is quadratic: the error is squared each time, which is why the digits pile up so fast.
The idea is far older than Newton. Babylonian scribes, between the 19th and 16th centuries BCE, are thought to have approximated square roots with a special case of it, and the same procedure appears in Hero of Alexandria, hence the name Heron's method. In 1427 the Persian mathematician Jamshīd al-Kāshī published an equivalent technique for extracting roots, building on al-Bīrūnī and Sharaf al-Dīn al-Ṭūsī, and in Japan Seki Kōwa used a version in the 1680s.
Newton's own version, written in 1669, looked quite different. He worked only with polynomials, repeatedly rewriting them in terms of the leftover error, and never tied the process to derivatives or gave a general formula. John Wallis first published it in 1685. In 1690 Joseph Raphson simplified the bookkeeping by drawing every correction from the original polynomial, and in 1740 Thomas Simpson finally described it the modern way, using calculus, extending it to pairs of equations and to optimisation problems.
It has weaknesses. You need the derivative, which may be hard or costly to compute; approximating it with a line through two nearby points gives the slower secant method. Bad starting points can send the iterations astray. In 1879 Arthur Cayley noticed how strangely the method behaves when hunting complex roots of polynomials above degree two, a puzzle that opened up the study of iterating rational functions.
Source: Newton's method
