Say you’re in a competition and you need to calculate the square root of two to several decimal places. Or perhaps you need to approximate the roots to an unwieldy polynomial like x^3+7x+2 . Newton’s method can be used in these situations.
Newton’s method. The Taylor expansion of f(x) around a is \displaystyle f(x)=f(a)+f^{(1)}(a)(x-a)+\frac{1}{2}f^{(2)}(x-a)^2 +\cdots . Taking the linear approximation, we have \displaystyle f(x)\approx f(a)+f'(a)(x-a). Set f(x)=0 (because we are finding the roots of f), and the approximation above becomes \displaystyle 0= f(a)+f'(a)(x-a)\implies x=a-\frac{f(a)}{f'(a)}.
Next we plug in an initial estimate for the root (this number we just make up and hope it is close to the root) for a and call it x_0. So now we have a fair approximation \displaystyle x_1=x_0-\frac{f(x_0)}{f'(x_0)}. To get a better approximation, we plug x_1 back into the equation in the place of x_0. In general, we can get an approximation x_n that is better than x_{n-1} by \displaystyle x_n=x_{n-1}-\frac{f(x_{n-1})}{f'(x_{n-1})}.
Example 1. Approximate \sqrt{3}.
This is the same as solving the equation \displaystyle f(x)=x^2-3=0. Since \displaystyle f'(x)=2x, and we make our initial approximation x_0=2, we have \displaystyle x_1=2-\frac{1}{4}=\frac{7}{4}=1.75, whereas \sqrt{3}\approx 1.732. That’s already an excellent estimate, but let’s keep on going until we are accurate to the thousandths place.
\displaystyle x_2=\frac{7}{4}-\frac{1/16}{7/2}=\frac{97}{56}\approx 1.732.Incredible!
Example 2. Estimate the roots of x^3+7x+2.
f(0)=2 is pretty close to 0, so we set x_0=2. Also, \displaystyle f'(x)=3x^2+7. \displaystyle x_1=0-\frac{2}{7}\approx -0.286. \displaystyle x_2=-\frac{2}{7}-\frac{-8/343}{355/49}=-\frac{702}{2485}\approx -0.28249. Indeed, one of the roots of x^3+7x+2 is -0.28249. But after guessing more values for x_0 to find the other roots, you will be hard-pressed to find any that result in an f(x) value that is close to 0. This is because x^3+7x+2 only has one real root. In a later post, I will explain how to determine the number of real solutions to a polynomial.
By the way, Newton was born on Christmas (sort of).