Introduction to Numerical Analysis

In mathematics we usually aim for an exact answer using tools like Algebra, Geometry and Calculus.

But sometimes we can't find an answer that way, or the answer takes too long to find.

Numerical Analysis to the rescue!

Numerical Analysis gives us an approximate answer using simple calculations, and we can often use it again and again getting better answers until the error is small.

Error does not mean a mistake or anything, it is how close we think we are to the true answer. Our goal is to have the lowest error with limited calculation time. Understanding how errors grow or shrink is important!

Numerical Methods

The methods are also called "algorithms" and are like recipes.

Newtons method

One of the famous algorithms is Newton's method.

It is very useful for finding roots (where a function crosses the axis).

We start with a guess x0, then find where that puts us on the curve.

Using that point (x0, f(x0)) and it's slope we find a new point on the axis x1 and that becomes our new guess. Repeat using x1 etc.

It does not always work (sometimes the slope sends us way away) but when it does work it is great!

Example: Newton's Method for √10

Newton's Method finds where a function equals zero (a "root").

So to find x = √10 we can use x2 − 10 = 0

The slope of x2 − 10 is 2x (using deriviatives)

Start with an initial guess
x0 = 3
The point on the curve is:
(3, 32 − 10) = (3, −1)
At x=3 the slope is
2x = 6
Update the guess using
x1 = x0 − y0/slope
x1 = 3 − (−1)/6 = 3.166...

After one round we have √10 ≈ 3.1666...

Our calculator says √10 = 3.16227766..., good so far.

Let's try another round:

Start with
x1 = 3.166...
The point on the curve is:
(3.166..., 3.166...2 − 10) = (3.166..., 0.027...)
At x=3.166... the slope is
2x = 6.33...
Update the guess using
x2 = x1 − y1/slope
x2 = 3.166... − 0.027.../6.3... = 3.16228...

That is really good!

Try one more round yourself and see what you get.

Other Methods

Bisection method

There are many clever and interesting methods. Each has strengths and weaknesses.

Another way to find roots is the Bisection Method, or "binary chop". We start either side of the root, one side with positive y, the other with negative y. We choose a value in between, calculate its value and then reduce the interval, making sure to keep one negative and one positive y value.

Integral Approximation

There are also many methods for doing integrals such as Simpson's Rule (see Integral Approximations and Integral Approximation Calculator).

Also have a play with a simple computer model of reflection inside an ellipse. The errors accumulate but it is still fun to watch, or the single pendulum or double pendulum animation (which also suffers from accumulating errors).

It is actually heaps of fun seeing answers getting better and better (or worse and worse) as the computer works through them.

More complex examples include:

If you become an expert in any of those you will have a good and satisfying career.