Contact Us View Cart

STEM Articles

An Introduction to Numerical Methods

Introduction

Analytical solutions are the only methods taught to solve math problems until students take upper-level calculus classes.  A simple example of an analytical solution is the equation for a line: y=mx+b.  You can see that in an analytical solution, all variables of the system are accounted for at the same time which leads to large and complex equations.

  For example, the analytical solution for x2 + x + 5 = 15 is:

The numerical solution is:

Note that the numerical solution is an approximation.  In most cases it will be close enough, which is fine for most engineering problems.  Typically mathematicians have more time and funding to find an analytical solution, but the demands of business usually necessitates an approximate solution for engineers working in industry. 

Analytical methods are typically only solvable in cases of “simple” models. Numerical methods, in contrast, provide ways to manipulate complex math problems so that they may be solved by simple processes. These methods allow for imperfect and complex models to be approximated, usually with great accuracy.  Numerical methods can account for more variables and dimensions than would be solvable when using analytical methods. 

They are implemented by many different industries from meteorologists creating weather models to automobile engineers generating crash test simulations. They are even employed to optimize models for financial firms and insurance companies.

Imagine a car company that is certifying the crash test rating on a new car model. There are several stringent requirements that new vehicles must meet in regard to passenger and pedestrian safety. Building complete prototype cars to crash test is expensive, so car companies run mathematical simulations of crash tests as part of an iterative design process.

An analytical solution accounts for every possible interaction between all components of the vehicle and with the objects of collision.  In the case of a car crash, this would be too difficult to model analytically.  To better use mathematical resources a car will be broken down into several systems, each of which is individually mathematically modeled.  Simulations of different scenarios are then computed and the summations of the results of these systems are modeled over a specified time interval.

When to use Numerical Methods

It is important to note the limitations of numerical methods so they can be used effectively. Numerical methods can only deliver approximate solutions to problems over a defined interval such as time or distance.  The error present in the solution is dependent on how the problem is solved. Analytical methods provide solutions that are valid at all points. Numerical methods are employed with large systems involving many interactions. Numerical methods are used when evaluating empirical information such as experimental data. Such data, no matter how exact and regulated the experiment was conducted, will involve some degree of error.  It would be a waste of time to employ exact analytical techniques in such a situation because the answer can never be more “correct” than the input data.

Numerical

  • The solution is only valid for a defined interval (time, distance)
  • A close approximation is good enough
  • The solution can be applied to many systems
  • Large systems with many interactions
  • Evaluating experimental data
  • Short time / little resources available to construct a solution
  • Offline solutions which are not time-sensitive

Analytical

  • The solution must be valid at any interval (time, distance)
  • An exact solution is required
  • The solution can be only applied to one system
  • Small closed systems
  • Large budget, time, and manpower available to construct a solution
  • No computational aids available (computers with math modeling software)
  • Real-time computations on limited computer hardware

When deciding to use analytical or numerical methods consideration must be given to many criteria. In practice some important factors are the time available to solve, the monetary resources, the computational resources available, and the allowable error. Analytical methods will require more involved mathematical computations and can result in extended computational times, which will both cause increased cost. Numerical methods use simpler mathematical computations, but require frequently repeated computations. Numerical methods may at times be impractical if there are no computational aids such as computer programs or tables of previously calculated values. One needs to consider the interval on which the solution needs to be valid. A larger interval requires more computations for numerical methods and thus more computational time.

The amount of error in a solution when using numerical methods is dependent upon the method or series of methods used and the order in which they are used. Not all methods are valid for all problems. However, some methods allow for quicker computations and thus provide an advantage when they are able to be used. When choosing a method or series of methods to use it is important to determine how accurate your solution needs to be and to track the error in your solution.

It is important for us to differentiate between two key concepts of error: accuracy and precision. Accuracy is how close a computed or measured value comes to the theoretical true value. Precision is how repeatable a value is. To illustrate this point think of throwing darts at a dart board. Being accurate is similar to throwing five darts and surrounding the bull’s eye. Precision is similar to throwing five darts and that all strike right next to each other, but are not necessarily on the bull’s eye.

Functions

A function is a mathematical relationship between at least one input and one constant to a single output. A function that most people are familiar with is the equation for a line:

In this function y is the output variable, x is the input variable, m is an integer, and b is a constant. This type of function is called a linear equation and when graphed will produce a straight line with constant slope.  This same function is also sometimes written with the following notation:

Where f(x) represents the function’s output for the variable that changes (the value along the x-axis).  f(x) is equivalent to y.  Functions become more complicated as more inputs are included and raised to powers. 

Functions do not have to start with f.  They can be named anything we like.  The function that describes the position of a body under constant acceleration is:

Where:

  • x(t) is the position of the body at time t.
  • xinitial is the initial position of the body at t=0
  • vinitialis the initial velocity of the body at t=0
  • t is the current time we want to find the position for
  • a is the acceleration of the body.  Acceleration must be constant for this particular equation to work.

Derivatives

A derivative of a function provides information about how a function is behaving at a particular point.  A derivative describes the rate of change, or slope, of a function at a given point. Imagine that the velocity (meters per second, m/s) of a vehicle is recorded and a function is generated to describe the data. The derivative of the velocity function provides the acceleration (meters per second per second, m/s2) equation, as acceleration gives the rate of change of the velocity.  

The function that describes the velocity of a body under constant acceleration is:

Where:

  • v(t) is the position of the body at time t.
  • vinitialis the initial velocity of the body at t=0
  • t is the current time we want to find the position for
  • a is the acceleration of the body.  Acceleration must be constant for this particular equation to work.

We will now derive the velocity formula analytically.  This example will show how much work it can be to find an analytical solution to even a simple problem.  We will use various rules you haven’t learned yet (highlighted in red) to take the derivative of the position equation which we will write as x’(t):

d/dt is just the notation for a derivative with respect to time.  The prime symbol in x’(t) means the same thing.  There are many different notations for differentiation which are beyond the scope of this article.  The notations were all developed by different mathematicians for their specific problem sets.  In the same way you wouldn’t go racing in a minivan, nor drive an F1 car to on the street, the notations are also specialized for specific applications.

Although these weird symbols look complicated, we can solve it in a few steps by using the rules for differentiation.  We will apply the easiest rule first:

If you have multiple functions separated by plus or minus signs, you can take the derivative of each one separately. This is called the sum or difference rule.

Thus our formula becomes:

The next simplest rule is probably:

The derivative of any constant is 0.

Applying this rule, therefore the equation becomes:

To further simplify the problem, constants can be factored out before taking the derivative.  This is known as the Kutz Rule: 

Constants can be factored outside of a derivative, thus focusing on differentiating the variable.

In the equation below, vinitial does not depend on time and thus may be factored out.  Acceleration, a, does not depend on time because the constraint needed to derive this equation is constant acceleration.  ½ is obviously a constant.

This leaves us with t and t2 to differentiate.  The derivative of a variable is 1, so d/dt(t) becomes 1:

To deal with t2, we use the power rule which states states:

Applying the power rule gives us:

Now we simplify the function by evaluating the constants.  This gives us the formula for velocity as defined above:

Obviously, to perform differentiation on all of our formulas requires knowledge of many rules you have not studied yet.  From here on out the derivatives and integrals will be provided, but this raises an interesting question:

“How do we find the derivative if we don’t know the rules of differentiation?”

The answer is that we can get a very accurate approximation of any function with numerical analysis.  Some functions are so difficult to evaluate that it is impossible to solve them by hand.  This is where being a 21st century student has its advantages.  We can use a computer to find a solution numerically!

For example, if we have a graph of position vs time that was made under constant acceleration, we can find velocity by measuring the slope at each point.  By strict definition however, the slope is undefined at each point because we don’t have a formula that represents the graph, only a picture.  For our purposes slope is defined as:

Make sure you understand this before proceeding:

We are now finding a numerical solution for velocity.  Previously we found an analytical solution to velocity which required the use of all of the differentiation rules highlighted in red.  Our analytical solution could tell us the velocity at any point in time.  The numerical solution we are getting ready to create will only tell us velocity over the time interval of the graph (0 to 15 seconds) and it will only be a close approximation.

In this graph we will now measure the slope at each point:

The slope at t=0 is 0

The slope at t=1 is 0.5

The slope at t=2 is 1

The slope at t=3 is 1.5

The slope at t=15 is 7.5

This gives us a velocity graph that looks like:

This graph matches the output from our analytical model.  If we do the same thing to the velocity graph, taking the slope at each point, then we will get an acceleration graph:

The slope at t=0 is 0.5

The slope at t=1 is 0.5

The slope at t=2 is 0.5

The slope at t=3 is 0.5

The slope at t=15 is 0.5

As you can see we have verified that the acceleration was constant.

Integration

Integration is the opposite of differentiation.  If you integrate an acceleration function you produce a velocity function.  Likewise you can integrate a velocity function to produce a position function. The integral of a function can also be computed as the area under the curve.

The analytical solution to integration is just as complex as differentiation.  You follow a series of rules to find the solution.  Some integrals are very complex and cannot be solved analytically.  Numerous methods have been invented to integrate numerically.  The rectangle rule presented below is one of the most simple and inaccurate methods.  In future articles we will study other methods to deliver a more precise answer.

Rectangle Rule

How would we measure the area under a curve?   The entire area would be infinite, but if we set an upper and lower limit on the x-axis, we can compute only part of the curve.  For a simple example, let’s measure the area under the acceleration graph we just made.  This should give us the velocity at t=15.

The area shape is a rectangle.  The area of a rectangle is base x height.  In this case it will be:

Looking at the velocity graph, the last data point (t = 15) is 7.5 m/s.

It’s apparent that we could find the area of the velocity graph with a triangle to get the final position value.  Let’s try it:

The formula for area of a triangle is ½ * base * height.

As you can see on the position graph below, we just found the position at t=15 seconds by integrating the velocity graph.

What about a more complex curve?  For the function y=x2, let’s set a lower limit of 15 and an upper limit of 40.

No geometric shapes fit this graph, but we know the area of a rectangle is base multiplied by height, so we can approximate by cutting the curve into small pieces and compute the rectangular area making up each piece.

The rectangle rule uses the area under the curve determined at equally spaced points across an interval x0to xalong the x-axis. The first step is to divide the interval into n rectangles. The length of each rectangle is found by equation 2.

The area of each rectangle is calculated using the height of the curve at some point where the top of each rectangle touches the curve. There are three different ways of determining the height for each segment.  The left hand method, mid-point method, right hand method all describe different locations where a rectangle touches the curve.

Using the left hand method the height for each segment is calculated as the y value of the curve at the left most point of the rectangle. Using the mid-point method the height for each segment is calculated as the y value of the curve at the mid-point of the rectangle. Using the right hand method the height for each segment is calculated as the y value of the curve at the right most point of the rectangle. These methods are illustrated below.




As you can see the left hand method tends to underestimate the area, the right hand method overestimates the area, and the midpoint method is a compromise between the two.  This only holds true for curves that tend to have a positive slope.  Curves with an overall negative slope will behave in an opposite manner.  We can minimize error by selecting the correct rule and increasing the number of rectangles used in our calculation. 

Let’s try an example problem from the real world. The following video shows Tony Schumacher setting the NHRA top-fuel dragster speed record in 2005. 

The dragster took 4.489 seconds to reach the end of a 1000-foot track. As he sped down the track the following acceleration data was recorded.  What was his top speed?

There are no shapes that are easy to find the area which approximate this area accurately.  We can get a back-of-the envelope estimate by using a rectangle and triangle.  This will be a good sanity check against our final answer:

The rectangle has the approximate dimensions of:

The triangle also has the approximate dimensions of 2 x 39.  This gives an area of:

Adding the area of the rectangle and triangle together gives a total area of:

The lower limit of integration will be 0 seconds and the upper limit will be 4.5 seconds.  To calculate each rectangle using the left-hand rule, the formula will be:

The first velocity data point will be 0 because the initial velocity is zero and i-1 does not exist for the first data point.  After that the next few points are:

When the car crosses the finish line the area is now:

This is close enough to our rectangle and triangle area estimate of 135 m/s to let us know that we are in the ballpark.

If we convert 149.79 m/s to miles per hour, the result is 335.1, which is a little low because of the error we generated with each time step.  The actual result was 336.15 mph, which gives us an error at the final data point of:

The formula for the right hand method is very similar, except acceleration from the current time step is used:

The midpoint rule uses a linear interpolation (average) of the previous and current acceleration values:

A comparison between left-hand, right-hand, and midpoint method error has been performed for a 100ms step size below:

The rectangle rule will work well when there are plenty of data points.  In the case of our dragster accelerometer data, the time step could not be reduced lower than 0.1 seconds (100ms) because the accelerometer data collection rate was a limiting factor.  What do you think would happen to the result if the time step was increased to 0.5 seconds?

As you can see, with a large time step, the accuracy is much worse!  In this case it is unacceptable. The next article in this series will provide more advanced ways of dealing with integration to further reduce error.

Acknowledgements

Special thanks to David Judah and Tyler Bishop for their help in preparing this article.

AddThis Social Bookmark Button

Featured Desolation Modules

Teacher Manuals

We've got your resources - Slides, manuals, and more! Click here (account registration required)

Profile