Round-off error

A roundoff error,[1] also called rounding error,[2] is the difference between the result produced by a given algorithm using exact arithmetic and the result produced by the same algorithm using finite-precision, rounded arithmetic.[3] Rounding errors are due to inexactness in the representation of real numbers and the arithmetic operations done with them. This is a form of quantization error.[4] When using approximation equations or algorithms, especially when using finitely many digits to represent real numbers (which in theory have infinitely many digits), one of the goals of numerical analysis is to estimate computation errors.[5] Computation errors, also called numerical errors, include both truncation errors and roundoff errors.

When a sequence of calculations with an input involving any roundoff error are made, errors may accumulate, sometimes dominating the calculation. In ill-conditioned problems, significant error may accumulate.[6]

In short, there are two major facets of roundoff errors involved in numerical calculations:[7]

The error introduced by attempting to represent a number using a finite string of digits is a form of roundoff error called representation error.[8] Here are some examples of representation error in decimal representations:

Increasing the number of digits allowed in a representation reduces the magnitude of possible roundoff errors, but any representation limited to finitely many digits will still cause some degree of roundoff error for uncountably many real numbers. Additional digits used for intermediary steps of a calculation are known as guard digits.[9]

Rounding multiple times can cause error to accumulate.[10] For example, if 9.945309 is rounded to two decimal places (9.95), then rounded again to one decimal place (10.0), the total error is 0.054691. Rounding 9.945309 to one decimal place (9.9) in a single step introduces less error (0.045309). This commonly occurs when performing arithmetic operations (See Loss of Significance).

Machine epsilon can be used to measure the level of roundoff error in the floating-point number system. Here are two different definitions.[3]

There are two common rounding rules, round-by-chop and round-to-nearest. The IEEE standard uses round-to-nearest.

Even if some numbers can be represented exactly by floating-point numbers and such numbers are called machine numbers, performing floating-point arithmetic may lead to roundoff error in the final result.

Machine addition consists of lining up the decimal points of the two numbers to be added, adding them, and then storing the result again as a floating-point number. The addition itself can be done in higher precision but the result must be rounded back to the specified precision, which may lead to roundoff error.[3]

From this example, it can be seen that roundoff error can be introduced when doing the addition of a large number and a small number because the shifting of decimal points in the mantissas to make the exponents match may cause the loss of some digits.

The subtracting of two nearly equal numbers is called subtractive cancellation.[3]

Errors can be magnified or accumulated when a sequence of calculations is applied on an initial input with roundoff error due to inexact representation.

An algorithm or numerical process is called stable if small changes in the input only produce small changes in the output and it is called unstable if large changes in the output are produced.[11]

A sequence of calculations normally occur when running some algorithm. The amount of error in the result depends on the stability of the algorithm. Roundoff error will be magnified by unstable algorithms.

The roundoff error is amplified in succeeding calculations so this algorithm is unstable.

Even if a stable algorithm is used, the solution to a problem may still be inaccurate due to the accumulation of roundoff error when the problem itself is ill-conditioned.

The condition number is introduced as a measure of the roundoff errors that can result when solving ill-conditioned problems.[7]