When will I run into problems when calculating with double values?

Question

Imagine I write a simple calculator application, that just calculates simple stuff like

1.5 + 30 + 9755 - 30 - 20000 + 999900.54

I remember slightly that there were some precision problems when using floating point numbers. At which point would my calculator app start to create wrong results? Most of the time, I would just calculate integers like 1 + 2 - 963422, but sometimes I may enter a floating point number. I have no big clue about where the precision problems would start to take effect. just the very last numbers of that double? like -963419.0000000000003655? Or how would that look like? And any idea how to catch those?

Answer 1

For a practical example, in IEEE754 32 bit (single precision) floating point the integers can be represented only up to 16777216, above that there are gaps. The next floating point number after 16777216 is 16777218.

Answer 2

Citing Wikipedia:

In addition to loss of significance, inability to represent numbers such as π and 0.1 exactly, and other slight inaccuracies, the following phenomena may occur:

• Cancellation: subtraction of nearly equal operands may cause extreme loss of accuracy. This is perhaps the most common and serious accuracy problem.
• Conversions to integer are not intuitive: converting (63.0/9.0) to integer yields 7, but converting (0.63/0.09) may yield 6. This is because conversions generally truncate rather than round. Floor and ceiling functions may produce answers which are off by one from the intuitively expected value.
• Limited exponent range: results might overflow yielding infinity, or underflow yielding a subnormal number or zero. In these cases precision will be lost.
• Testing for safe division is problematic: Checking that the divisor is not zero does not guarantee that a division will not overflow and yield infinity.
• Testing for equality is problematic. Two computational sequences that are mathematically equal may well produce different floating-point values. Programmers often perform comparisons within some tolerance (often a decimal constant, itself not accurately represented), but that doesn't necessarily make the problem go away.

In order to avoid such problems, you need to analyze your specific calculations in order to minimize error propagation.

Answer 3

Try 0.1 + 0.2 - 0.3.

Answer 4

Floating point math is full of pitfalls! Here are a few you should watch out for:

1) Adding very large and very small (in absolute value) numbers. IEEE-754 single precision floating point numbers have about 7 significant decimal digits of precision. So if you try to calculate 1000000.0 + 3.14159, you're going to lose most of the digits after the decimal point. This can occur if you use a naive algorithm to calculate a running sum of a large array (millions of small values). To see a better way to do it, check out Kahan summation .

2) There's a similar problem with subtracting two large, nearly equal numbers. The result may only have one or two significant figures. Again, the solution is often to rearrange the calculation in a way that avoids "subtracting two elephants to get a mouse".

Answer 5

It's hard to say "when", it depends on how big/small your numbers are, how many operations you make, and how much precision you need.

Some languages support a special object/construct for handing precise decimal operations. Java has BigDecimal: http://java.sun.com/javase/6/docs/api/java/math/BigDecimal.html

Answer 6

Floating-point precision "issues" are always present in every floating-point calculation.

Sometimes, you're lucky, and are working with numbers that only involve powers of 2: x.25, y.125, etc. The part to the right of the decimal point is 1/(2^k) for any integer, k.

If you work with any value where the decimal places are not a power of two, you have representation issues. 1./3., 1./5., 1./6., 1./7., 1./9., etc.

Floating point representation issues occur with all non-power-of-2 values.