About python's rounding error and floating-point numbers

Question

>>> 1/3
0.3333333333333333

>>> 1/3+1/3+1/3
1.0 

I can't understand why this is 1.0. Shouldn't it be 0.9999999999999999? So I kind of came up with the solution that python has an automatic rounding for it's answer, but if than, the following results can't be explained...

>>> 1/3+1/3+1/3+1/3+1/3+1/3
1.9999999999999998

>>> (1/3+1/3+1/3)+(1/3+1/3+1/3)
2.0

I thought rounding error occurred because there were only limited number of digits to use in the mantissa, and the exponent,(in floating point numbers) but 0.9999~~9 is not off the limit of the number of digits too.. Can somebody explain why these results came out like this?

Answer 1

Most Python implementations use a binary floating-point format, most commonly the IEEE-754 binary64 format. This format has no decimal digits. It has 53 binary digits.

When this format is used with round-to-nearest-ties-to-even, computing 1/3 yields 0.333333333333333314829616256247390992939472198486328125. Your Python implementation fails to show the full value by default; it shows “0.3333333333333333”, which misleads you.

When this number is added to itself, the result is 0.66666666666666662965923251249478198587894439697265625. This result is exact; it has no rounding error. (That is, it has no new rounding error; it is exactly the sum of 0.333333333333333314829616256247390992939472198486328125 with itself.)

When 0.333333333333333314829616256247390992939472198486328125 is added again, the real-number result does not fit in 53 bits. So the result must be rounded. This rounding happens to round upward, producing exactly 1.

When 0.333333333333333314829616256247390992939472198486328125 is added again, the result again does not fit, and is rounded. This time, the rounding happens to be downward, and produces 1.3333333333333332593184650249895639717578887939453125.

Subsequent additions produce 1.666666666666666518636930049979127943515777587890625 and then 1.9999999999999997779553950749686919152736663818359375, which your Python implementation displays as “1.9999999999999998”.

When you group the arithmetic as (1/3+1/3+1/3) + (1/3+1/3+1/3), then 1 is obtained for each parenthesized item, as explained above, and 1+1 is of course 2.

Answer 2

This is one of the subtle points of IEEE-754 arithmetic. When you write:

>>> 1/3
0.3333333333333333

the number you see printed is a "rounded" version of the number that is internally stored as the result of 1/3. It's just what the Double -> String conversion in the printing process decided to show you. But you already knew that.

Now you can ask, is there a way to find out what the difference is? Yes, use the fractions module:

>>> from fractions import Fraction
>>> Fraction(1, 3) - Fraction(1/3)
Fraction(1, 54043195528445952)

Ah, that's interesting. So it is slightly less than the actual value, and the difference is 1 / 54043195528445952. This is, of course, expected.

So, what happens when you "add" two of these together. Let's see:

>>> Fraction(2,3) - Fraction(1/3+1/3)
Fraction(1, 27021597764222976)

Again, you're close to 2/3rds, but still not quite there. Let's do the addition one more time:

>>> Fraction(1,1) - Fraction(1/3+1/3+1/3)
Fraction(0, 1)

Bingo! with 3 of them, the representation is exactly 1.

Why is that? Well, in each addition you get a number that's close to what you think the answer should be, but the internal rounding causes the result to become a close-by number that's not what you had in mind. With three additions what your intuition tells you and what the internal rounding does match up.

It is important to emphasize that the addition 1/3 + 1/3 + 1/3 does not produce a 1; it just produces an internal value whose closest representation as an IEEE-754 double-precision floating point value is 1. This is a subtle but important difference. Hope that helps!

Answer 3

This question may provide some answers to the floating point error Is floating point math broken?

With the brackets, the compiler is breaking down the addition into smaller pieces, reducing the possibility of a floating point which is not supposed to be there to carry on and keep 'accumulating' the floating point error. Most likely when split into groups of 3, the compiler knows the sum will be 1 and add 1 + 1 together