CODEWITHSUNDEEP
c++floating-point
CaptainCodeman
CaptainCodeman

Reputation: 2201

Is it possible for "floor" to return an inaccurate result due to floating point rounding error?

I understand that floating point numbers can often include rounding errors.

When you take the floor or ceiling of a float (or double) in order to convert it to an integer, will the resultant value be exact or can the "floored" value still be an approximation?

Basically, is it possible for something like floor(3.14159265) to return a value which is essentially 2.999999, which would convert to 2 when you try to cast that to an int?

Upvotes: 3

Views: 2490

Answers (3)

David Heffernan
David Heffernan

Reputation: 612993

Is it possible for something like floor(3.14159265) to return a value which is essentially 2.999999?

The floor() function returns an floating point value that is an exact integer. So the premise of your question is wrong to begin with.

Now, floor(x) returns the nearest integral value that is not greater than x. It is always true that

floor(x) <= x

and that there exists no integer i, greater than floor(x), such that i <= x.

Looking at floor(3.14159265), this returns 3.0. There's no debate about that. Nothing more to say.

Where it gets interesting is if you write floor(x) where x is the result of an arithmetic expression. Floating point precision and rounding can mean that x falls on the wrong side of an integer. In other words, the true value of the expression that yields x is greater than some integer, i, but that x when evaluated using floating point arithmetic is less than i.

Upvotes: 12

Tony Delroy
Tony Delroy

Reputation: 106116

Interestingly, floats can store a certain range of integers exactly, for example:

  • 1 is stored as mantissa 1 (binary 1) * exponent 2^0
  • 2 is stored as mantissa 1 (binary 1) * exponent 2^1
  • 3 is stored as mantissa 1.5 (binary 1.1) * exponent 2^1
  • 4 is stored as mantissa 1 * exponent 2^2
  • 5 is stored as mantissa 1.25 (binary 1.01) * exponent 2^2
  • 6 is stored as mantissa 1.5 (binary 1.1) * exponent 2^2
  • 7 is stored as mantissa 1.75 (binary 1.11) * exponent 2^2
  • 8 is stored as mantissa 1 (binary 1) * exponent 2^3
  • 9 is stored as mantissa 1.125 (binary 1.001) * exponent 2^3
  • 10 is stored as mantissa 1.25 (binary 1.01) * exponent 2^3 ...

As you can see, the way exponents increase works in with the perfectly-stored fractional values the mantissa can represent.

You can get a good sense for this by putting number into this great online conversion site.

Once you cross a certain threshold, there's not enough digits in the mantissa to divide the span of the increased exponents without skipping first every odd integer value, then three out of every four, then 7 out of 8 etc.. For numbers over this threshold, the issue is not that they might be different from integer values by some tiny fractional amount, its that all the representable values are integers and not only can no fractional part be represented any more, but as above some of the integers can't be either.

You can observe this in the calculator by considering:

Binary                             Decimal
+-Exponent Mantissa
0 10010110 11111111111111111111111 16777215
0 10010111 00000000000000000000000 16777216
0 10010111 00000000000000000000001 16777218

See how at this stage, the smallest possible increment of the mantissa is actually "worth 2" in terms of the decimal value represented?

When you take the floor or ceiling of a float (or double) in order to convert it to an integer, will the resultant value be exact or can the "floored" value still be an approximation?

It's always exact. What floor is doing is effectively wiping out any '1's in the mantissa whose significance (their contribution to value) is fractional anyway.

Basically, is it possible for something like floor(3.14159265) to return a value which is essentially 2.999999, which would convert to 2 when you try to cast that to an int?

No.

Upvotes: 5

rodrigo
rodrigo

Reputation: 98388

Small integers are representable exactly as floats, but big integers are not.

But, as others pointed out, big integers not representable by float will never be representable by a non-integer, so floor() will never return a non-integer value. Thus, the cast to (int), as long as it does not overflow, will be correct.

But how small is small? Copying shamelessly from this answer:

For float, it is 16,777,217 (224 + 1).
For double, it is 9,007,199,254,740,993 (253 + 1).

Note that the usual range of int (32-bits) is 231, so float is unable to represent all of them exactly. Use double if you need that.

Upvotes: 5

Related Questions