Understanding floating point precision

Question

Is it the case that:

1. Representable floating point values are densest in the real number line near zero?
2. Representable floating point values grow sparser (exponentially?) as the number line moves away from zero?
3. If the above two are true, does that mean there is less precision farther from zero?

Overall question: Does precision in some way refer to or depend on the density of numbers you can represent (accurately)?

Answer 1

Answers:

1. Representable floating point values are densest in the real number line near zero? Yes
2. Representable floating point values grow sparser (exponentially? No - It decreases hyperbolically) as the number line moves away from zero? Yes
3. If the above two are true, does that mean there is less precision farther from zero? Yes

Overall question: Does precision in some way refer to or depend on the density of numbers you can represent (accurately)?

See https://stackoverflow.com/a/24179424

I also recommend What Every Computer Scientist Should Know About Floating Point Arithmetic

Answer 2

Representable floating point values are densest in the real number line near zero?

In a full implementation of IEEE 754 floating point yes.

However in systems that do not support subnormals, there is a gap around zero which is substantially larger than the difference between the smallest nonzero value and the second smallest nonzero value.

Representable floating point values grow sparser (exponentially?) as the number line moves away from zero?

Yes, each time the value passes a power of 2, the gap between adjacent values doubles.

If the above two are true, does that mean there is less precision farther from zero?

That depends on how exactly you define "precision", one can talk about precision in either a relative sense ("significant figures") or an absolute sense ("decimal places").

Which is more appropriate depends on what exactly the numbers are used for. Loss of precision when moving away from zero tends to become a real concern if floating point numbers are used for things like coordinates or timestamps.

Answer 3

The term precision usually refers to the number of significant digits (bits) in the represented value. So precision varies with the number of bits (or digits) in the mantissa of representation. Distance from the origin has no role.

What you say is true about the density of floats on the real line. But in this case the right term is accuracy, not precision. FP numbers of small magnitude are far more accurate that larger ones. This contrasts with integers, which have uniform accuracy over their ranges.

I highly recommend the paper What Every Computer Scientist Should Know About Floating Point Arithmetic, which covers this and much more.

Answer 4

Floating point numbers are basically stored in scientific notation. As long as they are normalized, they consistently have the same number of significant figures, no matter where you are on the number line.

If you consider density linearly, then the floating point numbers get exponentially more dense as you get closer to 0.

As you get extremely closed to 0, and the exponent reaches its lowest point, the floating point numbers become denormalized. At this point, they have 1 extra significant figure and are thus more precise.