CODEWITHSUNDEEP
floating-pointbinaryprecisionieee-754
Harshit Singh
Harshit Singh

Reputation: 25

Precision of single precision floating point numbers (IEEE 754 standard)

In the single precision floating point representation of numbers according to IEEE 754 standard, we use 24 bits for mantissa part (23 bits + 1 implied bit). So the precision can be calculated as 2^24 = 10^x where x can be calculated by taking log on both sides as 24log 2 = xlog 10 => x= 7.2 ~ 7. From this we can conclude that precision is 7 in decimal system but the value 7 tells that we have 7 significant decimal places of precision or we have 7 decimal digits of precision? Is it considered to be 7 decimal digits (in total) or upto 7 decimal places of precision for decimal numbers.

Upvotes: 0

Views: 1501

Answers (1)

chux
chux

Reputation: 153303

precision can be calculated as 2^24 = 10^x ...

From this we can conclude that precision is 7 in decimal system but the value 7 tells that we have 7 significant decimal places of precision or we have 7 decimal digits of precision?

Is it considered to be 7 decimal digits (in total) or upto 7 decimal places of precision for decimal numbers.

No.

7 may work as a rough approximation of precision*1, yet floating point numbers are not quite distributed in a uniform logarithm fashion so “by taking log” fails.

For typical float, there are 223 or 8,388,608 values, linearly distributed in the range [1.0 … 2.0). Likewise for ranges [0.125 … 0.5), [128.0 … 256.0), …

With text as a decimal with precision of 7, there are 9*106 or 9,000,000 values linearly distributed in the range [1.0 … 10.0) and in other decades.

The problem is these ranges do not always line up to distinguish 7 significant decimal places.

Consider 8589952000.0f. The next float is 1024.0 away, yet the next 7 significant decimal is 1000.0 away. After 125 steps to the next float, the value is 8590080000.0f. But that was 128 steps of 1000.0. Some 7 significant decimals in that range were not distinctively representable as a float.

Some float have less than 7 significant decimals of distinctiveness.

Use floor((24-1) * log10(2)) --> 6 to determine a common float worst-case decimal precision or for floats in general, use FLT_DIG.

*1 log10(pow(2,23)) --> 6.9... is a better approximation.

Upvotes: 3

Related Questions