CODEWITHSUNDEEP
c++floating-point
Guillaume Paris
Guillaume Paris

Reputation: 10539

std::numeric_limits::digits10<float> and precision after the dot

When std::numeric_limits::digits10<float> return 7 does it mean that I have 7 significatif figures after the dot or that 7 with the left part?

For instance is it like:

1.123456
12.12345

or is it like

12.1234657

Upvotes: 2

Views: 399

Answers (1)

Marco A.
Marco A.

Reputation: 43662

From cppreference

The value of std::numeric_limits::digits10 is the number of base-10 digits that can be represented by the type T without change, that is, any number with this many decimal digits can be converted to a value of type T and back to decimal form, without change due to rounding or overflow. For base-radix types, it is the value of digits (digits-1 for floating-point types) multiplied by log 10(radix) and rounded down.

And later

The standard 32-bit IEEE 754 floating-point type has a 24 bit fractional part (23 bits written, one implied), which may suggest that it can represent 7 digit decimals (24 * std::log10(2) is 7.22), but relative rounding errors are non-uniform and some floating-point values with 7 decimal digits do not survive conversion to 32-bit float and back: the smallest positive example is 8.589973e9, which becomes 8.589974e9 after the roundtrip. These rounding errors cannot exceed one bit in the representation, and digits10 is calculated as (24-1)*std::log10(2), which is 6.92. Rounding down results in the value 6.

That means, e.g.

cout << std::numeric_limits<float>::digits10; // 6
cout << std::numeric_limits<float>::digits; // 24

the second one is the number of digits in the mantissa while the first one the number of decimal digits that can safely be represented across aforementioned conversions.

TL;DR: it's your first case.

Upvotes: 2

Related Questions