CODEWITHSUNDEEP
floating-pointdoublehex
jakstack
jakstack

Reputation: 2205

Java: Why the Hex value is different for same floating-point value?

So I have BigDecimal holding value 0.99 and I invoke:

I'm thinking since we are representing a small number like 0.99 we should get the same hex value regardless. Agree?

Code (failing test):

@Test public void testDelta() {
    BigDecimal rationalNumber = new BigDecimal("0.99").setScale(2,BigDecimal.ROUND_DOWN);

    String hexFromFloat = Float.toHexString(rationalNumber.floatValue());
    String hexFromDouble = Double.toHexString(rationalNumber.doubleValue());

    String hexFromFloatMsg = rationalNumber.floatValue() + " = " + hexFromFloat;
    String hexFromDoubleMsg = rationalNumber.doubleValue() + " = " + hexFromDouble;

    Assert.assertEquals(hexFromFloatMsg + ", " + hexFromDoubleMsg, hexFromDouble, hexFromFloat);
}

Output:

org.junit.ComparisonFailure: 0.99 = 0x1.fae148p-1, 0.99 = 0x1.fae147ae147aep-1 
Expected :0x1.fae147ae147aep-1
Actual   :0x1.fae148p-1

Upvotes: 0

Views: 429

Answers (2)

Eric Postpischil
Eric Postpischil

Reputation: 223553

The difference occurs in these two operations:

rationalNumber.floatValue()
rationalNumber.doubleValue()

Each of these converts a BigDecimal value of .99 to floating-point. In hexadecimal floating-point (with a decimal exponent for a power of two), .99 is 0x1.fae147ae147ae147…p-1. When this is converted to float, only 24 bits of the significand (fraction part) can be stored, because that is all the bits the float type has for a significand. (23 bits are stored explicitly; one is implicit from other parts of the encoding.) So, the conversion must round the exact value of .99 to something that fits in 24 bits. This produces 1.fae148p-1. If you write 1.fae147ae147ae147… in binary and count out 24 bits, you can see where the rounding occurs. Here they are with the first 24 bits in bold: 1.11111010111000010100011110101110000101000111…p-1. When rounding, we look at the bits being removed, see that they are more than half the lowest bit being kept (the first bit being removed is 1, and there are additional 1 bits beyond it), and decide to round up. So rounding produces 1.11111010111000010100100p-1. In hexadecimal, that is 1.fae148p-1.

When .99 is converted to double, 53 bits of the significand can be stored. So it is rounded at a different position. This produces 0x1.fae147ae147aep-1.

The two values are different, comparing them directly would report they are different, and converting them from the floating-point format to hexadecimal numerals produces different results.

Upvotes: 2

Kevin
Kevin

Reputation: 56129

since we are representing a small number like 0.99 we should get the same hex value regardless. Agree?

It's a small precision in decimal, but, as you can see, its hex representation repeats (0x1.f(ae147)*). It is being represented exactly, but it cannot be printed exactly.

Upvotes: -1

Related Questions