CODEWITHSUNDEEP
java
Peter
Peter

Reputation: 61

java floating point error when reading from binary file

I have a floating point number stored in a binary file 0.9999833107. When I read the number in it is either truncated or extended depending on whether I use float or double to store it. I read in the binary with the following code:

public double readFloat(RandomAccessFile fp){

    byte[] data = new byte[4];

    try {

        data[0] = fp.readByte();
        data[1] = fp.readByte();
        data[2] = fp.readByte();
        data[3] = fp.readByte();

    } catch (IOException e) {
            e.printStackTrace();
            return 0;
    }

    ByteBuffer buffer = ByteBuffer.wrap(data);

    return buffer.getDouble();
}

This method returns 0.9999833106994629. When I change the method to output a float the value returned is 0.9999833. Does anyone have any idea how I can get the number out that was written into the file?

Upvotes: 2

Views: 587

Answers (1)

Jon Skeet
Jon Skeet

Reputation: 1500953

If you're only storing 4 bytes, you could only possibly have 9 guaranteed significant digits of accuracy, as there are only 4294967296 values representable within 32 bits - and that's assuming you could use all the bits as mantissa, with nothing for exponent or sign.

I think it's far more likely that this was originally a value "close to" 0.9999833107, represented as a double, and then truncated to a float for storage. Note that 0.9999833107 itself can't be stored exactly as a double - the closest double value to that is 0.9999833107000000165243136507342569530010223388671875.

In other words, when reading it as a float you are getting "the number that was written into the file". It may not have been the number you had before the conversion for storage, but it's the number that was actually written into the file.

Upvotes: 3

Related Questions