Double precision - MS visual C++ 2005 Vs Matlab

Question

I am using MS visual C++ 2005 for implementing some mathematical algorithms in C++. My C++ code takes 10K double data type input values with precision of 12 decimal places(For e.g. 866.333333333333) which are generated in Matlab. Then my code does some calculations and gives result which is damage value of a mechanical entity which ought to be the same as matlab output when same algorithm is run in it with same input values.

My problem is that matlab gives 10k input values with precision of 12 decimal places but my c++ code makes them to 15 decimal places. That means 866.333333333333 from matlab would be used as 866.33333333333303 in C++. I know that this is what IEEE754 Floating point representation. But this very tiny little change at input does make measurable difference in the final result when i compare my C++ output with matlab output. I may sound weird but this is what may observation is. I wish i could share some code here but it is very confidential. I would really appreciate if anyone help me out to get through this.

Thanks in advance...!

Answer 1

It would be more accurate if this question (and any answers) were framed in terms of significant digits rather than decimal places. The IEEE standards do, as Itamar Katz has hinted, store double precision floating-point numbers in 52 bits for the significand (what some call the mantissa). The standard also implies an extra bit, so doubles have 53 significant bits. When the binary number is translated into a decimal representation this translates to 15 or 16 significant digits.

Neither Matlab nor Visual C++ can (without additional facilities such as arbitrary-precision libraries or using 128-bit f-p numbers) store doubles with more than the standard size of significand. If your program, in either language, presents a number to you with more than 15 (or 16) decimal digits you cannot trust any of the excess digits. They haven't come from the stored representation of the number, they have been added somewhere along the line between memory and screen -- perhaps a 'helpful' numeric formatter has simply extended the rightmost digit until you see the 19 digits you've requested (or whatever).

It's not entirely clear from the question how you transfer numbers from C++ to/from Matlab, or even that you do transfer numbers; maybe you are just trying to write a C++ program which reproduces the results of your Matlab program. (We do that a lot here so I have some experience in this field.)

If you use 'text' files then you are transferring not numbers but representations of numbers. If your program reads the text '15.833' into a double variable, it is not safe to make any assumptions about the the values taken by the extra digits in the significand. In particular you should not assume that they will be set to 0 -- well, I guess someone more knowledgeable about C++ might be able to tell us that the language standard does guarantee this but Matlab doesn't and I don't think that C++ does either. If you want to set the extra digits, specify them in your textual representation. Even this will not guarantee that you store the value exactly as specified in your text file, your variable will (probably) hold the nearest f-p number to the value in the text.

However, if your text file is written by Matlab (or C++) and writes 15 or 16 digits into the textual representation of a number then it should be a textual representation of the entire f-p number and, on reading by another program, should get translated into the same f-p number. Note, though, that I write 'should' and that the number has been translated at least twice and strange things happen when you take your eye off the numbers on the computer.

A better choice for bit-accurate transfer of data between C++ and Matlab is to use a binary file format, one which stores all 64 bits of a double as 64 bits. The Matlab MAT file format certainly stores IEEE754 numbers in the form specified by the standard.

It's possible that all of the preceding guff is irrelevant to another, underlying, issue. That issue may be that your algorithm is not stable -- which is a whole other topic.

To summarise:

1. Transfer numbers from program-to-program in their binary representation (whether you use a file or message-passing or other mechanism entirely).
2. Don't trust more than 15 significant digits in any decimal representation of double precision floating-point numbers.

Furthermore, unless you take special measures in your code, your programs probably gradually lose accuracy as they progress, making all the low-order digits of dubious reality. For the application that you hint at, it's unlikely that the science behind the code supports the hypothesis that two outputs which differ in the 15th significant digit represent different values. What is the accuracy of the measurements on which your inputs are based ?

Answer 2

1. Floating points data types are not defined by the number of decimal places, but by the number of bits used to represent a number, and the rules used to interpret these bits (i.e, the IEEE754 standard).
2. Generally it is not a trivial task to achieve the same output (given the same input) in two different "environments" (i.e platform/CPU/compiler etc.). As a loose example, consider the fact that a compiler might have some freedom in calculating an expression like a+b+c either as (a+b)+c or a+(b+c), which in floating point calculations might give different results. I strongly recommend you reconsider --- is getting exactly the same results really crucial?
3. Despite what's written in 2, in your case it might be possible. As a first step you should use binary representation of your data, not textual. In other words, hold the bytes that represent your data and pass it to your C++ code. For example, instead of passing 1.0, pass 0x3FF0000000000000 (although 1.0 is exactly binary-representable --- this is just an example).

Of course, some code would help. Try to build a toy example which shows your problem without revealing any confidential information.