CODEWITHSUNDEEP
c++floating-point
Faken
Faken

Reputation: 11822

Incorrect floating point math?

Here is a problem that has had me completely baffled for the past few hours...

I have an equation hard coded in my program:

double s2;

s2 = -(0*13)/84+6/42-0/84+24/12+(6*13)/42;

Every time i run the program, the computer spits out 3 as the answer, however doing the math by hand, i get 4. Even further, after inputting the equation into Matlab, I also get the answer 4. Whats going on here?

The only thing i can think of that is going wrong here would be round off error. However with a maximum of 5 rounding errors, coupled with using double precision math, my maximum error would be very very small so i doubt that is the problem.

Anyone able to offer any solutions?

Thanks in advance,

-Faken

Upvotes: 2

Views: 1138

Answers (4)

ezpz
ezpz

Reputation: 12047

s2 = -(0*13)/84+6/42-0/84+24/12+(6*13)/42;

yields 3

s2 = -(0.*13.)/84.+6./42.-0./84.+24./12.+(6.*13.)/42.;

does what you are expecting.

Upvotes: 3

ChrisW
ChrisW

Reputation: 56123

Some of this is being evaluated using integer arithmetic. Try adding a decimal place to your numbers, e.g. 6.0 instead 6 to tell the compiler that you don't want integer arithmetic.

Upvotes: 3

Amber
Amber

Reputation: 527378

You're not actually doing floating point math there, you're doing integer math, which will floor the results of divisions.

In C++, 5/4 = 1, not 1.25 - because 5 and 4 are both integers, so the result will be an integer, and thus the fractional part of the result is thrown away.

On the other hand, 5.0/4.0 will equal approx. 1.25 because at least one of 5.0 and 4.0 is a floating-point number so the result will also be floating point.

Upvotes: 17

duffymo
duffymo

Reputation: 309008

You're confusing integer division with floating point division. 3 is the correct answer with integer division. You'll get 4 if you convert those values to floating point numbers.

Upvotes: 7

Related Questions