CODEWITHSUNDEEP
unit-testingfitnessefitsharpdbfit
sduplooy
sduplooy

Reputation: 14720

FitNesse value comparison fails even though the actual and expected is the same

I run a function in SQL Server that returns some calculated values. When testing the results in FitNesse using dbFit it claims that the values are not the same (see the screenshot below).

The values are returned as floats and fails on the comparison of all decimals - even though it only displays two. Is there a way to limit the range of decimals that gets compared?

FitNesse values that are mismatched

Upvotes: 3

Views: 1254

Answers (1)

K.L.
K.L.

Reputation: 2487

You should NEVER compare floats, at least not strictly. Thats because of the binary system imperfections. 

float a = 0.15 + 0.15
float b = 0.1 + 0.2
if(a == b) // can be false!
if(a >= b) // can also be false!

Since the significant digits in a real number are limited, we cant store real numbers perfectly. Theres almost always an error. Try to write 0.1 in binary! What we see printed on the screen as 0.1 is really only an approximation. Its a vast subject, you might want to get into numerical analysys for details.

On a more general note, you'll find some info here: http://www.cygnus-software.com/papers/comparingfloats/comparingfloats.htm

as for your problem:

You should use ~= (approx. equal). You could find more info here: http://fitnesse.org/FitNesse.UserGuide.SliM.ValueComparisons

Upvotes: 3

Related Questions