CODEWITHSUNDEEP
c#doubleprecision
Alistair
Alistair

Reputation: 1979

double minus double giving precision problems

I have come across a precision issue with double in .NET I thought this only applied to floats but now I see that double is a float.

double test = 278.97 - 90.46;
Debug.WriteLine(test) //188.51000000000005

//correct answer is 188.51

What is the correct way to handle this? Round? Lop off the unneeded decimal places?

Upvotes: 10

Views: 9271

Answers (2)

foxwoods
foxwoods

Reputation: 500

This happens in many languages and stems from the fact that we cannot usually store doubles or floats precisely in digital hardware. For a detailed explanation of how doubles, floats, and all other floating point values are often stored, look at the various IEEE specs described on wikipedia. For example: http://en.wikipedia.org/wiki/Double_precision

Of course there are other formats, such as fixed-point format. But this imprecision is in most languages, and why you often need to use epsilon tests instead of equality tests when using doubles in conditionals (i.e. abs(x - y) <= 0.0001 instead of x == y).

How you deal with this imprecision is up to you, and depends on your application.

Upvotes: 5

Michael Petito
Michael Petito

Reputation: 13161

Use the decimal data type. "The Decimal value type is appropriate for financial calculations requiring large numbers of significant integral and fractional digits and no round-off errors."

Upvotes: 11

Related Questions