user2420472
user2420472

Reputation: 1177

C# floating point number's precision of display and calculation, which are different

I am doing calculation by C# on Win7.

I got a double number precision problem.

 double x = 3389774.583;

 string ss = "x is " + x + " x square is " + x * x + " , 11490571723552.8 - x*x is " + (11490571723552.8 - x * x) + "\n";

 Console.WriteLine(ss);

you may find the result is not 0 even tough 11490571723552.8 is the x*x.

I know it is the precision problem of display and calculation.

I can set up a threshold to make the result 0.

Are there other better ways ?

Thanks

Upvotes: 0

Views: 150

Answers (2)

Jaroslav Kubacek
Jaroslav Kubacek

Reputation: 1447

it's because square of 3389774.583 is not 11490571723552.8. If you will try windows calculator you will get value around 11490571723552.823889.

So there is no chance to get result 0 for (11490571723552.8 - x * x) where x is 3389774.583

Try

double x = 3389774.583;
double xx = x*x;
Console.WriteLine(xx); 

string ss = "x is " + x + " x square is " + x * x + " , 11490571723552.8 - x*x is " + (xx - x * x) + "\n";
Console.WriteLine(ss); 

Upvotes: 1

Gusdor
Gusdor

Reputation: 14332

Try rounding the output to a number of decimals with a format string:

Console.WriteLine(ss.ToString("F2"));

Upvotes: 0

Related Questions