CODEWITHSUNDEEP
objective-c
Ben Robinson
Ben Robinson

Reputation: 1745

Double Over Precision

Apologies if this has been asked already - not sure what to search for.

Simple bit of code:

double x = 4505;
double y = 1000;
double z = 1000000;
double result = (x * y) / z;

Answer should be 4.505; but I get:

result = 4.5049999999999999

The values of x, y and z could be anything, and sometimes I need that level of precision in the result but I can't see why this is happening.

The Question is how to I remove the rounding error so that I can re-run further calculations on the decimal value without getting erroneous results and at the same time maintain high level of precision for numbers that need it.

Upvotes: 1

Views: 36

Answers (1)

Droppy
Droppy

Reputation: 9721

It's simply a Floating Point Rounding Error. Also there is this.

If you want result rounded to 3 decimal places, then use:

result = floor(result * 1000.0) / 1000.0;

or just during presentation:

NSLog(@"result = %.3f", result);

Upvotes: 1

Related Questions