CODEWITHSUNDEEP
javascriptfloating-pointnumbers
Vinnie Gold
Vinnie Gold

Reputation: 49

What is the best way to deal with floating point errors?

So I have a Javascript script which adds small fractions together in a loop and its possible that it will add 0.2 to 0.1. Then that value is fed to another function but the issue is I need 0.3 to be fed exactly and not 0.30000000000000004.

What is the simplest way to ensure the number is corrected and exact. Note that its possible to get 0.25+0.125 etc being added to simply rounding to 1 decimal won't fix the issue.

It is also possible to have 0.2 + 0.10000000000000004 being added although this is very, very unlikely!

Upvotes: 3

Views: 7582

Answers (2)

Alex Barra
Alex Barra

Reputation: 11

What I use is this:

/// My value with floating point error
float fValue = 0.5000001;
fValue = (int)(fValue * 100) / 100;

You may adjust your float precision with changing '100' value.

And you can always put this on a nice function :D

Note: I use this in Unity Engine (C#).

Upvotes: 0

George Bahij
George Bahij

Reputation: 627

There is no simple method of avoiding rounding errors in general-purpose floating-point arithmetic. The number 0.3 does not have an exact binary floating-point representation.

I would suggest reading What Every Computer Scientist Should Know About Floating-Point Arithmetic to familiarize yourself with the trade-offs inherent to floating-point representation of numbers.

To actually solve your issue, you should ask yourself a few questions:

  • How strict is your requirement for precision? Why is 0.30000000000000004 outside your margin for error? Is it acceptable to round your results?
  • Is there any way you could represent your numbers and perform most of your arithmetic with integers? E.g. if you know that you'll only encounter rational numbers, they can be represented using an integer quotient and an integer denominator. From there, you can attempt to defer casting to float for as long as possible to prevent cumulative rounding errors.
  • If you cannot perform your calculations on integers, is there an alternate datatype you can use, such as BigDecimal?

Ultimately, when it comes to issues with floating-point precision, you'll often have to tailor the solution to the requirements posed by your specific issue.

Upvotes: 2

Related Questions