CODEWITHSUNDEEP
javascriptfloating-pointprecisionequalityfloating-point-precision
kjo
kjo

Reputation: 35321

How to test whether a float is an integer "within machine precision"?

The brief version of my question is:

What's considered "best practice" for deciding when a floating point number x and Math.round(x) may be considered equal, allowing for loss of precision from floating-point operations?

The long-winded version is:

I often need to decide whether or not a given floating point value x should be "regarded as an integer", or more pedantically, should be "regarded as the floating point representation of an integer".

(For example, if n is an integer, the mathematical expression

log10(10n)

is a convoluted way of representing the same integer n. This is the thinking that motivates saying that the result of the analogous floating-point computation may be regarded as "the representation of an integer".)

The decision is easy whenever Math.round(x) == x evaluates to true: in this case we can say that x is indeed (the floating point representation of) an integer.

But the test Math.round(x) == x is inconclusive when it evaluates to false. For example,

function log10(x) { return Math.log(x)/Math.LN10; }
// -> function()
x = log10(Math.pow(10, -4))
// -> -3.999999999999999
Math.round(x) == x
// -> false

EDIT: one "solution" I often see is to pick some arbitrary tolerance, like ε = 1e-6, and test for Math.abs(Math.round(x) - x) < ε. I think such solutions would produce more false positives than I'd find acceptable.

Upvotes: 3

Views: 337

Answers (3)

Richard Connamacher
Richard Connamacher

Reputation: 3216

That's a fun one, so I had to think a bit about it.

Here's a one-line solution, though it involves converting between number and string types so I don't know how optimal it is. But it will be far more accurate than just picking a minimum threshold and checking if the number's within that limit.

JavaScript numbers are double-precision 64-bit format, which has about 16 decimal digits of precision. That's total digits, not just the number of digits to the right of the decimal point.

JavaScript numbers also have a toPrecision() method that converts them to strings, rounded to a given precision (total digits, so good for your use). The following will round any number to the nearest 15 digits of precision, and then convert it back to a float.

function roundToPrecision(number, precision) {
    return parseFloat(number.toPrecision(precision));
}

x = roundToPrecision(x, 15);

Then your example will, in fact, be an integer: -4.

Edit: After some more thinking this will be way faster:

var integerDigits = (""+parseInt(Math.abs(x))).length,
    threshold = 1e-16 * Math.pow(10, integerDigits);

Math.abs(Math.round(x) - x) < threshold

http://jsperf.com/number-precision-rounding

Upvotes: 1

Patricia Shanahan
Patricia Shanahan

Reputation: 26185

Assuming x is non-zero, I think you should be looking at the ratio Math.abs(x-Math.round(x))/x. That deals with the fact that the floating point types each store a fixed number of significant bits, not a fixed number of digits after the decimal point.

Next, you need to determine the typical rounding error for your calculations. If x is the result of simple calculation that may be easy. If not, consider collecting some statistics from test cases for which you know the exact answer. Ideally, you will find that there is a significant difference between the largest value of the ratio for x integer, and the smallest value for x that should not be treated as an integer. If so, pick an epsilon in that range.

Upvotes: 1

Andreas Hagen
Andreas Hagen

Reputation: 2345

As you see in your example x is in fact not an integer at all. This is due to round-off errors earlier in your calculations, and thus you can in fact not know whether x was defined to be a nearly round number or a round number that got jagged up by round-off errors.

If you want to know what numbers is one or the other you will need to use the limit aproach you suggested yourself or use a high enough persicion that your numbers don't get jagged up in the first place. This last approach is not applicable in all cases.

There is also the possibility to track all mathematical operations symbolically, i.e. store 1/3 as 1/3 rather than 0.3333 and evaluate them on demand canceling out factors that can be canceled like you would when evaluating an expression by hand, but this is in almost all cases totally overkill. Not to mention how complex such a system would be. If this is the desired solution you can probably interface with MatLab or Mathematica or something to handle the evaluating, unless you are running this in a browser where it might be somewhat harder to do for browsers try the WolframAlpha API (why did't i think of that the first time?).

Nevertheless; If you can solve this problem by choosing ε in such a way you get a satisfactory result that is probably the best way to do it. If a static ε don't cut it you could try selecting it dynamically based on what type of calculations was done earlier to the number at hand. I.e. Numbers that are multiplied tend to create less of a fraction part than numbers that are divided, and so on. In cases where the number has not been subject to anything else than plus, minus and multiplication (not involving fractions) you can know how many decimal places it maximally can have as well and thus can pick a reasonable ε.

Upvotes: 5

Related Questions