CODEWITHSUNDEEP
javascript
Ksthawma
Ksthawma

Reputation: 1285

Does toPrecision behave like round then toString?

The return value of toPrecision is "A string ... in fixed-point or exponential notation rounded to precision significant digits."

8.235 => "8.23" is not rounding. Why different?

var toP = function (n) {
    console.log(n, n.toPrecision(3) );
}
    
toP(1.235);  // 1.235 "1.24"
toP(2.235);
toP(3.235);
toP(4.235);
toP(5.235);
toP(6.235);
toP(7.235);
toP(8.235);  // 8.235 "8.23"  why?
.as-console-wrapper{min-height:100%}

Upvotes: 5

Views: 308

Answers (1)

Niet the Dark Absol
Niet the Dark Absol

Reputation: 324650

As always, it's floating point (in)accuracy that is to blame.

1.235 in binary is 1.0011110000101000111101011100001010001111010111000011...
Truncated and converted back to decimal, you get 1.235000000000000098 which does indeed round up.

But 8.235 is 1000.0011110000101000111101011100001010001111010111...
Truncated and converted back to decimal, you get 8.234999999999999432, which rounds down.

Upvotes: 3

Related Questions