CODEWITHSUNDEEP
rubyprecisionbigdecimal
Stefan
Stefan

Reputation: 114138

How to prevent BigDecimal from truncating results?

Follow up to this question:

I want to calculate 1/1048576 and get the correct result, i.e. 0.00000095367431640625.

Using BigDecimal's / truncates the result:

require 'bigdecimal'
a = BigDecimal.new(1)
#=> #<BigDecimal:7fd8f18aaf80,'0.1E1',9(27)>
b = BigDecimal.new(2**20)
#=> #<BigDecimal:7fd8f189ed20,'0.1048576E7',9(27)>

n = a / b
#=> #<BigDecimal:7fd8f0898750,'0.9536743164 06E-6',18(36)>

n.to_s('F')
#=> "0.000000953674316406" <- should be ...625

This really surprised me, because I was under the impression that BigDecimal would just work.

To get the correct result, I have to use div with an explicit precision:

n = a.div(b, 100)
#=> #<BigDecimal:7fd8f29517a8,'0.9536743164 0625E-6',27(126)>

n.to_s('F')
#=> "0.00000095367431640625" <- correct

But I don't really understand that precision argument. Why do I have to specify it and what value do I have to use to get un-truncated results?

Does this even qualify as "arbitrary-precision floating point decimal arithmetic"?

Furthermore, if I calculate the above value via:

a = BigDecimal.new(5**20)
#=> #<BigDecimal:7fd8f20ab7e8,'0.9536743164 0625E14',18(27)>
b = BigDecimal.new(10**20)
#=> #<BigDecimal:7fd8f2925ab8,'0.1E21',9(36)>

n = a / b
#=> #<BigDecimal:7fd8f4866148,'0.9536743164 0625E-6',27(54)>

n.to_s('F')
#=> "0.00000095367431640625"

I do get the correct result. Why?

Upvotes: 3

Views: 264

Answers (1)

Max
Max

Reputation: 22315

BigDecimal can perform arbitrary-precision floating point decimal arithmetic, however it cannot automatically determine the "correct" precision for a given calculation.

For example, consider

BigDecimal.new(1)/BigDecimal.new(3)
# <BigDecimal:1cfd748, '0.3333333333 33333333E0', 18(36)>

Arguably, there is no correct precision in this case; the right value to use depends on the accuracy required in your calculations. It's worth noting that in a mathematical sense†, almost all whole number divisions result in a number with an infinite decimal expansion, thus requiring rounding. A fraction only has a finite representation if, after reducing it to lowest terms, the denominator's only prime factors are 2 and 5.

So you have to specify the precision. Unfortunately the precision argument is a little weird, because it seems to be both the number of significant digits and the number of digits after the decimal point. Here's 1/1048576 for varying precision

1   0.000001
2   0.00000095
3   0.000000953
9   0.000000953
10  0.0000009536743164
11  0.00000095367431641
12  0.000000953674316406
18  0.000000953674316406
19  0.00000095367431640625

For any value less than 10, BigDecimal truncates the result to 9 digits which is why you get a sudden spike in accuracy at precision 10: at that point is switches to truncating to 18 digits (and then rounds to 10 significant digits).

† Depending on how comfortable you are comparing the sizes of countably infinite sets.

Upvotes: 3

Related Questions