CODEWITHSUNDEEP
ruby-on-railsrubyprintfrounding
tsdbrown
tsdbrown

Reputation: 5058

Odd rounding problem using the ruby printf-format-specifier

Has anybody got any ideas on this one?

When we run:

printf("%.0f", 40.5)

On a windows box the return is "41" but on our production ubuntu server we're getting "40"

Upvotes: 1

Views: 1111

Answers (4)

Rick Regan
Rick Regan

Reputation: 3512

ScottJ's answer does not explain your problem -- 40.5 is exactly representable in binary. What's probably happening is this: Windows' printf "rounds half up," and Ubuntu's printf "rounds half even."

Upvotes: 1

aykoc
aykoc

Reputation: 11

use %.0g instead

Upvotes: 1

ScottJ
ScottJ

Reputation: 1092

Looks like a simple case of binary floating point imprecision. On one machine you get 40.499999999 which rounds to 40; on the other machine you get 40.500000000001 which rounds to 41.

If you need exact numbers then you should not use binary floating point. You can use fixed point decimal, or decimal floating point.

Edit: you're using BigDecimal, you say. Why not avoid any conversion to float by using #round and then #to_i? (Or #floor or #ceil instead of #round... it's not clear what your goal is.)

b = BigDecimal.new("40.5")
print b.round.to_i  # => 41

Upvotes: 0

ujh
ujh

Reputation: 4083

How about using .round instead? Rails even enhances it so that you can specify the precision (see API doc).

Upvotes: 2

Related Questions