CODEWITHSUNDEEP
rubyfloating-pointmodulo
Tony
Tony

Reputation: 19181

In ruby, why is "100.7".to_f.modulo(1) = 0.700000000000003?

This is very strange to me:

irb(main):012:0> "100.7".to_f.modulo(1)
=> 0.700000000000003

Why the 3 at the end?

irb(main):019:0> "10.7".to_f.modulo(1)
=> 0.699999999999999

Same thing here...we are only getting the remainder of this value divided by one. It should be exact.

Upvotes: 1

Views: 580

Answers (5)

tadman
tadman

Reputation: 211750

Welcome to floating point math. There are many numbers which cannot be represented in standard floating point notation and come out just a tiny bit off.

This is easily illustrated as follows:

(1..10).collect do |i|
  v = ((10**i).to_f + 0.7)
  puts "%13.1f = %.30f" % [ v, v.modulo(1) ]
end

Where the result is:

         10.7 = 0.699999999999999289457264239900
        100.7 = 0.700000000000002842170943040401
       1000.7 = 0.700000000000045474735088646412
      10000.7 = 0.700000000000727595761418342590
     100000.7 = 0.699999999997089616954326629639
    1000000.7 = 0.699999999953433871269226074219
   10000000.7 = 0.699999999254941940307617187500
  100000000.7 = 0.700000002980232238769531250000
 1000000000.7 = 0.700000047683715820312500000000
10000000000.7 = 0.700000762939453125000000000000

Notice that the larger the number gets, the lower the precision beyond the decimal place. This is because there is a fixed amount of precision available to represent the entire number.

Upvotes: 2

leeeroy
leeeroy

Reputation: 11456

Floating points are not exact. The short version is it's not possible to store an infinite amount of values in a finite amount of bits.

The longer version is What Every Computer Scientist Should Know About Floating-Point Arithmetic

Upvotes: 1

sepp2k
sepp2k

Reputation: 370465

What Every Computer Scientist Should Know About Floating-Point Arithmetic

Upvotes: 13

Joe Koberg
Joe Koberg

Reputation: 26769

This is typical floating point rounding. You simply cannot express every single decimal number in the fixed number of bits in a Float, so some values are rounded to the nearest value that can be represented.

Due to this, it is suggested that you don't compare Floats for equality. Compare for less than or greater than, but never exact equality.

http://en.wikipedia.org/wiki/Floating_point#Representable_numbers.2C_conversion_and_rounding

Simply, it is not the case that "it should be exact". Don't expect that from floating point decimals.

Upvotes: 3

Peter
Peter

Reputation: 132417

this is because it's not possible to represent all floating point numbers exactly.

Upvotes: 0

Related Questions