CODEWITHSUNDEEP
luanumbersintegerdivision
ByoTic
ByoTic

Reputation: 333

Lua interprets integer as non-integer number

For some reason Lua doesn't interpret integers as integers if I do the following: (integer%1 should be 0 but in this example I'll show it is actually

x = 4
for i=1,25 do
  x = x - 0.04
end

print(x) -- 3
print(x%1) -- 1
print(math.ceil(x) == x) --false
print(math.ceil(x)) -- 3

Probably it is a bug, but is there a way to evade it? this is really important to me. (and no, I can't use math.ceil(x) because I have in the way numbers such as 2.4 that I don't want to interpret as integers...

Upvotes: 0

Views: 1525

Answers (1)

hugomg
hugomg

Reputation: 69934

This is not a Lua specific problem and almost every other programming language is going to exhibit similar behaviour.

Floating point numbers are only able to precisely represent multiples of powers of two (this also includes integers). Since 0.04 (1/25) is not a power of two, your intermediate values (including 2.4) are not able to be precisely represented by floating point numbers so there will be some small roundoff error. They will look all right when you print them because the printing algorithm writes down the nearest decimal but the internal representation will not be exactly 2.4

I would suggest refactoring your programs to use integers instead of fractional numbers. For your internal representation, keep everything multiplied by 25 (so 2.4 would be 60), and increment and decrement by 1 instead of 0.04. When you actually need to use the values you can divide them by 25 to get the inexact floating point representation.

BTW, try printing x-3. This will reveal that x is slightly smaller than 3

Upvotes: 4

Related Questions