CODEWITHSUNDEEP
luaprecision
Velissel
Velissel

Reputation: 77

Losing precision in Lua

I have a function in lua, which is given 2 vectors, return the lambda multiplier of first vector to second one, here is my code

function Math.vectorLambda( v1,v2 )
    local v1Length,v2Length=math.sqrt(v1.x^2+v1.y^2),math.sqrt(v2.x^2+v2.y^2)
    if v1Length==0 then
        return nil
    else
        local dotProduct=v1.x*v2.x+v1.y*v2.y
        print(dotProduct,v1Length,v2Length,math.abs(dotProduct)==(v1Length*v2Length))
        if math.abs(dotProduct)==(v1Length*v2Length) then
            if v1.x~=0 then
                return v2.x/v1.x
            else
                return v2.y/v1.y
            end
        else
            return nil
        end
    end
end

However, if

--this is what I get from terminal and I believe that it does not display the whole number--
    v1={0.51449575542753,-0.85749292571254}
    v2={-10,16.666666666667}

the output is

-19.436506316151    1   19.436506316151 false

which is saying the absolute value of dotProduct and v1Length*v2Length are not the same... What is the reason for above, rather than I am blind? :(

BTW, the function is not stable..with exactly the same vectors, the function might has the same output except math.abs(dotProduct)==(v1Length*v2Length) gives true and hence return correct answer rather than nil, why?

Upvotes: 1

Views: 1106

Answers (1)

etandel
etandel

Reputation: 381

Floats are tricky. You most likely have differences on the smaller decimal places (I don't know for sure, since I get true here). Try printing the numbers with a bigger precision, using a function like:

function prec_print(v, prec)
  local format = '%.' .. prec .. 'f'
  print(string.format(format, value))
end

In any case, you should almost never use == to compare floating point equality. For floats, it's quite easy to get false for simple things like a+b-b==a. What you should probably do is to check whether de difference of the two values is less than some threshold:

function almost_equal(float1, float2, threshold)
  return math.abs(float1 - float2) <= threshold
end

But it's actually trickier than that (if, say, float1 and float2 are too far apart). Anyway, this read is mandatory for anyone working with floats: http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html

Cheers!

Upvotes: 1

Related Questions