CODEWITHSUNDEEP
kdb+
Rahul
Rahul

Reputation: 3969

Abrupt behaviour of KDB Real type

I am having an issue when comparing float and real. Here are some examples:

KDB version:2.7 (2012-02-20)

  1. 4.7e = 4.7f
    output: false

    Reason: Kdb converts 4.7e to 4.6999998e

    so, 4.7e < 4.7f ---> ouput:true

  2. 4.347e = 4.347f

    output:false

    Reason: 4.347e gets converted to 4.3470001e

    so, 4.347e>4.347f ---> output:true

I am not sure what is going on here. More examples you try,more varieties you'll get in answer.

Upvotes: 1

Views: 170

Answers (1)

terrylynch
terrylynch

Reputation: 13657

There's also some specific info on the kdb wiki (not sure if you've already seen them)

http://code.kx.com/q/cookbook/precision/#float-precision

http://code.kx.com/q/cookbook/precision/#comparison-tolerance

Upvotes: 1

Related Questions