CODEWITHSUNDEEP
kdb+
cjm2671
cjm2671

Reputation: 19486

Comparing floats in KDB?

I'm trying to compare numbers in KDB.

If I do:

exp 0.6931472

I get 2f as a response.

If I do (exp 0.6931472) = 2, I get 0b when I expect 1b.

What am I doing wrong?

Upvotes: 0

Views: 468

Answers (1)

SeanW17
SeanW17

Reputation: 56

The 2f that is initially returned does not mean exactly 2f, and adjusting the precision showcases this.

q)\P 10
q)exp 0.6931472
2.000000039

You can use -P command line arg to set precision within the session, which should make things clearer.

https://code.kx.com/q/basics/cmdline/#-p-display-precision

Upvotes: 4

Related Questions