Mathias Rönnlund
Mathias Rönnlund

Reputation: 4817

Kusto range query in Azure Data Explorer not using even steps

This query

range x from -1 to 1 step 0.2
| project x

produces this result. Can this be correct?

enter image description here

Upvotes: 2

Views: 481

Answers (2)

Slavik N
Slavik N

Reputation: 5298

Working with numbers with a floating point can’t be expected to be 100% precise.

I wrote a quick C program that starts with float f = 0.0f and then runs f += 0.1 100 times, while printing f every 10 iterations, and the result is:

1.000000
2.000000
2.999999
3.999998
4.999998
5.999997
6.999996
7.999995
8.999998
10.000002

You can see/modify/run see the code I wrote here: https://onlinegdb.com/HyylownHd

The lack of precision may happen not only in the arithmetic calculations, but also in the code that converts the float number to a string (to pass between layers or just before printing it).

As @avnera suggested, using decimal indeed gives you better precision, but you still shouldn't expect it to be 100% precise.

Upvotes: 2

Avnera
Avnera

Reputation: 7608

Yes, the type that is inferred based on the values provided is double (real) which is a floating point. If you would like to see precise numbers use the decimal type, for example:

range x from decimal(-1) to decimal(1) step decimal(0.2)

Upvotes: 2

Related Questions