CODEWITHSUNDEEP
sql-serverfloating-point
dtc
dtc

Reputation: 10296

How can a sum of a column in sql server where values are 2 decimal places result in 0000000000087?

I have a table that contains a float column.

There are many records in that table.

When I do a Group By it shows that the float column has only the values: 0.1 0.2 0.25 0.5 1 2 3

But when I do a SUM it results in: 58.350000000000087

Where does the 0000000000087 come from? Is it because I'm using a float type instead of a decimal/numeric? I read some forum posts about this but the math hurts my head. What would be the correct way to get accurate results?

Upvotes: 1

Views: 3557

Answers (2)

Eilon
Eilon

Reputation: 25704

IEEE 754 (and related) floating point numbers cannot accurately represent many decimal values. When you perform arithmetic operations, such as summation, you sometimes end up with values that cannot be well represented, and the error occurs.

Please see this article for more info: What Every Computer Scientist Should Know About Floating-Point Arithmetic

Depending on the SQL server you are using, there are numeric types that are precise for decimal values, and those that aren't. For example, as you have found, float is not precise for decimals. However, money and decimal are.

Upvotes: 11

quant_dev
quant_dev

Reputation: 6231

0.1 and 0.2 cannot be expressed precisely as binary floating-point numbers, hence you get a small "surplus".

Upvotes: 2

Related Questions