CODEWITHSUNDEEP
powershellpowershell-2.0
M Jeremy Carter
M Jeremy Carter

Reputation: 463

PowerShell math wrong

Why does PowerShell seem to give me wrong answers sometimes when the answer is between 0 and 1?

PS C:\users\me> 31.2 - 31
0.199999999999999   <<< wrong
PS C:\users\me> 31.2 - 30
1.2                 <<< right
PS C:\users\me>  31.2 % 10
1.2                 <<< right
PS C:\users\me> 30.2 % 10
0.199999999999999   <<< wrong
PS C:\users\me> 30.2 / 151
0.2                 <<< right

Upvotes: 2

Views: 469

Answers (2)

Keith Hill
Keith Hill

Reputation: 201672

Floating numbers are represented using base-2 numbers (mantissa and exponent). As a result there are often rounding errors like this that show up from time to time. You can read more about this here: http://randomascii.wordpress.com/2012/02/25/comparing-floating-point-numbers-2012-edition/

Another way to get a more satisfying result is to limit the precision of the displayed number e.g.:

c:\PS> "{0:F4}" -f (31.2 - 31)
0.2000

Upvotes: 3

MatthewMartin
MatthewMartin

Reputation: 33143

You need to use [decimal] to get man-in-the-street math.

[decimal]31.2 - [decimal]31

or 

31.2d - 31

Numeric literals default to double, which can't represent many decimals (because they require repeating fractions, involve rounding errors, etc)

ref: https://devcentral.f5.com/articles/powershell-abcs-t-is-for-type-literals

Upvotes: 7

Related Questions