Frank_Vr
Frank_Vr

Reputation: 659

simple subtraction in Visual studio with incorrect result

doing a simple subtraction such as this:

   Dim a, b As Double
   a = 81.4
   b = 76.0
   MsgBox(a - b)

you would expect it to result in 5.4 but instead you get 5.4000000000000001 what is going on?

Upvotes: 0

Views: 214

Answers (1)

Chuck Walbourn
Chuck Walbourn

Reputation: 41077

floating point numbers, even double precision floating point numbers, are not 'real' numbers which have infinite precision.

See What Every Computer Scientist Should Know About Floating-Point Arithmetic

Bruce Dawson's blog

Wikipedia

Upvotes: 2

Related Questions