CODEWITHSUNDEEP
excelvba
Michael
Michael

Reputation: 507

Excel VBA odd maths result

I'm having a problem with some logic statements in a Case Select. However, I've tracked it down to some odd maths by excel. The following formula is evaluating as True in the immediate window when I would expect it to be false

?(0.09 /0.1)<0.9

Can anyone explain why this would be?

Upvotes: 0

Views: 110

Answers (2)

Richard Morgan
Richard Morgan

Reputation: 7691

Floating point arithmetic may give inaccurate results in Excel.

For a more in-depth article, read What Every Computer Scientist Should Know About Floating-Point Arithmetic.

Here's a related question right here on StackOverflow.

@scott has the solution in his comment:

? round((0.09/0.1),1)<0.9

Upvotes: 2

Ahmad
Ahmad

Reputation: 12737

This might be because vb converts the expression to integer which resolves to zero. Try 

   ?  CDBL((0.09/0.1) < 0.9

Upvotes: 0

Related Questions