CODEWITHSUNDEEP
vbaexcelif-statement
JorgeSilva
JorgeSilva

Reputation: 35

How to see what the is being compared in a if statement

I'm having a problem with some vba code. I have a if statement that doesn't treat the same content equally. e.g: 0,1 equals 0,1, but a re-run 0,1 does not equal 0,1 (this values are shown by MVBA)

The code is long so before posting it i would like to know if it's possible to see the machine perspective in a if statement (hex, ascii...). This because, although the debug is telling me they are the same (through msgbox, vartype, etc), the if statement is not activated.

    pseudo code:
    x = 0,0000001 * 1*10^6 (which equals 0,1)
    y = 0,0001 * 1*10^3 (which also equals 0,1)

    if statement:
         x doesn't enter
         y does
    end if

Upvotes: 2

Views: 58

Answers (1)

z̫͋
z̫͋

Reputation: 1571

This is because the floating-point implementation may not be able to represent those number accurately due to the fact that they are encoded in a base 2 representation.

If you want to compare them, I would suggest using Cdec (wich converts to Decimal, a VBA custom base 10 floating-point)

Debug.Print (0.0000001 * 1 * 10 ^ 6) = (0.0001 * 1 * 10 ^ 3) ' False
Debug.Print CDec(0.0000001 * 1 * 10 ^ 6) = CDec(0.0001 * 1 * 10 ^ 3) ' True

While they both display 0.1, in fact 0.0000001 * 1 * 10 ^ 6 flaoting-point value is 0x3FB9999999999999 whereas 0.0001 * 1 * 10 ^ 3 returns 0x3FB999999999999A.

I'd recommend reading What Every Computer Scientist Should Know About Floating-Point Arithmetic

Upvotes: 2

Related Questions