CODEWITHSUNDEEP
c#vb6calculator
Michael Cook
Michael Cook

Reputation: 1707

Confused by calculation

I am porting an application from VB6 to C#. I have found one calculation in particular that is causing me an issue. It basically boils down to

opperandA *.01 / opperandB

My concrete example is:

1 * .01 / 12

In VB6 (and Windows Calculator) I get 8.3333333333e-4.

However, in C# (and every other Calculator) I get .00083333.

The second number makes sense to me, but I have to replication the first result and I want to understand it, so why does VB6 and Windows calculator produce an odd result?

Upvotes: 0

Views: 100

Answers (1)

Matthew Cheale
Matthew Cheale

Reputation: 106

8.3333333333e-4 is the same as 0.00083333. It equates to:

8.3333333333 * 10^-4
= 8.3333333333 times ( ten to the power of -4 )
= 8.3333333333 * 0.0001
= 0.00083333333
N.b. After rounding

The e stands for exponent and the relevant Wikipedia article is http://en.wikipedia.org/wiki/Exponentiation

Upvotes: 5

Related Questions