CODEWITHSUNDEEP
vb.netvb.net-2010
Albert Tobing
Albert Tobing

Reputation: 169

VB.NET Single data type calculation issue

I want to perform a basic calculation with fractional numbers using vb.net.

Dim a As Single= 7200.5
Dim b As Single= 7150.3
Dim c As Single= a - b

'Expected result = 50.2
MsgBox(a.ToString + " - " + b.ToString + " = " + c.ToString.Trim)
'Produced result is: 50.2002

Dim single1 As Single
Dim single2 As Single
Dim single3 As Single

single1 = 425000
single2 = 352922.2
single3 = single1 - single2

'Expected result is: 72077.8
MsgBox(single3.ToString)
'Produced result is: 72077.81

How can the results be so inaccurate for such a simple calculation? The problem is solved when I change the data type to Decimal, but Decimal objects consume more memory (16 bytes). Is there any alternative data type that i can use to perform simple fractional calculations with accurate results?

Upvotes: 3

Views: 9108

Answers (3)

Serge Bollaerts
Serge Bollaerts

Reputation: 324

You can use the Decimal data type instead. It will work great! This is because Decimal is a fixed point value, whereas Single and Double are floating point values (with loss of precision).

Upvotes: 0

Steven Doggart
Steven Doggart

Reputation: 43743

The Single and Double data types are not precise. They use the floating point method to store their values. Floating points use less memory and allow for faster calculations, but they are imprecise. That is the trade-off that you have to accept if you are going to use them. If precision is important, then they are not an option for you. Decimal is precise (to a certain number of fractional digits, that is), so usually, that is the best choice for precise fractional numbers in most cases. If you really need to save memory, and you are guaranteed that your numbers will be within a certain range, then you could use an Int16, Int32, or Int64 instead. For instance, if you only care about two fractional digits, you could simply multiply everything by 100 and then just divide by 100 (using Decimal types for the division) before displaying it. In that way, you can store many numbers and perform many operations using less memory, and only need to use the Decimal data type when you need to display a result.

Dim a As Integer = 720050 '7200.5
Dim b As Integer = 715030 '7150.3
Dim c As Integer = a - b
Dim cDisplay As Decimal = CDec(c) / CDec(100)
MessageBox.Display(String.Format("{0} - {1} = {2}", a, b, c))

Upvotes: 0

Sean Airey
Sean Airey

Reputation: 6372

This is to do with the way floating point numbers are stored in memory, and a Single in .Net is a single precision floating point number, which is much less accurate than a Decimal or a Double for storing decimal numbers.

When the computer calculates your number, it only has binary fractions to use and in a single precision floating point number, they're not very accurate.

See http://en.wikipedia.org/wiki/Single-precision_floating-point_format for more information.

EDIT: There's some more information specific to VB.Net here: http://msdn.microsoft.com/en-us/library/ae382yt8(v=vs.110).aspx

Upvotes: 5

Related Questions