CODEWITHSUNDEEP
vb.net
MM1
MM1

Reputation: 944

Comparing decimals of given numbers VB.NET

I'm trying to see if two given numbers, for example 1.134 and 1.135 have the same decimals. So far everything works correctly, but when working with number with a large number of decimals I have a problem: I get the following error for the following input 0.841666666666667, 0.841468253968254:

    Unhandled Exception:
System.OverflowException: Arithmetic operation resulted in an overflow.

This is my code

Function TrueOrFalse(ByVal x As Double, ByVal y As Double, ByVal n as Integer) As Boolean
   
    For i As Integer = 1 To n+1
        
        If  CInt(x) <> CInt(y) Then
            Return False
        End If
        x *= 10
        y *= 10
    Next
    Return True
End Function

I understand that it's because the Int type cannot contains that much digits, and when changing CInt to CLng everything works perfectly. Problem: I cannot use CLng. Is there any alternatives ? Thanks!

Upvotes: 1

Views: 553

Answers (2)

Heinzi
Heinzi

Reputation: 172478

If you exceed the range of an Integer, don't use an Integer. You can truncate Doubles just fine using Math.Truncate.

Using that built-in functionality, your method could be reduced to the following one-liner:

Function AreEqualUpToNDecimalPlaces(ByVal x As Double, ByVal y As Double, ByVal n as Integer) As Boolean
    Return Math.Truncate(x * (10.0^n)) = Math.Truncate(y * (10.0^n))
End Function

Test cases:

Console.WriteLine(AreEqualUpToNDecimalPlaces(1.134, 1.135, 2))  ' True
Console.WriteLine(AreEqualUpToNDecimalPlaces(1.134, 1.135, 3))  ' False

Console.WriteLine(AreEqualUpToNDecimalPlaces(0.841666666666667, 0.841468253968254, 3))  ' True
Console.WriteLine(AreEqualUpToNDecimalPlaces(0.841666666666667, 0.841468253968254, 4))  ' False

Upvotes: 0

Heinzi
Heinzi

Reputation: 172478

You could "cut off" the integer part after you compared it for equality, since you already know that this part is equal, i.e., instead of

CInt(1.134) = CInt(1.135)
CInt(11.34) = CInt(11.35)
CInt(113.4) = CInt(113.5)
etc.

you just compare

CInt(1.134) = CInt(1.135)
CInt(1.34) = CInt(1.35)
CInt(3.4) = CInt(3.5)

That way, your values won't get too large.

How to do that? Before multiplying x and y by ten, you remove the integer part, either straightforward with

x = x - CInt(x)
y = y - CInt(y)

or by abusing the Mod operator:

x = x Mod 1
y = y Mod 1

A few side notes:

  • TrueOrFalse is not a great name for your method, since it does not describe what the method does. AreEqualUpToNDecimalPlaces would be a better name.
  • Beware of floating point imprecisions when using the Double data type.
  • In case this is not a toy problem for educational purposes, note that there are simpler solutions to your concrete problem (I have added one example as a separate answer).

Upvotes: 1

Related Questions