CODEWITHSUNDEEP
vbscriptasp-classicnumeric
K. Weber
K. Weber

Reputation: 2773

Numeric overflow in ASP

I'm revisiting an old classic ASP application to add some new requisites, one of them needs to perform an aritmethic operation to generate an id according to some standard rule for bank files which:

The number I get as result of first operation is 1185626240142800 (many many magnitudes over max long number), I can convert to Double but then I can't get mod 97.

This rule is a standard for bank files so there is no workaround, I need to perform these operations.

In my Windows 7 64 bits I saw I can perform this operation in SQL Server, so as a bad hack it's not a bad idea, any other ideas?

BTW, this app will run on a Windows server 2003 32 bits, in case you think this will be an added problem

Upvotes: 0

Views: 452

Answers (1)

Salman Arshad
Salman Arshad

Reputation: 272276

Numbers larger than 2,147,483,647 cannot fit inside the Long datatype. You can implement the algorithm described in this article:

Pseudo code:

To find nBig % a:

Let d = the number of digits that can be processed at a time.

Repeat while val( nBig ) >= a
1. tmpStr := d digits from the LHS of nBig.
2. nBig := Remaining portion of nBig
3. tmpNum := toInteger( tmpStr )
4. tmpNum := tmpNum % a
5. tmpStr := toString( tmpNum )
6. nBig := tmpStr + nBig

Selecting d:

d should satisfy following constraint

max no. of digits in Mod a < d <= max. digits in tmpNum - max no. of digits in Mod a
(For eg: If a = 512 (Mod ranges 0..511), and tmpNum can store max 16 digits, then 3 < d <= 16-3)
(For eg: If a = 100 (Mod ranges 0..99), and tmpNum can store max 16 digits, then 2 < d <= 16-2)

VBScript/VBA Code:

Function LongMod(strDividend As String, numDivisor As Long) As Long

    Dim d As Long
    Dim strTemp As String
    Dim numTemp As Long

    d = 9 - Len(CStr(numDivisor - 1))

    While CDbl(strDividend) >= numDivisor
        strTemp = Left(strDividend, d)
        numTemp = CLng(strTemp) Mod numDivisor
        strDividend = CStr(numTemp) & Mid(strDividend, d + 1)
    Wend

    LongMod = CLng(strDividend)

End Function

LongMod("1185626240142800",               97) ' 21
LongMod("330542811101000000123456182900", 97) ' 38

Upvotes: 1

Related Questions