CODEWITHSUNDEEP
assemblyx86stack-overflowintel-fortranstdcall
Paul
Paul

Reputation: 144

Stack overflow occurs if I call an assembler routine too many times

I pruned everything to the min to show this problem

In Compaq Visual Fortran I write

    Do 1 I =1,N
    CALL SUBNAM(A,B,100)
1   CONTINUE

If "SUBNAM" is any routine written in Fortran , the DO loop count N can be as big as you like. N=100000 is no problem.

If SUBNAM is an assembler (Goasm) routine, even as simple as

_SUBNAM@12:  Push Ecx
             Pop  Ecx
             Ret

then N greater than about 86000 gives stack overflow. Even though the stack pointer is NOT corrupted!

If the assembler routine is only

_SUBNAM@12: RET

I do not get stack overflow with N as high as 100000

WTF???

Information: Fortran "decorates" subroutine names with a leading underscore and @mm at the end where mm is 4 times the number of arguments. This is the amount of stack in bytes it uses for pushing argument addresses. It is the calling program's responsibility to clear up the stack upon being returned to. I have peeked at the stack before and after the call to SUBNAM and the stack pointer does return to the pre-Call value.

I have disabled all traceback compiler options in case debugging info was being crested at every call. It may be, and maybe a Fortran SUBNAM includes code to erase the traceback info that the call to it created?

Does the Pentium processor have another stack that could be overflowing, other than ESP?

Upvotes: 1

Views: 105

Answers (0)

Related Questions