Difference between ebp based addressing and esp addressing

Question

I have written some code to learn about the call stack. I've done this with some inline assembly for passing parameters on stack. I've compiled it with gcc 4.1.2(on CentOS5.4) and it works well, then I compile it with gcc 4.8.4(on Ubuntu14.04.3) and run the program but it always crashes.

I discovered that there are differences in how the variables are referenced. The local variable is addressed using the EBP register in gcc 4.1.2(CentOS5.4) while the local variable is addressed using the ESP register in gcc 4.8.4(Ubuntu14.04.3). This seems to be the reason of that it crashes.

My question is, how can I control whether gcc uses EBP or ESP? Also, what is the difference between them?

Here is the C code:

double fun(double d) {
    return d;
}

int main(void) {
    double a = 1.6;
    double (*myfun)() = fun;
    asm volatile("subl $8, %esp\n"
                 "fstpl (%esp)\n");
    myfun();
    asm volatile("addl $8, %esp\n");
    return 0;
}

Here is the assembly in gcc 4.1.2, and it works

int main(void) {
    **......**

double a = 1.6;
    0x080483bf <+17>:     fldl   0x80484d0
    0x080483c5 <+23>:     fstpl  -0x18(%ebp)

double (*myfun) () = fun;
    0x080483c8 <+26>:     movl   $0x8048384,-0xc(%ebp)

asm volatile("subl $8, %esp\n"
             "fstpl (%esp)\n");                 
    0x080483cf <+33>:     sub    $0x8,%esp
    0x080483d2 <+36>:     fstpl  (%esp)        

myfun();
    0x080483d5 <+39>:     mov    -0xc(%ebp),%eax
    0x080483d8 <+42>:     call   *%eax
    0x080483da <+44>:     fstp   %st(0)

asm volatile("addl $8, %esp\n");
    0x080483dc <+46>:     add    $0x8,%esp

    **......**

here is the assembly in gcc 4.8.4. This is what crashes:

int main(void) {
    **......**

double a = 1.6;
    0x0804840d <+9>:    fldl   0x80484d0
    0x08048413 <+15>:   fstpl  0x8(%esp)

double (*myfun)() = fun;
    0x08048417 <+19>:   movl   $0x80483ed,0x4(%esp)

asm volatile("subl $8,%esp\n"
             "fstpl (%esp)\n");
    0x0804841f <+27>:   sub    $0x8,%esp
    0x08048422 <+30>:   fstpl  (%esp)      

myfun();
    0x08048425 <+33>:   mov    0x4(%esp),%eax
    0x08048429 <+37>:   call   *%eax
    0x0804842b <+39>:   fstp   %st(0)

asm volatile("addl $8,%esp\n");
    0x0804842d <+41>:   add    $0x8,%esp
    **......**

Answer 1

Most compilers create EBP-based stack frames. Or, at least they used to. This is the method that most people are taught that utilizes using EBP as a fixed base frame pointer.

Some compilers create ESP-based stack frames. The reason is simple. It frees up EBP to be used for other uses, and removes the overhead of setting up and restoring the stack frame. It is clearly much harder to visualize, since the stack pointer can be constantly changing.

The problem you are having might be because you are calling APIs that use stdcall calling convention, which end up trashing your stack, unintentionally, when they return to the caller. EBP must be preserved by the callee by cdecl and stdcall founction. However, stdcall routines will clean up the stack with ret 4 for example, thus shrinking its size. The caller must compensate for these types of mishaps, and reallocate space on the stack after the call returns.

GCC has the option -fomit-frame-pointer which will turn off EBP-based frames. It's on by default at most optimization levels. You can use -O2 -fno-omit-frame-pointer to optimize normally except for still setting up EBP as a frame pointer.

Answer 2

There's no real difference between using esp and ebp, except that esp changes with push, pop, call, ret, which sometimes makes it difficult to know where a certain local variable or parameter is located in the stack. That's why ebp gets loaded with esp, so that there is a stable reference point to refer to the function arguments and the local variables.

For a function like this:

int foo( int arg ) {
    int a, b, c, d;
    ....
}

the following assembly is usually generated:

# using Intel syntax, where `mov eax, ebx` puts the value in `ebx` into `eax`
.intel_syntax noprefix

foo:                                                          
    push ebp          # preserve
    mov ebp, esp      # remember stack
    sub esp, 16       # allocate local variables a, b, c, d

    ...

    mov esp, ebp      # de-allocate the 16 bytes
    pop ebp           # restore ebp
    ret

Calling this method (foo(0)) would generate something like this:

    pushd 0           # the value for arg; esp becomes esp-4
    call  foo         
    add   esp, 4      # free the 4 bytes of the argument 'arg'.

Immediately after the call instruction has executed, right before the first instruction of the foo method is executed, [esp] will hold the return address, and [esp+4] the 0 value for arg.

In method foo, if we wanted to load arg into eax (at the ...) we could use:

    mov eax, [ebp + 4 + 4]

because [ebp + 0] holds the previous value of ebp (from the push ebp), and [ebp + 4] (the original value of esp), holds the return address.

But we could also reference the parameter using esp:

   mov eax, [esp + 16 + 4 + 4]

We add 16 because of the sub esp, 16, then 4 because of the push ebp, and another 4 to skip the return address, to arrive at arg.

Similarly accessing the four local variables can be done in two ways:

  mov eax, [ebp -  4]
  mov eax, [ebp -  8]
  mov eax, [ebp - 12]
  mov eax, [ebp - 16]

or

 mov eax, [esp + 12]
 mov eax, [esp +  8]
 mov eax, [esp +  4]
 mov eax, [esp +  0]

But, whenever esp changes, these instructions must change aswell. So, in the end, it does not matter whether esp or ebp is used. It might be more efficient to use esp since you don't have to push ebp; mov ebp, esp; ... mov esp, ebp; pop ebp.

---

UPDATE

As far as I can tell, there's no way to guarantee your inline assembly will work: the gcc 4.8.4 on Ubunty optimizes out the use of ebp and references everything with esp. It doesn't know that your inline assembly modifies esp, so when it tries to call myfun(), it fetches it from [esp + 4], but it should have fetched it from [esp + 4 + 8].

Here is a workaround: don't use local variables (or parameters) in the function where you use inline assembly that does stack manipulation. To bypass the problem of casting double fun(double) to double fn() call the function directly in assembly:

void my_call() {     
    asm volatile("subl $8, %esp\n"
                 "fstpl (%esp)\n"
                 "call  fun\n"
                 "addl $8, %esp\n");
}

int main(void) {
    my_call();
    return 0;
}

You could also place the my_call function in a separate .s (or .S) file:

.text
.global my_call
my_call:
    subl  $8, %esp
    fstpl (%esp)
    call  fun
    addl  $8, %esp
    ret

and in C:

extern double my_call();

You could also pass fun as an argument:

extern double my_call( double (*myfun)() );
...
     my_call( fun );

and

.text
.global my_call
my_call:
    sub  $8, %esp
    fstp (%esp)
    call *12(%esp)
    add  $8, %esp
    ret

Answer 3

ebp is normally used for frame pointers. The first instructions for functions using frame pointers are

        push    ebp           ;save ebp
        mov     ebp,esp       ;ebp = esp
        sub     esp,...       ;allocate space for local variables

then parameters and local variable are +/- offsets from ebp

Most compilers have an option to not use frame pointers, in which case esp is used as the base pointer. If non-frame pointer code uses ebp as a generic register, it still need to be saved.

Answer 4

If you want to learn about the stack and parameter passing conventions (ABI), I suggest you look at the assembly generated by the compiler. You can do this interactively on this site: http://gcc.godbolt.org/#

Try various argument types, varadic functions, passing and returning floats, doubles, structures of different sizes...

Messing with the stack using inline assembly is too difficult and unpredictable. It is likely to fail in so many ways, you will not learn anything useful.