CODEWITHSUNDEEP
c++c
Jason
Jason

Reputation: 3307

Confused about __cdecl and __stdcall calling convention

__cdecl calling convention says that：caller cleanup stack.

__stdcall calling convention says that: callee cleanup stack.

So i try to test code below:

#include <...>

char *callee()
{
    char str[] = "abcd";

    return str;
}

int main()
{
    char *str;
    str = callee();
    printf("%s\n", str);

    return 0;
}

According to the two calling convention above, I think:

__cdecl stack cleanup by caller, so printf("%s\n", str) should output "abcd".

__stdcall stack cleanup by callee, so printf("%s\n", str) should output messy chars.

But in fact, two all output messy chars. I am confused. Thanks for help!

Upvotes: 7

Views: 3765

Answers (5)

Bo Persson
Bo Persson

Reputation: 92261

It doesn't work the way you thought, because in both cases the stack pointer is adjusted between each function call. The difference is just who does it.

With the __stdcall convention, the stack pointer is adjusted (removing all parameters) just before the function returns to the caller. This saves code, as it is only done in one place.

With the __cdecl convention, the stack pointer is adjusted by the calling code, just after the return from the called function. If you have functions with a variable number of parameters, like printf, it is hard for it to remove the parameters as there can be a different number for each point of gtcall. Therefore, the calling function will have to do the adjustment. It will know how many parameters were actually passed.

Upvotes: 1

qbert220
qbert220

Reputation: 11556

In the __cdecl case, the stack adjustment happens when the called function returns. By the time your printf executes the stack will have been adjusted.

Upvotes: 0

Erik
Erik

Reputation: 91270

Your error has no relation to calling convention. callee() is returning a pointer to a local variable, which is invalid as soon as callee() returns, and you get garbage data.

Upvotes: 2

Fred Foo
Fred Foo

Reputation: 363567

The behavior is undefined in either case. Most probably, the call to printf will reuse the stack space previously used by callee.

Upvotes: 0

sharptooth
sharptooth

Reputation: 170499

"Clean up the stack" actually means "adjust the stack pointer so that it matches the changes done to stack pointer when arguments and return address were pushed onto stack". In your example it would be irrelevant who cleans up the stack - you would get the same undefined behavior, because you try to access a stack-allocated variable that is overwritten by a series of later pushes of arguments needed to perform printf().

If you only write code in C or C++ you should not care of the exact details like "who cleans up the stack" - those details are critical only when writing in assembly. In C and C++ just be sure to mark function pointers with right calling conventions before calling a function through those pointers and the compiler will do the rest.

Upvotes: 11

Related Questions