CODEWITHSUNDEEP
c++oopassemblyx86return-value
Will Custode
Will Custode

Reputation: 4604

Returning Object Data from C++ Method Called in Assembly

I have a method written in C++ that returns an object. This method is called via assembly (for a very long-winded reason). Example:

Person DoStuff( int a )
{
    Person output;

    output.Name = "Koder";
    output.Age = 1337;
    output.Cash = 80.86;

    cout << "Given number is " << a << endl;

    return output;
}

I know return by value is nasty and bad, but this problem isn't about that. Calling this method looks something like this:

Variant vMethod = &DoStuff;
void* pMethod = vMethod.As<void*>( );

int paramVal = 78;

int* retVal = nullptr;

__asm
{
    push paramVal
    call pMethod
    mov retVal, EAX
}

When the return value of this method was an int, this worked perfectly. But now returning an object, the actual call instruction raises an exception. The exception, as you can guess, is on the return statement. I knew this would happen when I wrote it this way, but I have no clue how to retrieve not-primitive (or non-integer, for that matter) return values. Googling C++/assembly function calling didn't help a whole lot. Thanks to anyone who can help.

Upvotes: 1

Views: 796

Answers (1)

Rickard
Rickard

Reputation: 7277

It depends on what kind of calling Calling Convention that is used. It also depends on how the compiler handles the object. In this case your object contains at least 3 values, which makes it unlikely that all values will fit in a register (your code is assuming return value will be in EAX - which it would be if you were returning a int).

From looking at the assembly it seems you are using x86 32-bit assembly. In this case the calling convention says you should pass arguments on the stack, For this case I think the calling convention for x86 bit says that you need to reserve memory for the return value yourself and pass the address where the object should be stored as a parameter. On Windows the return pointer (address to store the object) is the second parameter and on Linux the first parameter.

Since you failed to do this the DoStuff method will probably overwrite the return address, and causing the program to crash on the return instruction.

Upvotes: 1

Related Questions