CODEWITHSUNDEEP
cvisual-c++assemblyx86inline-assembly
ravelinx
ravelinx

Reputation: 1785

Get returned char from a C function using inline assembly

I have this function that uses inline assembly that basically calls a C function, gets the returned value, and passes that value as a parameter to another function that returns a character. 

void convertText(FILE *arch, FILE *result)
{
    int i = 0;
    int n = arch->size;

    _asm {
    mov esi, 0
whileS:
    cmp esi, n
    jge end

    mov ebx, result
    mov ebx, [ebx]result.information ; Pointer to an array of characters

    push esi ; Push parameters to get5bitsFunc
    push arch ; Push parameters to get5bitsFunc

    call get5bitsFunc
    pop arch ; Restore values
    pop esi ; Restore values

    push eax ; push get5bitsFunc returned value to codify as parameter
    call codify
   mov edi, eax ;  <- HERE move returned value from codify to edi register
    pop eax ; restore eax

    inc esi
    jmp whileS
end:

    }
}

Think of codify as function of the type

unsigned char codify(unsigned char parameter) {
    unsigned char resp;

    // Do something to the parameter
    resp = 'b'; // asign value to resp
    return resp;
}

I have already tested codify and works fine returning the value I want using C code. The problem is that when I run and debug the convertText code in inline assembly in the line I have marked as "-> Here" the value returned in eax is something of the type 3424242 and not 97 or above in the ascii table that is what I need.

How can I get the char value?

Upvotes: 0

Views: 939

Answers (2)

Peter Cordes
Peter Cordes

Reputation: 365557

The Windows ABI apparently doesn't require functions returning char to zero- or sign-extend the value into EAX, so you need to assume that the bytes above AL hold garbage. (This is the same as in the x86 and x86-64 System V ABI. See also the tag wiki for ABI/calling convention docs).

You can't assume that zeroing EAX before calling codify() is sufficient. It's free to use all of EAX as a scratch register before returning with the char in AL, but garbage in the rest of EAX.

You actually need to movzx esi, al, (or MOVSX), or mov [mem], al or whatever else you want to do to ignore garbage in the high bytes.

Upvotes: 3

Brendon Boldt
Brendon Boldt

Reputation: 235

An unsigned char is only 1 byte while eax is a 32-bit (4 byte) register. If codify() is only returning 1 byte, then the return value will be stored in al (the first byte of eax) while leaving the rest of eax untouched (which would result in garbage). I would recommend xor eax, eax before calling codify() so you know that the register is clean before you store the return value in it.

Upvotes: 1

Related Questions