Base64 Assembler Fill Array Error "Operands different sizes" Visual Studio

Question

Im trying to make a Base64Encode in inline assembler in Visual Studio.

I got this func

char* Base64Encode(char* data, int len)
{
// Tabelle mit den Zeichen für die Codierung
const char* encodeTable = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
//
char* result;   


if (len > 0)            // sonst gibt es nichts zu tun ...
{
    int encodedLength   = ((len + 2) / 3) * 4;      // effektiv die Ganzzahlfassung von ceil(length/3)*4
    result = new char[encodedLength+1];             // +1 wg. Null-Terminierung
    _asm
    {           
        mov esi,data
        mov edi,encodeTable

        xor eax, eax
        // get 3 bytes
        mov ah, byte ptr[esi]
        mov al, byte ptr[esi+1]
        shl eax,16
        mov ah, byte ptr[esi+2]

        //
        mov edx,eax
        shl eax,6

        shr edx, 26
        mov bl, byte ptr[edi + edx]    
        mov [result],bl

        //
        mov edx, eax
        shl eax, 6                                    

        shr edx, 26
        mov bl, byte ptr[edi + edx]            
        mov[result+1], bl

        //manipulate in edx bitset3
        mov edx, eax
        shl eax, 6

        shr edx, 26
        mov bl, byte ptr[edi + edx]
        mov[result+2], bl

        //manipulate in edx bitset4
        mov edx, eax
        shl eax, 6

        shr edx, 26
        mov bl, byte ptr[edi + edx]
        mov[result+3], bl


    }
}
else
{
    result = "";
}
return result;
}

The encoding is working proper, I have in bl always the right letter, but the output is not working ( result array doesn't fill with the letters, im getting the error that the operands have different sizes, I am only allowed to make changes in the __asm function ) .

Could somebody help me how to fill the result-array with the letters I get in bl? Debugging always shows me the right letters in the bl ' s if i comment out all the result lines.

EDIT:

I get nothing in the result array when i use the byte ptr.

Any ideas?

EDIT2:

Answer 1

The issue in your code is a matter of indirection. You define and initialize a variable result like this in the C++ code:

char* result; 
result = new char[encodedLength+1]; 

result is a memory location that holds a pointer to an array of characters returned by new. result is not the memory location where data will be stored, but contains a pointer to that data area. You then access it in the ASM block like this:

mov [result],bl

The compiler/assembler(MASM) warned that there was an operand mismatch when it said Operands different sizes. It knew that result was the location of a 32-bit pointer (not single characters). Since result is the address containing a pointer the code above would have moved the contents of bl to the memory location result. This had the effect of changing the pointer (returned by new) not what result was pointing at.

You need to deal with indirection here. You want to get the address that is stored in result and use that as a base for memory addressing. You can choose an available register like ECX and MOV the contents of result into it. You could do that with something like this at the top of your ASM block:

mov ecx, dword ptr [result]

This takes the 32-bit(dword) value at memory location result and stores that in ECX. Now that we have the memory location to the beginning of the character buffer we can now modify all references of result in the ASM block and change it to ECX. Examples:

mov [result],bl 

would become:

mov byte ptr [ecx],bl 

and

mov[result+1], bl 

would become:

mov byte ptr [ecx+1], bl

The second example is called base plus displacement (or offset) addressing . That link also describes all the addressing modes on x86. If you were using 16-bit code (which you aren't) there are some extra restrictions in the register choices that can be use for base and indexing.

As well user3144770 also pointed out that you didn't null terminate your string (you only allocated space for it), so at the bottom of your ASM block you should have probably used something like:

mov byte ptr[ecx+4], 0

With the changes above your code could look something like:

char* Base64Encode(char* data, int len)
{
    // Tabelle mit den Zeichen für die Codierung
    const char* encodeTable = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
    //
    char* result;

    if (len > 0)            // sonst gibt es nichts zu tun ...
    {
        int encodedLength = ((len + 2) / 3) * 4;      // effektiv die Ganzzahlfassung von ceil(length/3)*4
        result = new char[encodedLength + 1];         // +1 wg. Null-Terminierung
        _asm
        {
            mov esi, data
            mov edi, encodeTable

                mov ecx, dword ptr [result]
                xor eax, eax
                // get 3 bytes
                mov ah, byte ptr[esi]
                mov al, byte ptr[esi + 1]
                shl eax, 16
                mov ah, byte ptr[esi + 2]

                //
                mov edx, eax
                shl eax, 6

                shr edx, 26
                mov bl, byte ptr[edi + edx]
                mov byte ptr [ecx], bl

                //
                mov edx, eax
                shl eax, 6

                shr edx, 26
                mov bl, byte ptr[edi + edx]
                mov byte ptr [ecx + edx + 1], bl

                //manipulate in edx bitset3
                mov edx, eax
                shl eax, 6

                shr edx, 26
                mov bl, byte ptr[edi + edx]
                mov byte ptr [ecx + 2], bl

                //manipulate in edx bitset4
                mov edx, eax
                shl eax, 6

                shr edx, 26
                mov bl, byte ptr[edi + edx]
                mov byte ptr [ecx + 3], bl

                mov byte ptr[ecx + 4], 0
        }
    }
    else
    {
        result = "";
    }
    return result;
}