CODEWITHSUNDEEP
cwinapidllinno-setuppascalscript
demberto
demberto

Reputation: 540

Calling a DLL function with an allocated character buffer that the function fills in Inno Setup

I am using (Unicode) Inno Setup 6.0.5 on Windows 10 64-bit.

The exported symbol, I want to use has the signature:

typedef int(__stdcall *GetDirVST2x86) (LPWSTR lpString1);

The Inno Setup [Code] section has its declaration as:

function GetDirVST2x86(var lpString1: String): Integer;
external 'GetDirVST2x86@files:R2RINNO.DLL stdcall setuponly';

where, lpString1 will contain a pointer to the wide-string after the function returns and R2RINNO.DLL is a 32-bit DLL.

Now my problem is, if I compile and run this setup, a read access violation occurs right when I try to retrieve the value. I get the correct result when I execute this same function from a C program. Removing the var from the prototype declaration in Inno script fetches an empty (or possibly) empty or blank string, so that doesn't help either.

I don't have the source for the DLL I wish to use, and I figured out the signature from IDA. The scripting engine Inno Setup seems hopelessly inadequate as it doesn't support pointers at all.

One interesting thing I observed was if I changed the type of lpString1 to Cardinal or Integer and used IntToStr to fetch the string I got the value of the directory in which the setup was getting created.

Here's a working C code:

#include <windows.h>
#include <stdio.h>

#define _UNICODE
#define UNICODE

typedef int(WINAPI *GetDirVST2x86) (LPWSTR );

int main() {
    HMODULE hModule = LoadLibrary("R2RINNO.DLL");
    if (NULL != hModule) {
        GetDirVST2x86 pGetDirVST2x86 = (GetDirVST2x86) GetProcAddress (hModule, "GetDirVST2x86");
        if (NULL != pGetDirVST2x86) {
            LPWSTR lpszVST2x86;
            pGetDirVST2x86(lpszVST2x86);
            wprintf(lpszVST2x86);
        }
        FreeLibrary(hModule);
    }
}

Here's the output:

C:\Program Files (x86)\Steinberg\VstPlugins

Here's the IDA screenshot of the function I want to use: GetDirVST2x86 IDA disassembly

Upvotes: 1

Views: 386

Answers (1)

Martin Prikryl
Martin Prikryl

Reputation: 202118

Pascal Script equivalent of the C declaration should be:

function GetDirVST2x86(lpString1: string): Integer;
  external 'GetDirVST2x86@files:R2RINNO.DLL stdcall setuponly';

(i.e. no var, as it is an input character pointer argument).

Assuming the function contract is that you (as a caller) allocate a buffer and provide it to the function to be filled in, you should call the function like this:

var
  Buf: string;
begin
  { Allocate buffer for the result large enough according to the API specification }
  SetLength(Buf, 1000);
  GetDirVST2x86(Buf);
  SetLength(Result, Pos(#0, Result) - 1);
end;

See also How to return a string from a DLL to Inno Setup?

Upvotes: 3

Related Questions