CODEWITHSUNDEEP
c++winapivisual-c++shellexecuteex
xariswon
xariswon

Reputation: 1

Unexpected/Inconsistent Behavior from ShellExecuteEx

I am creating self-configuring software tool that runs several other 3rd party installers as necessary, in Visual C++ (2015) using Qt. When the software finds it needs an unavailable library or driver, it will call the appropriate installer using ShellExecuteEx to open the appropriate executable.

If only one thing needs installed (ShellExecuteEx is run only once) everything behaves normally. If two things need installed at once, then roughly 25% of the time, once installer will run correctly, and the other will open the folder containing the executable, instead of running the executable. There is no consistency as to whether it is the first or second operation that runs correctly.

ShellExecuteEx claims to have run correctly in both cases (returns true). I have verified that the correct path to the executable files (and working directories) are being provided every time. I have tried using both the verb "open", and "NULL" in ShellExecuteEx, with no change in behavior.

If it matters, both executables request administrative privileges. When everything works, they both do so as expected. When one fails, it does not ask for admin privileges.

I cannot find any record of someone having similar problems, which probably means I'm misusing ShellExecuteEx in some basic way, but I am not seeing it. Thanks in advance for any suggestions or advice.

Here is the code:

if(dummy1Required && !dummy1Present){
        SHELLEXECUTEINFOA execinfo = {};
        execinfo.cbSize = sizeof(SHELLEXECUTEINFOA);
        execinfo.fMask = NULL;
        execinfo.hwnd = NULL;
        execinfo.lpVerb = NULL;
        execinfo.lpFile = QDir::toNativeSeparators(QCoreApplication::applicationDirPath() + "/Drivers/dummy1/install.bat").toLocal8Bit().data();
        execinfo.lpParameters = NULL;
        execinfo.lpDirectory = QDir::toNativeSeparators(QCoreApplication::applicationDirPath() + "/Drivers/dummy1").toLocal8Bit().data();
        printf("Dummy1 Executing: %s\n In directory:%s\n", QDir::toNativeSeparators(QCoreApplication::applicationDirPath() + "/Drivers/dummy1/install.bat").toLocal8Bit().data(), QDir::toNativeSeparators(QCoreApplication::applicationDirPath() + "/Drivers/dummy1").toLocal8Bit().data());
        fflush(stdout);
        execinfo.nShow = SW_SHOW;
        execinfo.hInstApp = NULL;
        execinfo.fMask = execinfo.fMask | SEE_MASK_NOCLOSEPROCESS;
        if(ShellExecuteExA(&execinfo)){
            WaitForSingleObject(execinfo.hProcess,INFINITE);
            CloseHandle(execinfo.hProcess);
        }else{
            printf("Failed to launch dummy1 installer because...%lu\n", GetLastError());
            fflush(stdout);
        }
    }

    if(dummy2Required && !dummy2Present){
        SHELLEXECUTEINFOA execinfo = {};
        execinfo.cbSize = sizeof(SHELLEXECUTEINFOA);
        execinfo.fMask = NULL;
        execinfo.hwnd = NULL;
        execinfo.lpVerb = NULL;
        execinfo.lpFile = QDir::toNativeSeparators(QCoreApplication::applicationDirPath() + "/Drivers/dummy2/setup.exe").toLocal8Bit().data();
        execinfo.lpParameters = "setupParamters.ini /qb /acceptlicenses y /norestart";
        execinfo.lpDirectory = QDir::toNativeSeparators(QCoreApplication::applicationDirPath() + "/Drivers/dummy2").toLocal8Bit().data();
        printf("Dummy2 Executing: %s\n In directory:%s\n", QDir::toNativeSeparators(QCoreApplication::applicationDirPath() + "/Drivers/dummy2/setup.exe").toLocal8Bit().data(), QDir::toNativeSeparators(QCoreApplication::applicationDirPath() + "/Drivers/dummy2").toLocal8Bit().data());
        fflush(stdout);
        execinfo.nShow = SW_SHOW;
        execinfo.hInstApp = NULL;
        execinfo.fMask = execinfo.fMask | SEE_MASK_NOCLOSEPROCESS;
        if(ShellExecuteExA(&execinfo)){
            WaitForSingleObject(execinfo.hProcess,INFINITE);
            CloseHandle(execinfo.hProcess);
        }else{
            printf("Failed to launch dummy2 installer because...%lu\n", GetLastError());
            fflush(stdout);
        }
    }

EDIT: I was incorrect. Much less frequently, I will observe this behavior if only one executable is run, so it isn't inherently linked to running two operations.

EDIT 2: PaulMckenzie points out that I wasn't properly initializing the structs, which is embarrassing. Unfortunately, fixing it did not change the behavior. Thanks, though!

Upvotes: 0

Views: 113

Answers (1)

PaulMcKenzie
PaulMcKenzie

Reputation: 35454

There is at least two mistakes in the code posted. 

if(dummy1Required && !dummy1Present){
        SHELLEXECUTEINFOA execinfo; // <-- Uninitialized
    ...
}

if(dummy2Required && !dummy2Present){
        SHELLEXECUTEINFOA execinfo; // <-- Uninitialized
     ...
}

You then go and set some members of SHELLEXECUTEINFOA, but did you set all of them? What if you missed some of the members?

For many Windows API functions that require usage of a struct such as SHELLEXECUTEINFOA, the struct should be initialized by zeroing out all of the members before setting any members. Since the struct is a local variable, when declared in this fashion leaves it uninitialized.

The probable reason for the erratic behavior is that this struct contains members that you may not know exist, but the Win32 API will be using. If these members are uninitialized, the API function will use whatever value the uninitialized member is holding.

The easiest way to initialize the struct is to simply use brace initialization:

SHELLEXECUTEINFOA execInfo = {};

The other way to zero out a struct that you may encounter if you see examples of Win32 API C programming is usage of ZeroMemory. However for C++, this is not needed due to the { } initialization syntax that exists in C++.

Upvotes: 1

Related Questions