When calling EnumServicesStatusEx() twice, i still get EROR_MORE_DATA in C++

Question

I call EnumServicesStatusEx() twice in my code, the first time should fail and put the correct buffer size in dwBuffNeeded so that when i call it the second time the buffer size should be correct. But, sometimes, not always i still get ERROR_MORE_DATA after the second call. Any ideas why? Thanks

DWORD pId=GetCurrentProcessId();
    SC_HANDLE hSCM    = NULL;
    PUCHAR  pBuf    = NULL;
    ULONG  dwBufSize   = 0x00;
    ULONG  dwBufNeed   = 0x00;
    ULONG  dwNumberOfService = 0x00;
    LPENUM_SERVICE_STATUS_PROCESS pInfo = NULL;

    hSCM = OpenSCManager( NULL, NULL, SC_MANAGER_ENUMERATE_SERVICE | SC_MANAGER_CONNECT );

    if (hSCM == NULL)
    {
        GetCustomLog().Log( SV_ERROR, 10004807, "Could not open Service Control Manager: %s", GetLastOSErrorString().c_str() );
        return;
    }

    //Query services once to get correct buffer size, always fails
    if ( EnumServicesStatusEx(
        hSCM,
        SC_ENUM_PROCESS_INFO,
        SERVICE_WIN32, 
        SERVICE_ACTIVE,
        NULL,
        dwBufSize,
        &dwBufNeed,
        &dwNumberOfService,
        NULL,
        NULL) == 0 )
    {

        DWORD err = GetLastError();
        if ( ERROR_MORE_DATA == err )
        {
            dwBufSize = dwBufNeed + 0x10;
            pBuf  = (PUCHAR) malloc(dwBufSize);

            //Query services again with correct buffer size
            if ( EnumServicesStatusEx(
                hSCM,
                SC_ENUM_PROCESS_INFO,
                SERVICE_WIN32, 
                SERVICE_ACTIVE,
                pBuf,
                dwBufSize,
                &dwBufNeed,
                &dwNumberOfService,
                NULL,
                NULL ) == 0 )

            {
//FAILS HERE
                GetCustomLog().Log( SV_ERROR, 10004808, "Could not enumerate services, error: %s", GetLastOSErrorString().c_str() );
                free(pBuf);
                return;
            }
        }
        else
        {
            GetCustomLog().Log( SV_ERROR, 10004809, "Could not enumerate services, error: %s", GetLastOSErrorString().c_str() );
            return;
        }

Answer 1

Because service information isn't static and theoretically anything can happen between two calls to EnumServicesStatusEx(), this API is best used iteratively.

As Kat Marsen suggests, you may of course repeatedly try to fetch all services (looping and growing a buffer as long as the API returns ERROR_MORE_DATA).
But this has 2 drawbacks:

1. You tell the SCM to constantly fill in information it actually has sent you back already (even it's rather rare, I actually want to avoid it).
2. According to the MSDN documentation, there's a hard upper limit of 256K bytes for the buffer. As I mentioned in a comment above, the latter is practically very unlikely; still you want to play safe as a programmer.

The iterative approach

In contrary we can IMO only presume that the author responsible for EnumServicesStatusEx() had a reason to design it this way:

• A buffer which is filled immediately if valid and big enough for at least one service.
• An output parameter for the remaining bytes the SCM assumes for the services yet to be iterated.
• Most importantly there's an output parameter telling you the resume point.

Both, the resume point and the remainder allow for a much easier iterative approach.

Let me showcase with complete real-world functions, working reasonably well, and featuring some C++ goodies.
chunk_fetch_all_win32_services pre-allocates some amount of memory, invokes a callback for every chunk, and lets the callback decide whether it consumes the memory or leaves it in order to be reused.
enum_all_win32_services consumes each chunk and in turn invokes a callback for each single service.

#include <type_traits>
#include <memory>
#include <stddef.h>
#include <Windows.h>

using std::unique_ptr;
using std::error_code;
using std::system_error;
using std::system_category;
using std::function;
using std::make_unique;


/** @short Fetch all win32 services in chunks.

    This function fetches all services of type `SERVICE_WIN32=(SERVICE_WIN32_OWN_PROCESS | SERVICE_WIN32_SHARE_PROCESS)`,
    in chunks as decided by the SCM (refer to EnumServicesStatusEx()).

    @param scmHandle Handle to the SCM with access right SC_MANAGER_ENUMERATE_SERVICE
    @param stateMask One of SERVICE_ACTIVE, SERVICE_INACTIVE, SERVICE_STATE_ALL
    In the callback you can decide whether you want to consume the passed memory or leave
    it in order to be reused.

    @note This is most probably rare but expect the callback being invoked multiple times
    in case the SCM didn't return information about all services at once.
 */
bool chunk_fetch_all_win32_services(SC_HANDLE scmHandle, DWORD stateMask, const function<bool(unique_ptr<ENUM_SERVICE_STATUS_PROCESS[]>&, DWORD /*nServices*/)>& cb, error_code* ec)
{
    // (optionally) preallocate
    // (the amount stems from Win XP's upper size of 64k;
    //  on a typical Win7 system there are about 220 services (about 34k))
    unique_ptr<BYTE[]> mem = make_unique<BYTE[]>(64 * 1024);
    DWORD nAllocated = 64 * 1024, nRemaining;
    DWORD resumePoint = 0;
    do
    {
        DWORD nServices;
        if (!EnumServicesStatusEx(scmHandle, SC_ENUM_PROCESS_INFO, SERVICE_WIN32, stateMask, mem.get(), nAllocated, &nRemaining, &nServices, &resumePoint, nullptr))
        {
            const int errorCode = GetLastError();
            if (errorCode != ERROR_MORE_DATA)
            {
                if (!ec)
                    throw system_error{ errorCode, system_category(), "Can't enumerate services" };
                ec->assign(errorCode, system_category());
                return false;
            }
        }

        if (nServices)
        {
            // memory initialized, transfer ownership to typed pointer
            unique_ptr<ENUM_SERVICE_STATUS_PROCESS[]> cache{ LPENUM_SERVICE_STATUS_PROCESS(mem.release()) };
            if (!cb(cache, nServices))
                // early bail-out requested
                return true;

            // document that the typed pointer can be 'downgraded' again without the need to destroy objects
            static_assert(std::is_trivially_destructible_v<ENUM_SERVICE_STATUS_PROCESS>);
            // possibly reuse existing buffer
            mem.reset(PBYTE(cache.release()));
        }

        if (nRemaining)
        {
            // re-allocate if buffer too small or consumed by callback
            if (!mem || nAllocated < nRemaining)
                mem = make_unique<BYTE[]>(nRemaining),
                nAllocated = nRemaining;
        }

        // loop as long as there are more services to be enumerated
    } while (nRemaining);

    return true;
}

/** @short Enumerate all win32 services.

    This function enumerates all services of type `SERVICE_WIN32=(SERVICE_WIN32_OWN_PROCESS | SERVICE_WIN32_SHARE_PROCESS)`.

    @param scmHandle Handle to the SCM with access right SC_MANAGER_ENUMERATE_SERVICE
    @param stateMask One of SERVICE_ACTIVE, SERVICE_INACTIVE, SERVICE_STATE_ALL
    @param cb Callback receiving process information of a single service process information and the number of services.

    @note This is most probably rare but expect the number of services passed to the callback to change
    in case the SCM didn't return information about all services at once; if, then this of course happens
    after the provided number of information items have been iterated.
 */
bool enum_all_win32_services(SC_HANDLE scmHandle, DWORD stateMask, const function<bool(ENUM_SERVICE_STATUS_PROCESS&, DWORD /*nServices*/)>& cb, error_code* ec)
{
    return chunk_fetch_all_win32_services(scmHandle, stateMask, [&cb](unique_ptr<ENUM_SERVICE_STATUS_PROCESS[]>& cache, DWORD nServices)
    {
        for (size_t idx = 0; idx < nServices; ++idx)
        {
            if (!cb(cache[idx], nServices))
                // early bail-out requested
                return true;
        }

        return true;
    }, ec);
}

Answer 2

I too had the same problem, but when querying SERVICE_STATE_ALL, which I thought would require the same amount of memory on each call (unless a service had been installed/uninstalled). It wasn't enough to simply retry with a buffer of the size returned in the pcbBytesNeeded argument:

BOOL WINAPI EnumServicesStatusEx(
  _In_         SC_HANDLE hSCManager,
  _In_         SC_ENUM_TYPE InfoLevel,
  _In_         DWORD dwServiceType,
  _In_         DWORD dwServiceState,
  _Out_opt_    LPBYTE lpServices,
  _In_         DWORD cbBufSize,
  _Out_        LPDWORD pcbBytesNeeded,
  _Out_        LPDWORD lpServicesReturned,
  _Inout_opt_  LPDWORD lpResumeHandle,
  _In_opt_     LPCTSTR pszGroupName
);

Unlike other WIN32 API calls, this is not returning the absolute number of bytes needed, but the additional bytes needed relative to the cbBufSize parameter. As an experiment, I provided a deliberately small buffer and just doubled it each time to find out what the system would return in pcbBytesNeeded in response. On this system sizeof(ENUM_SERVICE_STATUS_PROCESS) is 56 bytes. The last row is the smallest buffer resulting in a successful call.

+-----------+----------------+
| cbBufSize | pcbBytesNeeded | 
+-----------+----------------+
|      112  |         37158  |
|      224  |         37013  |
|      448  |         36766  |
|      896  |         36374  |
|     1792  |         35280  |
|     3584  |         33202  |
|     7168  |         28972  |
|    14336  |         20765  |
|    28672  |          4215  |
|    32032  |             0  |
+-----------+----------------+

You can see each row roughly adds up to the required buffer size, and also that the required buffer size as far as the system is concerned is not very predictable. The number of service entries returned in the successful call in this case was 233, which only requires 13048 bytes. It turns out the strings pointed to by the ENUM_SERVICE_STATUS_PROCESS lpDisplayName and lpServiceName pointers are just stored at the tail-end of the provided buffer (I always wondered where the backing for those was, and why it was stable and didn't need to be separately freed). Anyway this explains the somewhat non-deterministic responses as well as the odd numbers: the system likely has the current entry which doesn't fit and knows exactly how much size it needs for that, but guesses about the remainder.

The code below is reliable and usually takes exactly two calls but sometimes can take three (even with SERVICE_STATE_ALL). I don't know why three are needed and the underestimation by the system sticks around for multiple minutes at a time but eventually resolves itself. I've never seen it need four calls.

int EnumerateAllServices(SC_HANDLE hSCM) {
  void* buf = NULL;
  DWORD bufSize = 0;
  DWORD moreBytesNeeded, serviceCount;
  for (;;) {
    printf("Calling EnumServiceStatusEx with bufferSize %d\n", bufSize);
    if (EnumServicesStatusEx(
        hSCM,
        SC_ENUM_PROCESS_INFO,
        SERVICE_WIN32,
        SERVICE_STATE_ALL,
        (LPBYTE)buf,
        bufSize,
        &moreBytesNeeded,
        &serviceCount,
        NULL,
        NULL)) {
      ENUM_SERVICE_STATUS_PROCESS* services = (ENUM_SERVICE_STATUS_PROCESS*)buf;
      for (DWORD i = 0; i < serviceCount; ++i) {
        printf("%s\n", services[i].lpServiceName);
      }
      free(buf);
      return 0;
    }
    int err = GetLastError();
    if (ERROR_MORE_DATA != err) {
      free(buf);
      return err;
    }
    bufSize += moreBytesNeeded;
    free(buf);
    buf = malloc(bufSize);
  }
}

The "+=" is the 'trick', and is not documented very clearly (but, in hindsight, I now understand the nuance in the MSDN explanation of the pcbBytesNeeded parameter):

pcbBytesNeeded [out]

    A pointer to a variable that receives the number of bytes
    needed to return the remaining service entries, if the buffer
    is too small.