CODEWITHSUNDEEP
cudagpu
Duc Nguyen
Duc Nguyen

Reputation: 761

Free memory occupied by cudaMemGetInfo

I have the following simple code to find available GPUs

int * getFreeGpuList(int *numFree) {
    int * gpuList;
    int nDevices;
    int i, j = 0, count = 0;

    cudaGetDeviceCount(&nDevices);
    gpuList = (int *) malloc(nDevices * sizeof(int));
    for (i = 0; i < nDevices; ++i) {
        cudaSetDevice(i);
        size_t freeMem;
        size_t totalMem;
        cudaMemGetInfo(&freeMem, &totalMem);
        if (freeMem > .9 * totalMem) {
            gpuList[j] = i;
            count++;
            j++;
        }
    }
    *numFree = count;
    return gpuList;
}

The problem is that cudaMemGetInfo occupies some memory (~150MB in my case) in each GPU. This code is a part of a bigger program that runs for a long time, and I often run several processes at the same time, so in the end the memory occupied by this function is significant. Could you please let me know how I can free the GPU memory occupied by cudaMemGetInfo? Thanks!

Upvotes: 1

Views: 577

Answers (1)

Duc Nguyen
Duc Nguyen

Reputation: 761

Based on some insight from talonmies above that cudaSetDevice creates a context and occupies some memory in the device, I found out that cudaDeviceReset can "explicitly destroys and cleans up all resources associated with the current device in the current process" without affecting other processes on the same device.

Update Nov 26: If one wants to query GPU info, it's better using the NVML library. In my experience, it is much faster and does not take up memory for simple memory and name queryings.

Upvotes: 1

Related Questions