CODEWITHSUNDEEP
ccuda
ritter
ritter

Reputation: 7699

CUDA driver API equivalent for cudaSetDevice

What is the CUDA driver's API equivalent for the runtime API function cudaSetDevice?

I was looking into the driver API and cannot find an equivalent function. What I can do is

cuDeviceGet(&cuDevice, device_no);
cuCtxCreate(&cuContext, 0, cuDevice);

which is not equivalent since beside setting the device it also creates a context. The runtime API cudaSetDevice does not create a context per se. In the runtime API the CUDA context is created implicitly with the first CUDA call that requires state on the device.

Background for this question: CUDA-aware MPI (MVAPICH2 1.8/9) initialization requires the CUDA device to be set before MPI_init is called. Using the CUDA runtime API this can be done with 

cudaSetDevice(device_no);
MPI_init();

However, I don't want to use the call to the CUDA runtime since the rest of my application is purely using the driver API and I'd like to avoid linking also to the runtime.

What's wrong in creating the context already before MPI is initialized? In principle nothing. Just wondering if there is an equivalent call in the driver API.

Upvotes: 3

Views: 2418

Answers (2)

einpoklum
einpoklum

Reputation: 131525

Actually, cudaSetDevice() isn't exactly like creating to retrieving a context as though cuCtxCreate() was called. It's very similar, but there is a special context which the CUDA runtime API uses. This context is called the device's primary context. There are specific driver API functions for working with this special context:

CUresult cuDevicePrimaryCtxGetState ( CUdevice dev, unsigned int* flags, int* active );
CUresult cuDevicePrimaryCtxRelease ( CUdevice dev );
CUresult cuDevicePrimaryCtxReset ( CUdevice dev );
CUresult cuDevicePrimaryCtxRetain ( CUcontext* pctx, CUdevice dev );
CUresult cuDevicePrimaryCtxSetFlags ( CUdevice dev, unsigned int  flags );

So, why you want to achieve the equivane of cudaSetDevice(), that would be involve (ignoring error checking) something like:

CUcontext* primary_context;
cuDevicePrimaryCtxRetain(&primary_context, device_id);
cuCtxSetCurrent(primary_context);

Notes:

  • You should call the Release function at some point to reduce the reference count; but - don't do it without setting another current context.
  • You can either replace-replace-replace the current context, or push and then finally pop the current context from a stack of context. The replace works on the top of the stack (so it's like pop, then push).

Upvotes: 1

talonmies
talonmies

Reputation: 72349

You can find information about this in the Programming Guide Appendix about the Driver API, but the short version is this:

  • cuCtxCreate acts as the first cudaSetDevice call (that is it creates a context on the driver context stack)
  • The cuCtxPushCurrent() and cuCtxPopCurrent() pair (or cuCtxSetCurrent depending on which API version you are using) acts as any subsequent cudaSetDevice call (that is it pushes or selects a previously created context to be the active context for all subsequent API calls until the context is popped off the driver context stack or deselected)

Upvotes: 5

Related Questions