CODEWITHSUNDEEP
cudaparallel-processinggpu
Prashanth
Prashanth

Reputation: 11

How to run 2 CUDA applications simultaneously?

I'm very new to CUDA and GPU programming. I currently have 2 CUDA applications (A.cu and B.cu) and would want to run each of them separately and calculate the total time of execution. But I would like to run both applications simultaneously and check the execution time. Is there a way to do this?

In other words, how can I launch two kernels concurrently from 2 different applications at the same time so that they run in parallel?

Any help in this regard will be useful. Thanks in advance.

Upvotes: 1

Views: 3834

Answers (1)

Robert Crovella
Robert Crovella

Reputation: 151799

If you have 2 GPUs, you can launch one application (in one process) on one GPU and one on the other (for example using the environment variable CUDA_VISIBLE_DEVICES. However with a single GPU, you can launch two applications, but the kernels of these applications will be serialized. That is, while a kernel from one application is running, the GPU will not schedule a kernel from another application.

You can avoid this limitation on a GPU with cc 3.5 by using the Hyper-Q feature. One way this feature is exposed is through the CUDA Multi-Process Server

However, if you are new to CUDA and GPU programming, that is probably not what you want. In addition, it requires a Quadro or Tesla GPU.

To answer your question then, to run two applications at the same time, first be sure that your GPU is set to a Compute Mode of Default. You can query this and modify this setting using the nvidia-smi utility. If it's not settable on your GPU or "N/A", it should be in the Default mode.

After that you can simply launch both applications however you like. They should both find and attempt to use the GPU, however the serialization of kernels may not give you the results you want.

Limitations of your GPU, such as global memory size, may be in effect on both applications at the same time. For example, if each application expects to allocate 1.5GB of memory on the GPU, and you have a 2GB GPU, then most likely the CUDA api functions (e.g. cudaMalloc) of one application will return errors.

In short, especially for a beginner, I can't think of any good reasons why you'd want to do this. Presumably you're after an additional level of "parallelism" but you won't achieve that in simple fashion, when the kernels are being launched from separate linux processes.

Upvotes: 6

Related Questions