CODEWITHSUNDEEP
cudanvidia
Alvin
Alvin

Reputation: 950

Does CUDA allow multiple applications on same gpu at the same time?

I have Tesla K20m GPU card from NVIDIA. In CUDA 5.0 onwards multiple processes from the same application on same GPU is allowed. Does CUDA allow execution of different applications on same GPU at the same time?

Upvotes: 3

Views: 3906

Answers (2)

Raz Rotenberg
Raz Rotenberg

Reputation: 609

Multiple applications may run at the same time on the same GPU. Namely, multiple applications can have a CUDA context at the same time and launch kernels, copy memory, etc...

But kernels from different CUDA contexts cannot be executed simultaneously on the same GPU. Meaning, at the very same slice of time, only kernels from a single CUDA context may be executed on a GPU. This may cause a GPU underutilization if kernels do not occupy the entire GPU resources (memory + compute), and some of the resources may be left unused.

MPS enables that by actually having a server with a single CUDA context, and all client processes communicate with the GPU device through this server, and eventually using its single CUDA context. This enables actual concurrency between kernel launches from different (logical) CUDA contexts.

Upvotes: 2

Maja Piechotka
Maja Piechotka

Reputation: 7216

Depends what do you mean by 'at the same time'. If you mean 'two applications have CUDA contexts on same card at the same time' then yes.

Though you may want to use MPS to get full benefits and reduce context switching. See also this question.

Upvotes: 2

Related Questions