CODEWITHSUNDEEP
cudapytorchtorchnumba
BS.
BS.

Reputation: 123

Any guarantees that Torch won't mess up with an already allocated CUDA array?

Assume we allocated some array on our GPU through other means than PyTorch, for example by creating a GPU array using numba.cuda.device_array. Will PyTorch, when allocating later GPU memory for some tensors, accidentally overwrite the memory space that is being used for our first CUDA array? In general, since PyTorch and Numba use the same CUDA runtime and thus I assume the same mechanism for memory management, are they automatically aware of memory regions used by other CUDA programs or does each one of them see the entire GPU memory as his own? If it's the latter, is there a way to make them aware of allocations by other CUDA programs?

EDIT: figured this would be an important assumption: assume that all allocations are done by the same process.

Upvotes: 1

Views: 157

Answers (1)

talonmies
talonmies

Reputation: 72344

Will PyTorch, when allocating later GPU memory for some tensors, accidentally overwrite the memory space that is being used for our first CUDA array?

No.

are they automatically aware of memory regions used by other CUDA programs ...

They are not "aware", but each process gets its own separate context ...

... or does each one of them see the entire GPU memory as his own?

.... and contexts have their own address spaces and isolation. So neither, but there is no risk of memory corruption.

If it's the latter, is there a way to make them aware of allocations by other CUDA programs?

If by "aware" you mean "safe", then that happens automatically. If by "aware" you imply some sort of interoperability, then that is possible on some platforms, but it is not automatic.

... assume that all allocations are done by the same process.

That is a different situation. In general, the same process implies a shared context, and shared contexts share a memory space, but all the normal address space protection rules and facilities apply, so there is not a risk of loss of safety.

Upvotes: 3

Related Questions