CODEWITHSUNDEEP
pythonpytorchgpu
Guillaume Chevalier
Guillaume Chevalier

Reputation: 10888

Can't send pytorch tensor to cuda

I create a torch tensor and I want it to go to GPU but it doesn't. This is so broken. What's wrong?

def test_model_works_on_gpu():
    with torch.cuda.device(0) as cuda:
        some_random_d_model = 2 ** 9
        five_sentences_of_twenty_words = torch.from_numpy(np.random.random((5, 20, T * d))).float()
        five_sentences_of_twenty_words_mask = torch.from_numpy(np.ones((5, 1, 20))).float()
        pytorch_model = make_sentence_model(d_model=some_random_d_model, T_sgnn=T, d_sgnn=d)

        five_sentences_of_twenty_words.to(cuda)
        five_sentences_of_twenty_words_mask.to(cuda)
        print(type(five_sentences_of_twenty_words), type(five_sentences_of_twenty_words_mask))
        print(five_sentences_of_twenty_words.is_cuda, five_sentences_of_twenty_words_mask.is_cuda)
        pytorch_model.to(cuda)
        output_before_match = pytorch_model(five_sentences_of_twenty_words, five_sentences_of_twenty_words_mask)

        assert output_before_match.shape == (5, some_random_d_model)
        print(type(output_before_match))
        print(output_before_match.is_cuda, output_before_match.get_device())

tests/test_model.py:58: RuntimeError

<class 'torch.Tensor'> <class 'torch.Tensor'>
False False
<class 'torch.Tensor'>

>       print(output_before_match.is_cuda, output_before_match.get_device())
E       RuntimeError: get_device is not implemented for tensors with CPU backend

Also:

>>> torch.cuda.is_available()
True
>>> torch.cuda.device_count()
2

And:

pip freeze | grep -i torch
torch==1.0.0
torchvision==0.2.1

Upvotes: 1

Views: 10711

Answers (3)

iacob
iacob

Reputation: 24181

Your issue is the following lines:

five_sentences_of_twenty_words.to(cuda)
five_sentences_of_twenty_words_mask.to(cuda)

.to(device) only operates in place when applied to a model.

When applied to a tensor, it must be assigned:

five_sentences_of_twenty_words = five_sentences_of_twenty_words.to(cuda)
five_sentences_of_twenty_words_mask = five_sentences_of_twenty_words_mask.to(cuda)

Upvotes: 6

Guillaume Chevalier
Guillaume Chevalier

Reputation: 10888

Fixed by doing this:

def test_model_works_on_gpu():
    device_id = 0
    with torch.cuda.device(device_id) as cuda:
        some_random_d_model = 2 ** 9
        five_sentences_of_twenty_words = torch.from_numpy(np.random.random((5, 20, T * d))).float()
        five_sentences_of_twenty_words_mask = torch.from_numpy(np.ones((5, 1, 20))).float()
        pytorch_model = make_sentence_model(d_model=some_random_d_model, T_sgnn=T, d_sgnn=d)

        five_sentences_of_twenty_words = five_sentences_of_twenty_words.cuda(device_id)
        five_sentences_of_twenty_words_mask = five_sentences_of_twenty_words_mask.cuda(device_id)
        print(type(five_sentences_of_twenty_words), type(five_sentences_of_twenty_words_mask))
        print(five_sentences_of_twenty_words.is_cuda, five_sentences_of_twenty_words_mask.is_cuda)
        pytorch_model = pytorch_model.cuda(device_id)
        output_before_match = pytorch_model(five_sentences_of_twenty_words, five_sentences_of_twenty_words_mask)

        assert output_before_match.shape == (5, some_random_d_model)
        print(type(output_before_match))
        print(output_before_match.is_cuda, output_before_match.get_device())
        assert output_before_match.is_cuda
        assert five_sentences_of_twenty_words.is_cuda
        assert five_sentences_of_twenty_words_mask.is_cuda

I also used the add_module method in my modules.

Upvotes: 0

user3901687
user3901687

Reputation: 141

To transfer a "CPU" tensor to "GPU" tensor, simply do:

cpuTensor = cpuTensor.cuda()

This would take this tensor to default GPU device. If you have multiple of such GPU devices, then you can also pass device_id like this:

cpuTensor = cpuTensor.cuda(device=0)

Upvotes: 1

Related Questions