CODEWITHSUNDEEP
pythonpytorch
Fadwa
Fadwa

Reputation: 1931

RuntimeError: Expected a 'cuda' device type for generator but found 'cpu'

I am trying to train PeleeNet pytorch and got the following error

enter image description here

train.py line 80 enter image description here

pelee_voc train configuration enter image description here

Upvotes: 13

Views: 27954

Answers (6)

Rahul Bhatia
Rahul Bhatia

Reputation: 21

In case none of the above answers work, try adding the following line and hopefully it should solve the issue -

torch.set_default_device('cuda')

Upvotes: 1

Bas Krahmer
Bas Krahmer

Reputation: 631

As per this comment:

torch.utils.data.DataLoader(
    ...,
    generator=torch.Generator(device='cuda'),
)

There is no need to disable data shuffling or to modify source code as suggested in other answers.

Upvotes: 1

BJMG
BJMG

Reputation: 43

Just wrote a quick code to Automate @Dwijay Bane 's answer

import os
import inspect
import torch

# Find the location of the torch package
package_path = os.path.dirname(inspect.getfile(torch))
full_path=os.path.join(package_path,'utils/data/sampler.py')
# Read in the file
with open(full_path, 'r') as file :
  filedata = file.read()

# Replace the target string
filedata = filedata.replace('generator = torch.Generator()', 'generator = torch.Generator(device=\'cuda\')')
filedata = filedata.replace('yield from torch.randperm(n, generator=generator).tolist()', 'yield from torch.randperm(n, generator=generator, device=\'cuda\').tolist()')

# Write the file out again
with open(full_path, 'w') as file:
  file.write(filedata)

Upvotes: 1

Zaccharie Ramzi
Zaccharie Ramzi

Reputation: 2316

Reading the link provided in @Dwijay 's answer, I found an answer that does not require you to do any source code change. Indeed, it is very dangerous I would say to change PyTorch source code. But the idea of modifying the Generator is the good one. Indeed by default the random number generator generates numbers on CPU, but we want them on GPU.

Therefore, one should actually modify the data loader instantiation to fit the use of the default cuda device. This is highlighted in this GitHub comment:

data_loader = data.DataLoader(
    ...,
    generator=torch.Generator(device='cuda'),
)

This fix worked for me in PyTorch 1.11 (and worked for this other user in PyTorch 1.10).

Upvotes: 25

Dwijay Bane
Dwijay Bane

Reputation: 118

I had same issue but on ubuntu20.04 I have tried turning shuffle off as mentioned and that worked but its not correct way as it will make your training worse.

Keep the shuffle ON and follow below step, these would vary according to pytorch version:

  1. In file "site-packages/torch/utils/data/sampler.py" located in anaconda or wherever.
  • [Modify line 116]: generator = torch.Generator()
    • change to generator = torch.Generator(device='cuda')
  • [Modify line 126]: yield from torch.randperm(n, generator=generator).tolist()
    • change to yield from torch.randperm(n, generator=generator, device='cuda').tolist()

Line number could be different for different version but point to note is adding device='cuda' to functions.

Hope this helps!!!

Upvotes: 4

Fadwa
Fadwa

Reputation: 1931

Turning the shuffle parameter off in the dataloader solved it. Got the answer form here.

Upvotes: 6

Related Questions