CODEWITHSUNDEEP
pythonwindowspytorchmultiprocessing
yangxiang_li
yangxiang_li

Reputation: 179

Using `multiprocessing' in PyTorch on Windows got errors-`Couldn't open shared file mapping: <torch_13684_4004974554>, error code: <0>'

I am currently running a PyTorch code on Windows10 using PyCharm. This code firstly utilised DataLoader function (`num_workers'=4) to load training data:

train_loader = DataLoader(train_dset, batch_size, shuffle=True,
                              num_workers=4, collate_fn=trim_collate)

Then, in training process, it utilised a `for' loop to load training data and train the model:

for i, (v, norm_bb, q, target, _, _, bb, spa_adj_matrix,
                    sem_adj_matrix) in enumerate(train_loader):

Error: I got the following error messages when running above `for' loop:

    0%|          | 0/6934 [00:00<?, ?it/s]Traceback (most recent call last):
  File "E:\PyTorch_env\lib\multiprocessing\popen_spawn_win32.py", line 89, in __init__
    reduction.dump(process_obj, to_child)
  File "E:\PyTorch_env\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
  File "E:\PyTorch_env\lib\site-packages\torch\multiprocessing\reductions.py", line 286, in reduce_storage
    metadata = storage._share_filename_()
RuntimeError: Couldn't open shared file mapping: <torch_13684_4004974554>, error code: <0>
python-BaseException
Traceback (most recent call last):
  File "E:\PyTorch_env\lib\multiprocessing\spawn.py", line 115, in _main
    self = reduction.pickle.load(from_parent)
EOFError: Ran out of input
python-BaseException
  0%|          | 0/6934 [00:07<?, ?it/s]

It seems that there are some issues with the `multiprocessing' function on Windows10.

The environment settings:

  1. Windows10, PyCharm
  2. PyTorch v1.0.1, torchvision v0.2.2, Python 3.7.11
  3. One GPU node

Could you please let me know if there are any possible solutions for this?

Many thanks!

Upvotes: 0

Views: 1229

Answers (1)

Tomasz Kaczmarski
Tomasz Kaczmarski

Reputation: 136

Use as above num_workers=0 and for error expected long datatype but got Int stead. apply criterion(outputs_t.float(), target_t.flatten().type(torch.LongTensor))

Upvotes: 2

Related Questions