CODEWITHSUNDEEP
pythonmultiprocessing
Mahdi Haqverdi
Mahdi Haqverdi

Reputation: 11

Multiprocessing in Python

I am trying to apply multiprocessing in the simplest way in Python 3 but it does not work on my laptop. I am using Windows.

from multiprocessing import Process

# a dummy function
def f(x):
    print(x)

if __name__ == '__main__':
    p = Process(target=f, args=('some text',))
    p.start()
    p.join()

print('Done')

It did not end as it was expected. Instead, I got this error:

Traceback (most recent call last):
File "C:\Users\Mahdi\anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 3437, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-2-46a0e604292b>", line 1, in <module>
runfile('C:/Users/Mahdi/Mahdi Code/test.py', wdir='C:/Users/Mahdi/Mahdi Code')
File "C:\Program Files\JetBrains\PyCharm 2021.2.2\plugins\python\helpers\pydev\_pydev_bundle\pydev_umd.py", line 198, in runfile
pydev_imports.execfile(filename, global_vars, local_vars)  # execute the script
File "C:\Program Files\JetBrains\PyCharm 2021.2.2\plugins\python\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "C:/Users/Mahdi/Mahdi Code/test.py", line 25, in <module>
p.start()
File "C:\Users\Mahdi\anaconda3\lib\multiprocessing\process.py", line 121, in start
self._popen = self._Popen(self)
File "C:\Users\Mahdi\anaconda3\lib\multiprocessing\context.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "C:\Users\Mahdi\anaconda3\lib\multiprocessing\context.py", line 327, in _Popen
return Popen(process_obj)
File "C:\Users\Mahdi\anaconda3\lib\multiprocessing\popen_spawn_win32.py", line 93, in __init__
reduction.dump(process_obj, to_child)
File "C:\Users\Mahdi\anaconda3\lib\multiprocessing\reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <function f at 0x00000218267C3D30>: attribute lookup f on __main__ failed
Done
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Users\Mahdi\anaconda3\lib\multiprocessing\spawn.py", line 116, in spawn_main
exitcode = _main(fd, parent_sentinel)
File "C:\Users\Mahdi\anaconda3\lib\multiprocessing\spawn.py", line 126, in _main
self = reduction.pickle.load(from_parent)
EOFError: Ran out of input

Does anyone know about this?

Upvotes: 1

Views: 157

Answers (2)

Shubhank Gupta
Shubhank Gupta

Reputation: 825

You can run the same code via the Terminal or Pycharm. I haven't checked other IDE's.

It ran fine on the terminal and the output is

some text
Done

Upvotes: 0

Ahmed AEK
Ahmed AEK

Reputation: 18090

some interactive IDEs don't support multiprocessing such as jupyter lab, there are tricks around this involving non-standard multiprocessing modules, but the most straight-forward solution is to not use an interactive environment for multiprocessed code, and instead run python in script mode using VsCode or Pycharm or through the terminal. (Spyder also works but you have to run the code as a script)

Upvotes: 5

Related Questions