CODEWITHSUNDEEP
pythonpython-3.xmultiprocessingimporterrorpython-3.4
Ryan G.
Ryan G.

Reputation: 305

Is multiprocessing in Python 3.4 broken?

I'm trying to use the multiprocessing module in Python 3. However, everytime I try to set up a Pool, I get the following traceback.

Traceback (most recent call last):
  File "testmp.py", line 7, in <module>
    with Pool(5) as p:
  File "/usr/lib/python3.4/multiprocessing/context.py", line 118, in Pool
    context=self.get_context())
  File "/usr/lib/python3.4/multiprocessing/pool.py", line 150, in __init__
    self._setup_queues()
  File "/usr/lib/python3.4/multiprocessing/pool.py", line 243, in _setup_queues
    self._inqueue = self._ctx.SimpleQueue()
  File "/usr/lib/python3.4/multiprocessing/context.py", line 110, in SimpleQueue
    from .queues import SimpleQueue
  File "/usr/lib/python3.4/multiprocessing/queues.py", line 20, in <module>
    from queue import Empty, Full
ImportError: cannot import name 'Empty'

The simplest code that can easily trigger this is the first example in the multiprocessing module documentation.

from multiprocessing import Pool

def f(x):
    return x*x

if __name__ == '__main__':
    with Pool(5) as p:
        print(p.map(f, [1, 2, 3]))

(source: https://docs.python.org/3.4/library/multiprocessing.html)

My question: Is this a bug in Python 3.4? Or am I doing something wrong? The equivalent code works in Python 2.7.

Upvotes: 1

Views: 2739

Answers (1)

Martijn Pieters
Martijn Pieters

Reputation: 1122342

You have a local module named queue; it interferes with the standard library module.

Remove it or rename it and your code will work again:

stackoverflow-3.4 mj$ cat test.py 
from multiprocessing import Pool

def f(x):
    return x*x

if __name__ == '__main__':
    with Pool(5) as p:
        print(p.map(f, [1, 2, 3]))

stackoverflow-3.4 mj$ touch queue.py
stackoverflow-3.4 mj$ bin/python test.py 
Traceback (most recent call last):
  File "test.py", line 7, in <module>
    with Pool(5) as p:
  File "/.../python3.4/multiprocessing/context.py", line 118, in Pool
    context=self.get_context())
  File "/.../python3.4/multiprocessing/pool.py", line 150, in __init__
    self._setup_queues()
  File "/.../python3.4/multiprocessing/pool.py", line 243, in _setup_queues
    self._inqueue = self._ctx.SimpleQueue()
  File "/.../lib/python3.4/multiprocessing/context.py", line 110, in SimpleQueue
    from .queues import SimpleQueue
  File "/.../lib/python3.4/multiprocessing/queues.py", line 20, in <module>
    from queue import Empty, Full
ImportError: cannot import name 'Empty'
stackoverflow-3.4 mj$ rm queue.py
stackoverflow-3.4 mj$ bin/python test.py 
[1, 4, 9]

If you have trouble locating it, run python3 -c 'import queue; print(queue.__file__)'.

Upvotes: 6

Related Questions