CODEWITHSUNDEEP
pythonmultiprocessingpipe
agemO
agemO

Reputation: 303

queue.close() "raises" BrokenPipeError

Using python 3.7 on Unix, the following code:

import multiprocessing

queue = multiprocessing.Queue()
queue.put(1)
# time.sleep(0.01)  Adding this prevents the error
queue.close()

"raises" (in the background) a BrokenPipeError:

Traceback (most recent call last):
  File "/usr/lib/python3.7/multiprocessing/queues.py", line 242, in _feed
    send_bytes(obj)
  File "/usr/lib/python3.7/multiprocessing/connection.py", line 200, in send_bytes
    self._send_bytes(m[offset:offset + size])
  File "/usr/lib/python3.7/multiprocessing/connection.py", line 404, in _send_bytes
    self._send(header + buf)
  File "/usr/lib/python3.7/multiprocessing/connection.py", line 368, in _send
    n = write(self._handle, buf)
BrokenPipeError: [Errno 32] Broken pipe

To me this clearly looks like a bug, since queue.close() together with queue.join_thread() only exist to avoid that kind of bugs.
Am I missing something?

Notice that the BrokenPipeError is only raised in the background thread which is internally used by python to feed the queue, so from the main process point of view no error is raise and the "only" consequence is just spurious tracebacks printing.

(Also related to multiprocessing.Queue fails intermittently. Bug in Python?)

Upvotes: 3

Views: 348

Answers (1)

Ahmed AEK
Ahmed AEK

Reputation: 17616

This is a bug in python, it happens in 3.7, 3.8 and 3.9, it is resolved in python 3.10, and later versions.

This is a link to the issue on cpython github

Upvotes: 3

Related Questions