CODEWITHSUNDEEP
pythonmultithreadingsocketszeromqpyzmq
fuglede
fuglede

Reputation: 18201

Receiving zmq messages on background thread fails on Windows

I'm trying to set up a hello world style example of asynchronous communication between two peers with zmq.PAIR by receiving messages on a background thread while using console input to send messages:

server.py:

import zmq
import threading

context = zmq.Context()
socket = context.socket(zmq.PAIR)
socket.bind('tcp://*:5556')

def print_incoming_messages():
    while True:
        msg = socket.recv_string()
        print(f'Message from client: {msg}')

recv_thread = threading.Thread(target=print_incoming_messages)
recv_thread.start()

while True:
    msg = input('Message to send: ')
    socket.send_string(msg)

client.py:

import zmq
import threading

context = zmq.Context()
socket = context.socket(zmq.PAIR)
socket.connect('tcp://127.0.0.1:5556')

def print_incoming_messages():
    while True:
        msg = socket.recv_string()
        print(f'Message from server: {msg}')

recv_thread = threading.Thread(target=print_incoming_messages)
recv_thread.start()

while True:
    msg = input('Message to send: ')
    socket.send_string(msg)

This works completely fine on a Linux machine but socket.send_string blocks in either process when run from the Windows 10 command prompt. What is the reason for this discrepancy?

The socket is set up properly, flushing all outputs make no difference. The reading itself also works as expected as may be verified by navigating to 127.0.0.1:5556 in a browser. Looking at the loopback interface in Wireshark also reveals that the connection is set up properly, yet no messages are sent.

If I comment out recv_thread.start() in the client, however, messages are sent through as may be verified in Wireshark, which suggests that somehow socket.recv_string is blocking the socket from sending even though it isn't doing so on Linux.

I am also able to achieve the desired behavior by using two sets of PUSH/PULL (cf. this answer) but that doesn't quite help explain what's going on in the example at hand.

This is on Python 3.7.1, pyzmq 18.0.0, and libzmq 4.3.1 on both systems.

Upvotes: 2

Views: 1225

Answers (1)

minrk
minrk

Reputation: 38588

zmq sockets are not threadsafe, so running send and recv on the same socket in different threads should not be expected to work. Different threading behaviors on different platforms may be responsible for the difference in behavior you are seeing, but this code could also result in segfaults eventually due to the thread-unsafety of zmq sockets. Using a Lock might solve the problem.

As a side note, PAIR is a rarely-used socket type, and not often intended for use in production or inter-process communication. Most real-world instances of PAIR are as inproc sockets for inter-thread communication. PAIR can have weird behavior on reconnect, for example. Using PUSH-PULL for one-way or DEALER-DEALER for two-way communication is likely to behave in a more expected fashion.

Upvotes: 2

Related Questions