ZeroMQ 2 REQ/REP to allow nested talk

Question

I want the following to happen:

There is a main_socket, where anyone can talk to.

Clients will send "READ" and wait for "OK" from Server before doing so.

Clients will send "WRITE", when they get a "OK", then it will mean they can do a write action. Everyone should wait for this. So I thought another REQ/REP channel, which is nested in the main loop. The server will start listening for a message on the write_socket, and when the client has written, it will send a message to the write_socket.

But this unfortunately does not work, and I have no idea why.

server.py

import time
import zmq

context = zmq.Context()
main_socket = context.socket(zmq.REP)
main_socket.bind("tcp://*:5555")

write_socket = context.socket(zmq.REP)
write_socket.bind("tcp://*:5556")

print("ready to receive")
while True:
    #  Wait for next request from client
    print("server receiving main")
    message = main_socket.recv()
    print("server received main", message)
    if message == b"WRITE":
        main_socket.send(b"WRITE")    
        print("server receiving write")
        message = write_socket.recv()
        print("server received write", message)
        write_socket.send(b"OK")    
    else:
        main_socket.send(b"OK")

client.py

import zmq
import time

context = zmq.Context()

#  Socket to talk to server
print("Connecting to main server…")
main_socket = context.socket(zmq.REQ)
main_socket.connect("tcp://localhost:5555")

print("Connecting to write server…")
write_socket = context.socket(zmq.REQ)
write_socket.connect("tcp://localhost:5556")

print("starting")
t1 = time.time()
for i in range(10000):
    print("client sending main", b"WRITE")
    main_socket.send(b"WRITE")
    print("client receiving main")
    message = main_socket.recv()
    print("client received main", message)

    print("client writing")
    print("writing...")
    print("client written")

    time.sleep(5)
    print("client sending write", b"WRITE")
    write_socket.send(b"WRITE")

    print("client receiving write")
    message = write_socket.recv()
    print("client received write", message)

This prints the following:

server

ready to receive
server receiving main
server received main b'WRITE'
server receiving write

client

Connecting to read server…
Connecting to write server…
starting
client sending main b'WRITE'
client receiving main
client received main b'WRITE'
client writing
client written
client sending write b'WRITE'
client receiving write

How can I make this case work?

Answer 1

Your design concept is fragile in two principal layers:

so even if solving just the former will leave you absolutely powerless in the latter.

---

1 An Inherent Fragility of the Transport-Class :
First, the associated transport-class level fragility ( not getting the previously released or externally used tcp-port ). Given this, one might move for some co-located architecture move the signalling/messaging services onto some non-tcp based transport classes, most probably some of the other { inproc:// | ipc:// | vmci:// } but that does not solve the LAN/WAN-distributed operations scenario, plus, it would still contain a remarkable risk of not solving the second kind of fragility, that also requires some robust solution:

---

[2] An Inherent Fragility of the REQ/REP Scaleable Formal Communitations Pattern : Any serious application design ought bear in mind, that the elementary ( primitive ) REQ/REP pattern is vulnerable for falling into a non-salvageable mutual dead-lock. For details about this principal danger, kindly check my other posts on ZeroMQ, the REQ/REP deadlocks are quite frequent kind of collisions, but principally speaking here, if the application is going to be meant seriously for some kind of field deployment, then kindly forget to use the "naked"-REQ/REP primitive, otherwise you leave your application to a free-fall into a hierarchical-FSA deadlock, that cannot be self-detected from inside your application and thus cannot be reactively handled, or salvaged.

The good news is, there are many other simple pattern archetypes available in ZeroMQ, as building blocks, that allow one to integrate and fast-prototype a robust signalling / messaging application groundworks. So do not hesitate to empower your application with much richer inter-node signalling / messaging, that would not deadlock each other ( the less an any-to-any infrastructure-wide deadlock ... ).

Going this way, your both your try: / except: + assert Exception-handlers and non-blocking { .recv() | .send() }-s would remain in control should some otherwise un-expected condition would appear ( and be sure it -- sooner or later -- will appear ) in-vivo.

Answer 2

This is just a hunch, but: the problem might be that it is possible to successfully bind 0.0.0.0:5556 even though there also is a bound socket say for localhost:5556 i.e. 127.0.0.1:5556.

In that case a socket bound to 0.0.0.0:5556 would continue to receive connections to any other address except 127.0.0.1; and including IPv6, but the connection attempts to 127.0.0.1:5556 would be accepted by another TCP server socket elsewhere.