communication with 0mq

Question

I'm trying to create a system for collecting data sent I have two small scripts, a receiver.py, which should receive the data, and sender that should send it, for the moment I try with the 1-1 connection, but finally I need multiple senders and one receiver which would process the incoming data. I try to accomplish this using 0mq publisher/subscriber pattern.

#receiver.py

def receive():
    context = zmq.Context()
    socket = context.socket(zmq.SUB)
    socket.setsockopt(zmq.SUBSCRIBE, 'Child:') 
    socket.bind('tcp://localhost:5000')
    while True:
        print 'Parent received: %s' % socket.recv()

receive()

#sender.py

def send(data):
    context = zmq.Context()
    socket = context.socket(zmq.PUB)
    socket.connect('tcp://localhost:5000')
    socket.send('Sender: %i' % data)
    socket.close()
    print "sent"

send(10)

When I start receiver.py it just waits for the data, doesn't receive anything when I run sender.py. I'd be grateful for suggestions, actually I'm not even sure if the publisher/subscriber is the best pattern for my scenario (multiple sensors sending data over the local network to one server for real time processing).

Answer 1

In order to understand what happen I've rewrite it:

import zmq
import threading
import time

def receive():
    context = zmq.Context()
    socket = context.socket(zmq.SUB)
    socket.bind("tcp://127.0.0.1:5000")
    socket.setsockopt(zmq.SUBSCRIBE, '')
    while True:
        print 'Parent received: %s' % socket.recv()

threading.Thread(target=receive).start()

def send(data):
    context = zmq.Context()
    socket = context.socket(zmq.PUB)
    socket.connect("tcp://localhost:5000")
    while data:
        socket.send('Sender: %i' % data)
        data -= 1
        time.sleep(1)

Output:

>>> send(10)
Parent received: Sender: 9
Parent received: Sender: 8
Parent received: Sender: 7
Parent received: Sender: 6
Parent received: Sender: 5
Parent received: Sender: 4
Parent received: Sender: 3
Parent received: Sender: 2
Parent received: Sender: 1

The key points are:

1. As described in Why doesn't zeromq work on localhost? you MUST use 127.0.0.1 to bind your localhost
2. The publisher doesn't expose a name so we must remove Child: from subcriber or change it to Sender: or change in send() Sender: in Child: (I've chose the first)
3. Maybe you need some kind of events that say the connection of subscriber is active before send data (zeromq is an asynchronous framework) otherwise you lose the first message. As proof you can change send() like follow example and you doesn't lose any message.

.

def send(data):
    context = zmq.Context()
    socket = context.socket(zmq.PUB)
    socket.connect("tcp://localhost:5000")
    while data:
        time.sleep(1)
        socket.send('Sender: %i' % data)
        data -= 1
    socket.close()

Output:

>>> send(10)
Parent received: Sender: 10
Parent received: Sender: 9
Parent received: Sender: 8
Parent received: Sender: 7
Parent received: Sender: 6
Parent received: Sender: 5
Parent received: Sender: 4
Parent received: Sender: 3
Parent received: Sender: 2
Parent received: Sender: 1

Answer 2

If you're not able to get the examples to work, it's a good sign that either there's something wrong with your ZMQ library installation, or perhaps your library version is not compatible with the binding you're using. Start checking there, and always try to run an example verbatim first to make sure that everything at least works with reference code.

But, I do see at least one issue with your code that would cause you to never receive messages.

In your subscriber, you subscribe to 'Child:', but in your publisher you never send a message that matches that. The "correct" way to do this would be to send a multi-frame message, but for simplicity of code you can also send a string that begins with your topic, like so:

socket.send('Child: Sender: %i' % data)

Alternatively, you can alter your subscriber to fit your current message pattern:

socket.setsockopt(zmq.SUBSCRIBE, 'Sender:')

And, last but not least, if you want to subscribe to everything that might be sent by your publishers, you can subscribe to the empty string:

socket.setsockopt(zmq.SUBSCRIBE, '')

... this last one is probably correct for your situation. So, the resulting code would be as follows:

#receiver.py

def receive():
    context = zmq.Context()
    socket = context.socket(zmq.SUB)
    socket.setsockopt(zmq.SUBSCRIBE, '') 
    socket.bind('tcp://localhost:5000')
    while True:
        print 'Parent received: %s' % socket.recv()

receive()

#sender.py

def send(data):
    context = zmq.Context()
    socket = context.socket(zmq.PUB)
    socket.connect('tcp://localhost:5000')
    socket.send('Sender: %i' % data)
    socket.close()
    print "sent"

send(10)

Addressing sub-points, questions and comments:

• In this scenario it's more appropriate for the subscriber to bind() and the publishers to connect() (as you've done), ZMQ doesn't care which you use based on socket type alone. Your subscriber is your "server", your publishers are your "clients", so SUB should bind(), PUB should connect().
• PUB/SUB is a fine pattern for your scenario, it's good whenever communication is strictly one-way, as in your case.