CODEWITHSUNDEEP
phpzeromq
J-Rou
J-Rou

Reputation: 2286

zeromq - always loses the first message

I've been trying to set up a simple "pipeline" with zeromq between my local computer and a remote server. I tested the script on my local computer and it worked perfectly, but when I tried to use the remote computer as the end (The one with the SOCKET_PULL), I started having problems. The first message sent by any process was always lost.

Th code is simple,

The push (sender.php):

<?php
//Use the specified port or 5555 for sending jobs
$port = $_SERVER['argc'] > 1 ? $_SERVER['argv'][1] : '5555';
$jobN = $_SERVER['argc'] > 2 ? $_SERVER['argv'][2] : '';


$context = new ZMQContext();

//  Socket to send messages on
$sender = new ZMQSocket($context, ZMQ::SOCKET_PUSH);

//If I want to connect to the server I use this line
$sender->connect("tcp://my-server-address.com:$port");

//If I want to connect to localhost I use this line
//$sender->connect("tcp://localhost:$port");

$sender->send('job-1' . ($jobN ? " $jobN" : ''));
$sender->send('job-2' . ($jobN ? " $jobN" : ''));
$sender->send('job-3' . ($jobN ? " $jobN" : ''));
$sender->send('job-4' . ($jobN ? " $jobN" : ''));
$sender->send('job-5' . ($jobN ? " $jobN" : ''));
$sender->send('job-6' . ($jobN ? " $jobN" : ''));
$sender->send('job-7' . ($jobN ? " $jobN" : ''));
$sender->send('job-8' . ($jobN ? " $jobN" : ''));
$sender->send('job-9' . ($jobN ? " $jobN" : ''));
$sender->send('job-10' . ($jobN ? " $jobN" : ''));
echo 'done';

The receiver (worker.php)

<?php
//Use the specified port or 5556 for getting finished jobs
$port = $_SERVER['argc'] > 1 ? $_SERVER['argv'][1] : '5555';

//  Prepare our context and socket
$context = new ZMQContext();
$receiver = new ZMQSocket($context, ZMQ::SOCKET_PULL);
$receiver->bind("tcp://*:$port");

$count = 1;
while(true) {
    $string = $receiver->recv();
    echo "Received $string $count\n";
    $count += 1;
}

If I run the code in my local ("php worker.php" in one terminal and "php sender.php" in another).

I get 

Received job-1 1
Received job-2 2
Received job-3 3
Received job-4 4
Received job-5 5
Received job-6 6
Received job-7 7
Received job-8 8
Received job-9 9
Received job-10 10

but if I use the remote (sender in mu local and receiver in the remote), I get

Received job-2 1
Received job-3 2
Received job-4 3
Received job-5 4
Received job-6 5
Received job-7 6
Received job-8 7
Received job-9 8
Received job-10 9

Am I doing something wrong??

NOTE: I can't put locks on the sender (and SOCKET_REQ locks the execution waiting for the reply). In any way, 0mq is supposed to work without locks. NOTE: My local is a mac computer and the server is an amazon instance with ubuntu (I don't think that will affect, but I'm writing it just in case it does).

EDIT: Just to clarify, I don't want to send data trough internet, I just wanted to test zeromq with high latency when I encountered this problem.

Upvotes: 3

Views: 3757

Answers (2)

Glenn Wesley
Glenn Wesley

Reputation: 81

The problem is what Pieter's documentation calls the "slow joiner". This is misnamed because it implies it's the subscriber/listener is the root of the problem. This is not the case.

The problem is most likely on the publisher side. You are sending your message before the publisher is connected.

How can this be? zmq_connect or zmq_bind returns so you must be ready. No, you are not.

Apparently the connection/publishing setup is not complete when the connect/bind returns. The connection is finished some time after the function returns.

So how do you solve the problem? The easy way is to wait. The problem with this is you don't know how long to wait and the actual time depends on the particular network situation and the hardware involved.

A better way, but more involved, is to set up a local subscriber for your publisher address and repeatedly publish an administrative message which once received lets you know you are up and ready.

Upvotes: 7

J-Rou
J-Rou

Reputation: 2286

Solved the problem, I had different versions of zeromq installed on my local and on the server. If you get the same error, check the versions.

NOTE: Zeromq does work with internet. The only thing (at least in php) is that you need to be sure that the message was sent completely before closing the process. in my test, I sent several times 75000 messages of random sizes (Average size: 1MB) between different computers and I got all the messages as expected.

Upvotes: 4

Related Questions