CODEWITHSUNDEEP
pythonmultithreadingmessage-queuezeromqpyzmq
Arsenal591
Arsenal591

Reputation: 1626

ZeroMQ: set LINGER=0 does not work as expected

I'm using Python bindings for ZeroMQ. My libzmq version is 4.2.5 and my pyzmq version is 17.1.2.

I'm trying to let a "producer" transmit a large amount of data to a "consumer". The code of the "producer" is :

# producer.py
import zmq
import time
import os

ctx = zmq.Context()
sock = ctx.socket(zmq.PUB)
sock.bind('tcp://*:8000')

x = os.urandom(1000000000) # large amount of data, requires much time to transmit it
sock.send(x)
print('Done')

sock.setsockopt(zmq.LINGER, 0)
sock.close()

t1 = time.time()
ctx.term() # I expect this should return immediately
print(time.time() - t1)

And the code of "consumer" is :

# consumer.py
import zmq

ctx = zmq.Context()
sock = ctx.socket(zmq.SUB)

sock.setsockopt_string(zmq.SUBSCRIBE, '')
sock.connect('tcp://localhost:8000')

data = sock.recv()

I expect the ctx.term() in the producer.py should return immediately, since the LINGER of the socket is already set to zero. But when I run these codes, the ctx.term() does not return immediately as expected. Instead, it takes tens of seconds for that function to return, and all of the large data have been successfully received by the consumer.py.

I am trying to figure out why, and I wish someone help me out a little.

Upvotes: 4

Views: 5973

Answers (2)

WallStProg
WallStProg

Reputation: 401

AFAIK, this is still a problem

It can be avoided by setting the linger option at socket create time, rather than right before close.

Upvotes: 0

user3666197
user3666197

Reputation: 1

Q : "ZeroMQ: set LINGER=0 does not work as expected"

ZeroMQ set LINGER=0 does IMHO work as expected ( as documented ):

ZeroMQ documentation is clear in stating that all the zmq_setsockopt() calls ( wrapped, for a use in python, into a method .setsockopt() ) take effect, i.e. modify Socket-instances' behaviour.

Older versions of ZeroMQ documentation ( being in use in my Projects with ZeroMQ-wrapped distributed systems since v2.x ) were more explicit on this:

Caution: All options, with the exception of ZMQ_SUBSCRIBE, ZMQ_UNSUBSCRIBE, ZMQ_LINGER, ZMQ_ROUTER_MANDATORY and ZMQ_XPUB_VERBOSE only take effect for subsequent socket bind/connects.

Having this in mind, the sock.setsockopt( LINGER, 0 ) indeed instructs the Socket()-instance sock not to wait on a respective <aContextINSTANCE>.term() until all its attempts to complete all the yet-enqued-only messages get fully propagated to the queue-head-end and there processed into the wireline-protocol and under its supervision being successfully sent or accepted to have been lost on their network-wide way to the neighbouring counterparty.

Yet, this does not say, what is going to be done with the data-in-transit, that the Context()-instance is already moving down the wire.

As far as I have worked extensively with ZeroMQ since v2.x, nothing IMHO reminds me about a way, how to interrupt an ongoing message transport, using the ZeroMQ semantics exposed to the public API, beyond the LINGER-instructed behaviour which might be paraphrased like :
" IGNORE ANY KNOWN SENDS/RECEIVES THAT STILL WAIT ITS TURN IN QUEUE ",
yet, this does not stop the progress of sending the data-in-transit down the line.

ZeroMQ works intentionally this way.

One might like to read more about ZeroMQ internals here or perhaps to just have a general view from the orbit-high perspective as in "ZeroMQ Hierarchy in less than a Five Seconds".

Epilogue : For Use in a Last Resort Case Only

If indeed in an ultimate need to have some way to stop even these in-transit message flows, feel free to post a new question on how to make things work that wild way.

Upvotes: 4

Related Questions