CODEWITHSUNDEEP
message-queuezeromqmessaging
muella91
muella91

Reputation: 329

ZeroMQ Messaging Queues

I am working for a while with ZeroMQ. I read already some whitepapers and a lot from the guide, but one question remained open to me:

Lets say we use PUB-SUB. Where or on which system are the messaging queues? On the publisher system side, on the subscriber system side or something in between?

Sorry for the maybe stupid question. This picture is from the Broker vs. Brokerless whitepaper.

enter image description here

Upvotes: 6

Views: 4187

Answers (2)

user3666197
user3666197

Reputation: 1

Q : Where or on which system are the messaging queues?

The concept of the Zen-of-Zero, as built-into ZeroMQ uses smart and right enough approaches, on a case by case principle.

There cases, where there are no-Queues, as you know them, at all :

for example, the inproc:// transport-class has no Queue, as one may know them, as there are just memory-regions, across which the messages are put and read-from. The whole magic thus appears inside the special component - the Context()-instance. There the message-memory-mapping takes place. The inproc:// case is a special case, where no I/O-thread(s) are needed at all, as the whole process is purely memory-mapping based and the Context()-instance manipulates its internal states, so as to emulate both an externally provided abstraction of the Queue-alike behaviour and also the internal "Queue"-management.

Others, similarly, operate localhost-located internal Queue(s) endpoints :

for obvious reason, as the ZeroMQ is a broker-less system, there is no "central"-place and all the functionality is spread across the participating ( coordinated and cooperating ) nodes.

Either one of the Queue-ends reside inside the Context()-instances, one in the one-side's localhost-itself, the other in the "remote"-host located Context()-instance, as was declared and coordinated during building of the ZMTP/RFC-specified socket-archetype .bind()/.connect() construction. Since a positive acknowledgement was made for such a ZMTP-specific-socket, the Queue-abstraction ( implemented in a distributed-system-manner ) holds till such socket does not get .close()-ed or the Context()-instance did not get forcefully or by a chance .term()-ed.

For these reasons, the proper capacity sizings are needed, as the messages may reside inside the Context()-operated memory before it gets transported across the { network | ipc-channel }-mediated connection ( on the .send()-side ) or before being .recv()-ed ( from inside the remote Context()-instance ) by the remote application-code for any further use of the message-payload data ( a Zero-copy message-data management is possible, yet not all use-cases indeed use this mode for avoiding replicated memory-allocations and data-transfer costs ).

Upvotes: 3

jamesdillonharvey
jamesdillonharvey

Reputation: 1042

Every zeromq socket has both send and recv queues (The limits are set via high water mark).

In the case of a brokerless PUB/SUB the messages could be queued on the sender or the receiver.

Example:

  • If there is network congestion the sender may queue on its send queue
  • If the receiver is consuming messages slower than they arrive they will be queued on the receiver recv queue

If the queues reach the high water mark the messages will be lost.

Upvotes: 1

Related Questions