CODEWITHSUNDEEP
zeromqpublish-subscribepyzmq
Rahul.Shikhare
Rahul.Shikhare

Reputation: 179

How to get the Message/Events count in ZMQ Publisher/Subscriber QUEUE

We implemented the ZMQ PUB/SUB mechanism for message transfer between distributed processes. But because of the subscriber's processing time, the messages are processed with some delay (sometimes the delay is in hours because of the number of queued messages). To overcome this delay I am planning to scale UP/DOWN the subscribers-processes depending upon the number of pending messages into Publisher Queue.

Is there any mechanism to get the count/length of the ZMQ Publisher Queue?

Currently, I am considering the Publisher-server RAM utilization threshold to scale UP/DOWN the Subscriber processes.

Upvotes: 0

Views: 771

Answers (1)

user3666197
user3666197

Reputation: 1

Q : How to get the Message/Events count in ZMQ Publisher/Subscriber Queue?

PUB-side has two options:
1 ) simple counting of all .send()-calls avoids the "litter bug" in your program design
2 ) if not using the explicit counting, may solve the "litter" sorting out, using socket_monitor tooling ( at your additional costs of doing so )

SUB-side is a passive recipient of message(s), thus it has but one option:
- create your own, application-specific detector, using the socket_monitor tool (analyzing all the behind-the-API events, no other way to cheat behind the curtain of the published API abstraction)

Q : Is there any mechanism to get the count/length of the ZMQ Publisher Queue?

(ref.above) + depends on where the Observer "mechanism" stands:

Adding another, independently working signalling control-plane would help cross-breed these principles, so as to augment all the relevant control-data from both sides to get the full picture of the queued traffic.

Upvotes: 0

Related Questions