CODEWITHSUNDEEP
zeromq
breathingdust
breathingdust

Reputation: 205

Correct socket types for a message catchup mechanism?

I have a single publisher application (PUB) which has N number of subscribers (SUB)

These subscribers need to be able to catch up if they are restarted, or fall down and miss messages.

We have implemented a simple event store that the publisher writes to. We have implemented a CatchupService which can query the event store and send missed messages to the subscriber.

We have implemented in the subscriber a PUSH socket which sends a request for missed messages. The subscriber also has a PULL socket which listens for missed messages on a seperate port.

The subscriber will:

  1. Detect a gap
  2. Send a request for missed messages to our CatchupService, the request also contains the address on which to send the results to.
  3. The catchup service has a PULL socket on which it listens for requests
  4. When the CatchupService receives a request it starts a worker thread which:
    1. Gets the missed messages
    2. Opens a PUSH socket connecting to the subscribers PULL socket
    3. Sends the missed messages to the subscriber.

This seems to work quite well however we are unsure if we are using the right socket types for this sort of application. Are these correct or should be using a different pattern.

Upvotes: 0

Views: 68

Answers (1)

S.Richmond
S.Richmond

Reputation: 11558

Sounds okay. Otherwise 0MQ is able to recovery from message loss when peers go offline for a short time. Take a look at the Socket Options and specifically option ZMQ_SNDHWM.

I don't know just how guaranteed the 0MQ recovery mechanisms are so maybe you're best to stay with what you've got, but it is something to be aware of.

Upvotes: 1

Related Questions