CODEWITHSUNDEEP
message-hub
Bob Dill
Bob Dill

Reputation: 1010

why do I need to poll message hub?

I am looking at the node-js express example for message-hub on blue mix and am puzzled for why I need to poll message hub from my server. I thought that the whole idea behind a pub-sub model is that I don't have to (over) load my server polling the message service to find out if new messages exist for me to consume. In the example provided, lines 211-213 in the app.js file contain the following: 

  // Set up an interval which will poll Message Hub for
  // new messages on the 'livechat' topic.
  produceInterval = setInterval(function() { ...},250);

This now has my server polling message hub every 250 mSec when what I want is to completely avoid a polling model and be notified by message hub when a message exists for me to consume.

Upvotes: 0

Views: 89

Answers (2)

Hans Jespersen
Hans Jespersen

Reputation: 8335

With KafkaConsumer the poll() function means that your app is polling the client buffer, not necessarily polling across the network to the Kafka broker. Kafka clients often prefetch and cache data in this client side buffer for better performance, lower latency, and better network efficiency.

If you want an async callback style interface that "pushes" data to your app then it's very easy to wrap the polling interface and make it look like a push. Ultimately every push API has something hidden under the covers that is calling poll on a tcp socket.

Upvotes: 1

Edoardo Comar
Edoardo Comar

Reputation: 551

in brief: Kafka achieves its scalability by using a consumer-pull model, not a server-push.

in detail: It's worth going through the Kafka documentation first. In particular, your question is answered here http://kafka.apache.org/documentation/#design_pull

HTH, Edo

Upvotes: 2

Related Questions