CODEWITHSUNDEEP
apache-kafkakafka-consumer-apiapache-pulsar
Vinay B
Vinay B

Reputation: 733

Apache Pulsar vs Kafka - do consumers pull (poll) messages off the topics?

I know that in Kafka, the consumer pulls messages off the broker topics (pull) ?

I get the feeling that Pulsar works the same way, considering that the receive method blocks. But I can't find a confirmation. Can someone point me to a reference or correct me ?

Thanks

Upvotes: 5

Views: 1554

Answers (2)

Hieu Vo
Hieu Vo

Reputation: 21

In Pulsar document:

There is a queue at the consumer side to receive messages pushed from the broker. You can configure the queue size with the receiverQueueSize parameter. The default size is 1000). Each time consumer.receive() is called, a message is dequeued from the buffer.

So broker pushes messages to a queue on consumer side. When the receive method is invoked, a message will be dequeued and returned.

Pulsar consumer will regularly send permit request to Pulsar broker to ask for more message when the half of the queue is consumed. This is described here.

In short, as described here

Pulsar also uses a push-based approach but with an API that simulates consumer pulls.

Upvotes: 2

Giorgos Myrianthous
Giorgos Myrianthous

Reputation: 39800

Pulsar's Documentation clearly explains how message consumption works:

The Pulsar Consumer origin reads messages from one or more topics in an Apache Pulsar cluster.

The Pulsar Consumer origin subscribes to Pulsar topics, processes incoming messages, and then sends acknowledgements back to Pulsar as the messages are read.

Messages can be received from brokers either synchronously (sync) or asynchronously (async).

receive method receives messages synchronously. The consumer process will be blocked until a message becomes available. For example, 

Message msg = consumer.receive();

An asynchronous receive will return immediately with a value of type CompletableFuture that completes once a new message is available. For example, 

CompletableFuture<Message> asyncMessage = consumer.receiveAsync();

Upvotes: 2

Related Questions