CODEWITHSUNDEEP
javascalamessagingapache-kafka
Max Sang
Max Sang

Reputation: 218

Consume all of a Kafka topic and then immediately disconnect?

I'm mostly using Kafka for traditional messaging but I'd also like the ability to consume small topics in a batch fashion, i.e. connect to a topic, consume all the messages and immediately disconnect (not block waiting for new messages). All my topics have a single partition (though they are replicated across a cluster) and I'd like to use the high-level consumer if possible. It's not clear from the docs how I could accomplish such a thing in Scala (or Java). Any advice gratefully received.

Upvotes: 1

Views: 1161

Answers (1)

Lundahl
Lundahl

Reputation: 6562

The consumer.timeout.ms setting will throw a timeout exception after the specified time if no message is consumed before and this is the only option you have with the high level consumer afaik. Using this you could set it to something like 1 second and disconnect after that if it's an acceptable solution.

If not, you'd have to use the simple consumer and check message offsets.

Upvotes: 1

Related Questions