CODEWITHSUNDEEP
apache-kafka
Magick
Magick

Reputation: 5112

Apache Kafka client with selector?

I am investigating Apache Kafka, and my use-case requires that the client be able to filter messages, and that the filtering happens on the broker, rather than the client. This requirement is identical to using a JMS Selector.

It is my understanding, though I maybe incorrect, that Kafka does not have a selector, and that filtering is done on the client. So all messages are sent to client, and client is responsible for filter.

Is there anyway to achieve a JMS Selector approach, where filtering happens on the broker?

Upvotes: 5

Views: 5764

Answers (2)

Roman Sinyakov
Roman Sinyakov

Reputation: 606

You're right that filtering is done on the client-side.

Spring Cloud Stream provides an elegant way of implementing that with the support of dispatching messages to multiple handler methods annotated with @StreamListener based on conditions: https://docs.spring.io/spring-cloud-stream/docs/current/reference/htmlsingle/#spring-cloud-stream-overview-producing-consuming-messages

Upvotes: 0

serejja
serejja

Reputation: 23851

No, you can't do anything like that. The best you can do is filter on the client side. Apache Kafka API supports only fetching of kind "give me at most 1 megabyte of messages for topic N partition M starting from offset X".

You can refer to Wire Protocol reference to see possible options for a fetch request, but it does not contain anything you are looking for.

Actually, Kafka is that fast because every fetch is just a sequential read, so I don't think this will be implemented in future releases (but I'll be glad if I'm wrong :)).

Upvotes: 4

Related Questions