CODEWITHSUNDEEP
streamapache-kafkamessage-queuerpcgrpc
FindingTheOne
FindingTheOne

Reputation: 189

Reading Stream of data in Producer and push to Kafka Topic

I want to read data from a server listening to stream of data in Producer and send to Topic. The source of data is an client-side streaming RPC where the client writes a sequence of messages and sends them to the server using a provided stream.

I did found an example where Producer connects to a Twitter Client and reads data. However, my goal is to read the data as soon it comes on the server listening to stream in Producer and send to Kafka Topic.

Are there any examples similar or anyone has came across this scenario?

Any help will be appreciated.

Upvotes: 2

Views: 1114

Answers (1)

Michal Borowiecki
Michal Borowiecki

Reputation: 4324

It would help to know the specifics of the streaming RPC mechanism you are using, but as first port of call I'd look to akka-stream-kafka (aka reactive-kafka) as it is tailored for specifically for streaming scenarios: http://doc.akka.io/docs/akka-stream-kafka/current/producer.html

Check out the examples there. Also, alpakka is an initiative for integration patterns based on akka-stream but the list of connectors supported is so far somewhat limited: http://developer.lightbend.com/docs/alpakka/current/

Upvotes: 1

Related Questions