CODEWITHSUNDEEP
multithreadingscala
nir
nir

Reputation: 3868

scala - best way to do parallel constant polling and processing

I am trying to figure out what is the best way to do constant polling in async and non blocking way. Entire goal of the application is to start few threads and with each thread do constant polling on external service (kafka) to get data; each thread then can process that data or hand it over to some other thread. I don't see a way to do this just with scala Future as it requires timeout value. I can set it to a year but that still doesn't seem like a good solution. e.g. Await.result(future, 365 days) Any pointers ?

Upvotes: 0

Views: 209

Answers (1)

Knows Not Much
Knows Not Much

Reputation: 31546

There are couple of Async Non-Blocking Kafka libraries. You can write a consumer in any of these to pull data from Kafka topics.

https://github.com/cakesolutions/scala-kafka-client

https://github.com/akka/reactive-kafka

Upvotes: 1

Related Questions