CODEWITHSUNDEEP
apache-sparkhiveakkaapache-nifiakka-stream
ASe
ASe

Reputation: 535

Akka stream vs Hive Stream

I am working on a requirement where we need to read messages from Kafka and save (sink) to Hive. I can think about of multiple implementation using different technologies:

  1. Akka stream - where source will be Kafka source and sink to hive
  2. Hive Stream - using hive streaming
  3. Spark streaming
  4. nifi - https://nifi.apache.org/

What would be best way to handle large set of kafka messages to stream with Hive?

Thanks Arun

Upvotes: 1

Views: 391

Answers (1)

Dennis Jaheruddin
Dennis Jaheruddin

Reputation: 21563

Best is of course a very vague concept, but I personally like NiFi as a data movement solution.

If you are looking for fast development, and clear monitoring then the intuitive GUI should prove very valuable.

If you find that you cannot get enough performance, or good enough latency, you might be able to improve with Spark Streaming, but often that should not be needed.

Ful disclosure: Have not worked with Akka Streams, and work for Cloudera a driving force behind Nifi, Spark and Hive

Upvotes: 0

Related Questions