Reputation: 1265
What is the difference between the two? As far as I understand both of them are based on the concept of directed acyclic graph (DAG) and Storm processes data in real time and Airflow rather moves the whole thing from one stage to the other. Is it the only difference between them? What kind of jobs are both of them suitable for?
Upvotes: 1
Views: 1141
Reputation: 5967
Airflow is an orchestrating engine like Azkaban or Oozie. Storm is a near real-time processing engine which has largely been replaced by Spark Streaming or Heron (Twitter’s 2nd generation replacement for Storm).
Upvotes: 1