Apache Flink vs Twitter Heron?

Question

There are a lot of questions comparing Flink vs Spark Streaming, Flink vs Storm and Storm vs Heron.

The origin of this question is from the fact that both Apache Flink and Twitter Heron are true stream processing frameworks (not micro-batch, like Spark Streaming). Storm has been decommissioned by Twitter last year and they're using Heron instead (which is basically Storm reworked).

There are nice presentations by Slim Baltagi on Flink and Flink vs Spark: https://www.youtube.com/watch?v=G77m6Ou_kFA

Nice research by Ilya Ganelin on various streaming frameworks: https://www.youtube.com/watch?v=KkjhyBLupvs

Pretty interesting thoughts on Flink vs Storm: What is/are the main difference(s) between Flink and Storm?

But I haven't seen any comparison of new Storm/Heron vs Apache Flink.

Both of the projects are pretty young, both support using previously written Storm applications and many other things. Flink is more fitting into Hadoop ecosystem, Heron is more into Twitter based ecosystem stack.

Any thoughts?

Answer 1

Heron, Stream processing engine developed by twitter and donated to Apache on 26th FEB 2018. As per Twitter, the throughput is 10–14x higher than that of Storm in all experiments, Similarly latency is 5-15x lower than Storm’s latency.

Other then throughput and latency it provides

• Easy debugging(Every task runs in process-level isolation).
• Handling spikes and congestion(using backpressure mechanism).
• Fully backward compatible with Storm which means only pom file changes required.

https://blog.twitter.com/engineering/en_us/a/2015/flying-faster-with-twitter-heron.html https://apache.github.io/incubator-heron/

Answer 2

All of the points in the referenced article comparing Apache Flink and Apache Storm also apply to Twitter's Heron. Heron provides exactly the same type of semantics and functionality as Storm. Heron is really best understood simply as a re-implementation of Storm that better fits Twitter's operational requirements.