CODEWITHSUNDEEP
virtual-machineapache-sparkvirtualizationspark-streaming
Xingjun Wang
Xingjun Wang

Reputation: 423

Run Spark streaming in virtual machines

Is there any obvious performance degradation or drawback when deploy Spark streaming cluster in virtualized environment like Xen or KVM? What's the main reason?

Upvotes: 0

Views: 835

Answers (1)

Daniel Darabos
Daniel Darabos

Reputation: 27455

The usual caveats about virtualization apply, but there is nothing specific to Spark or Spark Streaming.

I don't know of an article that would directly address this question. But the Spark petasort benchmark was run on EC2 and the article pays close attention to performance: https://databricks.com/blog/2014/10/10/spark-petabyte-sort.html

Upvotes: 1

Related Questions