How do I submit a Spark jar to a EMR cluster?

Question

I have used the online AWS console to launch my cluster along with Apache Spark. I have a fat jar based on my Spark app and I have uploaded it to a S3 Bucket. When I try to send it as a Step with a Custom Jar, the process fails. Any pointers would be greatly appreciated.

Answer 1

Use EMR bootstrap to install Spark, and submit the job as described in the documentation: https://github.com/awslabs/emr-bootstrap-actions/blob/master/spark/examples/spark-submit-via-step.md