CODEWITHSUNDEEP
scalaapache-spark
Corey Wu
Corey Wu

Reputation: 1159

Is there a way to submit a Spark application to a cluster without creating a jar?

I am building a service which takes in a string of Spark code to execute on a cluster. Is there any way for me to set the Spark context to the cluster and execute without building a jar and submitting it?

Upvotes: 0

Views: 172

Answers (1)

Holden
Holden

Reputation: 7452

Indeed, you can use the spark shell, or look at something like the IBM Spark Kernel, Zeppelin, etc. to have a long running Spark Context you can submit code to and have it run. As you are almost certainly already aware of, be very careful with accepting strings and executing them on the cluster (e.g. only from a trusted source).

Upvotes: 2

Related Questions