CODEWITHSUNDEEP
pythonnode.jsresthadoopapache-spark
Katchy
Katchy

Reputation: 85

REST API interacting with Spark

I am trying to connect my Spark context with REST API. So my flow is this, i send request from my UI- nodejs to REST API which need to interact with the spark context ( Spark job works above hive tables ) which contains the needed response and send it back to UI for display. Is there any way to achieve this ? UI <->REST API<-> Spark (HDFS)

Upvotes: 2

Views: 1849

Answers (1)

yeiniel
yeiniel

Reputation: 2456

You can use Apache Livy. It is specifically designed for this use case. We have used it in the past to control batch and regular jobs on a Spark cluster.

Upvotes: 4

Related Questions