CODEWITHSUNDEEP
javascriptpythonapache-sparkpysparkbigdata
shivanshuiitg
shivanshuiitg

Reputation: 29

Monitor Spark Jobs in real time

I want to monitor Spark Jobs in real time

For that I have fetch data from Spark Monitoring REST API and display in visualizations. One solution is to Poll the API at every 1 second. But I want to do it without polling.

Can we do something like Webhook.

Thanks in Advance :)

Upvotes: 1

Views: 781

Answers (1)

Mehrez
Mehrez

Reputation: 695

I don't know Webhook but i think you can use SparkListener to push spark scheduler events in real time to your backend application.

Upvotes: 1

Related Questions