CODEWITHSUNDEEP
apache-sparkapache-spark-standalone
MegaOwIer
MegaOwIer

Reputation: 98

How to get result from Spark after submitting a job via REST API?

When I submit a Spark job through API /v1/submissions/create on port 6066 and check the status of it by /v1/submissions/status/{driver-id}, I only get something like this

{
  "action" : "SubmissionStatusResponse",
  "driverState" : "FINISHED",
  "serverSparkVersion" : "3.1.2",
  "submissionId" : "driver-20211226153006-0000",
  "success" : true,
  "workerHostPort" : "xx.xx.xx.xx:xx",
  "workerId" : "worker-20211226152727-xx.xx.xx.xx-xx"
}

Is there any way to get more information about this job, such as execution time, memory used or the output of this job, etc?

Upvotes: 3

Views: 157

Answers (0)

Related Questions