CODEWITHSUNDEEP
scalaapache-sparkconfigurationapache-spark-sql
dwarakesh tp
dwarakesh tp

Reputation: 137

How to get the details of user who submitted the spark job from within the job context?

I'm working on logging the details of the spark job executions. I would like to get the username of the user who submitted the spark job, from within the context of the spark job.

Any help is appreciated. Thanks..

Upvotes: 3

Views: 3502

Answers (1)

notNull
notNull

Reputation: 31480

Try the below command to get the spark user who submitted the job.

Spark >= 2.0

spark.sparkContext.sparkUser

(or)

spark < 2.0

sc.sparkUser

Upvotes: 2

Related Questions