CODEWITHSUNDEEP
pythonapache-sparkpysparkapache-toree
Abhinav Singh
Abhinav Singh

Reputation: 27

Fail to create Spark session

This might be a naive question.

When I am creating a spark context using pyspark from below code, after running the status of application starts appearing on spark dashboard under "Running application"

conf = pyspark.SparkConf().setAppName('blah').setMaster('spark://<some-ip>:8000')
sc = pyspark.SparkContext(conf=conf)

But When I am creating a spark session from code snippet below there is no status of the application shown on spark dashboard.

spark = SparkSession.builder
  .appName('blah blah')
  .master('spark://<some -ip>:8000')
  .getOrCreate()

I am not sure why this is working like this, as I am completely new to Spark so kindly help.

Upvotes: 1

Views: 250

Answers (0)

Related Questions