CODEWITHSUNDEEP
apache-sparkpyspark
user110235
user110235

Reputation: 29

Pyspark only runs using sudo

I am trying to get pyspark to run in terminal on Mac OS X; when I run it without sudo, I get the following message:

enter image description here

However, if i type 'sudo pyspark' this happens

enter image description here.

Is there a way I can type 'pyspark' into terminal without it asking me for my password i.e. having to write sudo first? Thanks

Upvotes: 2

Views: 1204

Answers (1)

BigPeter
BigPeter

Reputation: 61

I also encountered this problem.The machine is mac and I install pyspark by brew. In my situation, the problem is caused by the ~/.bash_profile. The problematic spark export statement is as follows: 

export SPARK_HOME=/usr/local/Cellar/apache-spark/2.4.0  
export PATH=$PATH:$SPARK_HOME

In this situation, pyspark only runs using sudo.
After I modify spark export statement as follows, 

export SPARK_HOME=/usr/local/Cellar/apache-spark/2.4.0/libexec
export PATH=$PATH:$SPARK_HOME

I can run pyspark by command 'pyspark'.

Upvotes: 5

Related Questions