CODEWITHSUNDEEP
pythonpyspark
Anurag Pandey
Anurag Pandey

Reputation: 463

how to integrate pyspark on jupyter notebook

i have followed instructions to integrate pyspark with jupyter but after i was done i was only able to run pyspark on the command prompt .

Basically when i am using ipython in my command it is working but when i change to jupyter it says 

'"Jupyter "' is not recognized as an internal or external command,
operable program or batch file.

please SO people i am really exhausted now , please help

note : i followed this tutorial to do my integration .

till now i have followed this command : 

  set PYSPARK_DRIVER_PYTHON=ipython 
  set PYSPARK_DRIVER_PYTHON_OPTS = notebook 
  pyspark

it opens pyspark in cmd itself

Upvotes: 1

Views: 1174

Answers (1)

Jean-François Fabre
Jean-François Fabre

Reputation: 140148

Windows set command does not accept spaces before the = sign.

fix this like that:

set PYSPARK_DRIVER_PYTHON=ipython 
set PYSPARK_DRIVER_PYTHON_OPTS=notebook 
pyspark

The system wants to run the "Jupyter %PYSPARK_DRIVER_PYTHON_OPTS%" executable but instead tries to run '"Jupyter "'

hence the error you're getting.

Upvotes: 1

Related Questions