CODEWITHSUNDEEP
pythonapache-sparkpysparkanaconda
Mohammad Rasheed
Mohammad Rasheed

Reputation: 31

Running pyspark in (Anaconda - Spyder) in windows OS

Dears, I am using windows 10 and I am familiar with testing my python code in Spyder. however, when I am trying to write ïmport pyspark" command, Spyder showing "No module named 'pyspark'" Pyspark is installed in my PC and also I can do import pyspark in command prompt without any error. I found many blogs explaining how to do this in Ubuntu but I did not find how to solve it in windows.

Upvotes: 3

Views: 8074

Answers (2)

cuarota AND
cuarota AND

Reputation: 21

Well for using packages in Spyder, you have to install them through Anaconda. You can open "anaconda prompt" and the write down the blew code:

conda install pyshark

That will give you the package available in SPYDER.

Upvotes: 2

user10414767
user10414767

Reputation: 1

Hi I have installed Pyspark in windows 10 few weeks back. Let me tell you how I did it. I followed "https://changhsinlee.com/install-pyspark-windows-jupyter/".

So after following each step precisely you can able to run pyspark using either command promp or saving a python file and running.

When you run via notebook(download Anaconda). start anacoda shell and type pyspark. now you don't need to do "ïmport pyspark". run your program without this and it will be alright. you can also do spark-submit but for that I figured out that you need to remove the PYSPARK_DRIVER_PATH and OPTS PATH in environment variable.

Upvotes: -1

Related Questions