CODEWITHSUNDEEP
pythonpysparkpycharm
Alonso
Alonso

Reputation: 141

Unresolved reference 'pyspark' on pycharm

I am a begginer programming in python and I just wanted to import some features from pyspark package. The problem is that, when I write this:

from pyspark.ml.feature import HashingTF, IDF, Tokenizer

it is said "Unresolved reference pyspark". Why is that?

Here are the steps I followed to create the proyect:

  1. New project using the interpreter python 3.6
  2. New python file called "main.py"
  3. Once the error pop-up I choose "Install pyspark package" but it does not solve it.

Can anyone help me out please?

enter image description here

Upvotes: 0

Views: 3661

Answers (1)

Kartik Nagappa
Kartik Nagappa

Reputation: 191

Do you have pyspark in your Python library?

From the screenshot, it looks like you've got the zip file (pyspark...tar.gz), but you need to unzip it prior to use.

The screenshot shows your Python library root as python3.6. Try extracting pyspark...tar.gz to python3.6/site-packages and see if that fixes your problem.

In addition, here's a cool tutorial using PySpark+Jupyter: https://blog.sicara.com/get-started-pyspark-jupyter-guide-tutorial-ae2fe84f594f

Upvotes: 1

Related Questions