CODEWITHSUNDEEP
python-2.7pysparkapache-spark-sqlsnappydatasnappydb
techie95
techie95

Reputation: 515

from pyspark.sql.snappy import SnappyContext - ImportError: No module named snappy

even after reinstalling pyspark and snappydata whenever I try to import pyspark.sql.snappy import SnappyContext from the code below:

from pyspark.sql.snappy import SnappyContext
from pyspark.storagelevel import StorageLevel
SparkContext._ensure_initialized()

spark = SparkSession.builder.appName("test")  \
                        .master("local[*]") \
                        .config("spark.snappydata.connection", "localhost:1527") \
                        .getOrCreate() 

snappy = SnappySession(spark)
snappy.sql("SELECT col1, min(col2) from TABLE1")

I get error:

Traceback (most recent call last):
  File "testpy.py", line 4, in <module>
    from pyspark.sql.snappy import SnappyContext
ImportError: No module named snappy

Please help!

Upvotes: 2

Views: 761

Answers (1)

Rishitesh Mishra
Rishitesh Mishra

Reputation: 459

This was a known problem in last released version. This has been fixed in latest master. $SNAPPY_HOME/bin/pyspark refers to the python scripts inside $SNAPPY_HOME/pyspark folder. Unfortunately, some build changes stopped copying SnappyData python scripts in the folder. You can build the current master to work with pyspark.

Upvotes: 1

Related Questions