CODEWITHSUNDEEP
pysparkapache-zeppelindelta-lake
Jay Ng
Jay Ng

Reputation: 141

How to import Delta Lake module in Zeppelin notebook and pyspark?

I am trying to use Delta Lake in a Zeppelin notebook with pyspark and seems it cannot import the module successfully. e.g.

%pyspark
from delta.tables import *

It fails with the following error:

ModuleNotFoundError: No module named 'delta'

However, there is no problem to save/read the data frame using delta format. And the module can be loaded successfully if using scala spark %spark

Is there any way to use Delta Lake in Zeppelin and pyspark?

Upvotes: 4

Views: 3522

Answers (1)

Jay Ng
Jay Ng

Reputation: 141

Finally managed to load it on zeppelin pyspark. Have to explicitly include the jar file 

%pyspark
sc.addPyFile("**LOCATION_OF_DELTA_LAKE_JAR_FILE**")
from delta.tables import *

Upvotes: 4

Related Questions