CODEWITHSUNDEEP
jsonjarhiveamazon-emrapache-zeppelin
user 923227
user 923227

Reputation: 2715

How to add a jar in zeppelin?

How to add a jar in Zeppelin for %hive interpreter?

I have tried

%z.dep('');
add jar <jar path>

Also zeppelin hive interpreter throws ClassNotFoundException

Adding to ./interpreter/hive/ throughs thrift exception while add jar says file not found.

How to do it? I am trying to add jsonserde.jar by Amazon to parse DynamoDB imports.

Upvotes: 9

Views: 24438

Answers (5)

Devendra Parhate
Devendra Parhate

Reputation: 155

  • Get the dependency declaration from Maven Respository website

  • Create Zeppelin dependency declaration in following format: <groupId>:<artifactoryId>:<version> = ml.dmlc:xgboost4j-spark:0.90

  • Stop/Restart Zeppelin spark and run the command

    %dep
z.load("<groupId>:<artifactoryId>:<version>")

Example:

%dep
z.load("ml.dmlc:xgboost4j-spark:0.90")

Upvotes: 0

Zouzias
Zouzias

Reputation: 2360

Example, tested in Zeppelin 0.8.1:

%dep
z.addRepo("Spark Packages Repo").url("http://dl.bintray.com/spark-packages/maven")
z.addRepo("OSS SNAPSHOTS").url("https://oss.sonatype.org/content/repositories/snapshots")
z.load("org.zouzias:spark-lucenerdd_2.11:0.3.7")

Upvotes: 2

Dat
Dat

Reputation: 5833

Updated on Zeppelin 0.8.x

You can now load external dependencies or jars using %dep or %spark.dep (if using Spark). Documentation is on zeppline website: Dynamic Dependency Loading

%dep z.load("/path/to/your/packages.jar")

or

%spark.dep
z.reset() // clean up previously added artifact and repository

// add maven repository
z.addRepo("RepoName").url("RepoURL")

// add maven snapshot repository
z.addRepo("RepoName").url("RepoURL").snapshot()

// add credentials for private maven repository
z.addRepo("RepoName").url("RepoURL").username("username").password("password")

Upvotes: 4

user 923227
user 923227

Reputation: 2715

I have the version 0.5.6 for me the following worked:

%hive
add jar /home/hadoop/jsonserde.jar

Upvotes: 0

Rockie Yang
Rockie Yang

Reputation: 4925

The recommended way is using Zeppelin's Dependency Management

It can add jar file or maven artifact.

the dependency jar will be downloaded to local-repo.

NOTE: If the jar file is compiled from source, when you compile again, it will NOT be synchronised automatically(download again). You need go to interpreter setting, click edit and OK will trigger another download to local-repo.

NOTE: If you use one scala version first, and compiled again with another version. It will report Exception in thread “main” java.lang.NoSuchMethodError: scala.reflect.api.JavaUniverse.runtimeMirror. remove already downloaded jar with rm -rf local-repo/*

NOTE: z.dep is deprecated.

Upvotes: 8

Related Questions