CODEWITHSUNDEEP
bashdatabricksazure-databricks
Aneesh
Aneesh

Reputation: 166

How to install a jar in databricks using ADF

We are able to install the jar file using the UI method to a particular cluster. But our requirement to install it on all the ondemand clusters in the workspace. We are using the below shell script to download the jar file to DBFS. Not sure how we can refer/install this jar in all cluster using a global init script

curl https://repo1.maven.org/maven2/com/databricks/spark-xml_2.12/0.12.0/spark-xml_2.12-0.12.0.jar >/dbfs/FileStore/jars/maven/com/databricks/spark_xml_2_12_0_12_0.jar

Any help would be really appreciated!!

Upvotes: 4

Views: 1618

Answers (2)

Alex Ott
Alex Ott

Reputation: 87244

In the global init script you can just download this file into /databricks/jars/ directory - then it will be picked up by cluster

Upvotes: 0

P.K
P.K

Reputation: 46

There is an alternate solution for adding jar library to the job cluster which is called from Azure data factory while running our job.

In ADF, while calling the notebook we have the option to include the jar directory in DBFS or we can able to give the Maven coordinates.

ADF SETTINGS

Upvotes: 3

Related Questions