CODEWITHSUNDEEP
azure-data-factoryazure-databricksdatabricks-workflows
Anil Kumar
Anil Kumar

Reputation: 565

How to install job dependent libraries and .whl package on Databricks Job cluster

We have the .whl package(customised module) that needs to be installed on the Databricks job cluster the moment when the new job cluster spins-up for ADF jobs. Let me know how we can push the .whl package to the newly spined-up cluster.

Any pointers would help.

Upvotes: 2

Views: 4405

Answers (1)

Alex Ott
Alex Ott

Reputation: 87174

When creating pipeline in Azure Data Factory, and adding Databricks activity, click onto "Settings", expand item "Append libraries", and click "New". Then select wheel as library type, and specify path to the library on DBFS (it should be uploaded there).

enter image description here

Upvotes: 3

Related Questions