CODEWITHSUNDEEP
azurecurldatabricksspark-notebook
veilupearl
veilupearl

Reputation: 929

How to import notebook from local machine to Azure Databricks portal?

How to import notebook from local in Azure Databricks?

I have sample notebook in DBC format on my local machine and I need to import via Notebook Rest API.

curl -n -H "Content-Type: application/json" -X POST -d @- https://YOUR_DOMAIN/api/2.0/workspace/import <<JSON
{
  "path": "/Users/[email protected]/new-notebook",
  "format": "SOURCE",
  "language": "SCALA",
  "content": "Ly8gRGF0YWJyaWNrcyBub3RlYm9vayBzb3VyY2UKcHJpbnQoImhlbGxvLCB3b3JsZCIpCgovLyBDT01NQU5EIC0tLS0tLS0tLS0KCg==",
  "overwrite": "false"
}
JSON

Refer this doc

They are given as destination file path but not mention about source file path instead they given as content. But How could I add the source file to import notebook?

Upvotes: 1

Views: 1643

Answers (2)

kdazzle
kdazzle

Reputation: 4300

The reason that the source file path is ignored is because you're supposed to convert that file to base64 and put that string in the content. So then the path becomes irrelevant.

If you don't want to do that and don't mind using curl, the docs also say that you can also manage that like so:

curl -n -F path=/Users/[email protected]/project/ScalaExampleNotebook -F language=SCALA \
  -F [email protected] \
  https://<databricks-instance>/api/2.0/workspace/import

Otherwise, if you happen to be looking for how to import a directory...I spent a couple of hours looking myself. It uses the databricks-cli library, which is in Python.

$ pip install databricks-cli and then

from databricks_cli.workspace.api import WorkspaceApi
from databricks_cli.sdk.api_client import ApiClient


client = ApiClient(
    host='https://your.databricks-url.net',
    token=api_key
)
workspace_api = WorkspaceApi(client)
workspace_api.import_workspace_dir(
    source_path="/your/dir/here/MyProject",
    target_path="/Users/[email protected]/MyProject",
    overwrite=True,
    exclude_hidden_files=True
)

Upvotes: 0

Silvio
Silvio

Reputation: 4207

If you have a DBC file then the format needs to be DBC and language is ignored.

Also, the content property needs to be the DBC file bytes Base64 encoded, per the docs:

The content parameter contains base64 encoded notebook content

If using bash you could simply do base64 notebook.dbc

Upvotes: 4

Related Questions