CODEWITHSUNDEEP
azureazure-databricks
Harry Leboeuf
Harry Leboeuf

Reputation: 745

Troubles with dbutils.notebook.run in Azure Databricks

I'm getting a error when trying to do the following. I don't have a idea what it could be...

jobStartTime = datetime.strftime(datetime.now(), "%Y-%m-%dT%H:%M:%S%Z")
dbutils.notebook.run("/yptoCommon/LogPowerBI", 60, { "jobStartTime": jobStartTime, "NoteBook": "Oeps", "AllFiles": "10", "ProcessedFiles": "5", "NewRecords": "1" })

The error I'm getting is.

com.databricks.WorkflowException: com.databricks.NotebookExecutionException: FAILED --------------------------------------------------------------------------- Py4JJavaError Traceback (most recent call last) <command-4345148969112219> in <module> 1 jobStartTime = datetime.strftime(datetime.now(), "%Y-%m-%dT%H:%M:%S%Z") ----> 2dbutils.notebook.run("/yptoCommon/LogPowerBI", 60, { "jobStartTime": jobStartTime, "NoteBook": "Oeps", "AllFiles": "10", "ProcessedFiles": "5", "NewRecords": "1" })
/local_disk0/tmp/1583836135359-0/dbutils.py in run(self, path, timeout_seconds, arguments, NotebookHandlerdatabricks_internal_cluster_spec) 134 arguments, 135 databricks_internal_cluster_spec, --> 136 self.shell.currentJobGroup) 137 138 def repr_(self):

/databricks/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py in call(self, *args) 1255 answer = self.gateway_client.send_command(command) 1256 return_value = get_return_value( -> 1257 answer, self.gateway_client, self.target_id, self.name) 1258 1259 for temp_arg in temp_args:

/databricks/spark/python/pyspark/sql/utils.py in deco(a, *kw)

Any help is welcome.

Upvotes: 0

Views: 4409

Answers (1)

CHEEKATLAPRADEEP
CHEEKATLAPRADEEP

Reputation: 12788

As per my repro, here are my observation on the above issue.

You will receive this error message, when you have any errors in the execution of any cells in the notebook. You need to fix the errors in the cell and use "dbutils.notebook.run" to run the notebook.

Example1: Notebook which has an error in the execution of a cell

Here I have a notebook named "notebook2", which has an error message while executing a cell as shown below:

enter image description here

When I execute the notebook using "dbutils.notebook.run", I will see the same error message as yours because the notebook has error in executing a cell.

enter image description here

Example2: Successfully able to execute notebook without any issue

Now going back to the "notebook2", I will remove the cell which contains an error message.

enter image description here

When I execute the code using "dbutils.notebook.run", I will able to successfully able to run the command without any issue.

enter image description here

Upvotes: 1

Related Questions