CODEWITHSUNDEEP
sqldatabricks
mytabi
mytabi

Reputation: 779

Databricks notebook hang forever

Actually I am doing some dataframe work for ETL, the dataframe is read from Azure datawarehouse . and seems somehow to notebook hang forever , but I don't know where it is and why it hang so long !!!

enter image description here

Any one has idea and the experience ?

Upvotes: 1

Views: 1920

Answers (1)

CHEEKATLAPRADEEP
CHEEKATLAPRADEEP

Reputation: 12768

There are various rare scenarios and corner cases that can cause a streaming or batch job to hang. It is also possible for a job to hang because the Databricks internal metastore has become corrupted.

Often restarting the cluster or creating a new one resolves the problem. In other cases, run the following script to unhang the job and collect notebook information, which can be provided to Databricks Support.

For more details, refer "How to Resolve Job Hands and Collect Diagnostic Information".

Hope this helps.

Upvotes: 2

Related Questions