CODEWITHSUNDEEP
azure-data-factoryazure-databricks
shama khan
shama khan

Reputation: 33

Fail azure data factory pipeline if notebook execution skipped

enter image description hereI have build an ADF pipline and i am executing multiple databricks notebooks in the pipeline. When one of the notebook fails the remianing command in the notebook skipped but the pipeline did not fail. I want to make sure that the pipeline execution stops if the notebook fails and next command skipped

I have tried trigering an error so that the pipeline fails

NA

I want to make sure that the pipeline execution stops if the notebook fails and next command skipped

Upvotes: 2

Views: 1400

Answers (2)

Manish Jain
Manish Jain

Reputation: 237

Use assert; here a the Scala code snippet:

try {
  ... ... ...
}
catch {
  case e: Exception => {
    println("exception caught while ... ... : " + e)
    assert(false)
  }
}

This will fail your Notebook Activity in Azure Data Factory.

Upvotes: 0

Hauke Mallow
Hauke Mallow

Reputation: 3182

If you connect the Azure Databricks acitivities as follows:

Pipeline Design

Notebook2-Activity will only be executed if the first Notebook-Activity is successfull.

Here the first activity fails and the next is not executed (Monitoring view of Azure Data Factory):

enter image description here

Upvotes: 1

Related Questions