CODEWITHSUNDEEP
azure-databricksazure-data-factory
harshith
harshith

Reputation: 91

unit test cases for databricks notebook

I have 5 files which I'm fetching and using it to create a dataframe, I'm using these dataframes to create table and left join tables. I wanted to know what all unit testing should be done and how to write unit test case

Upvotes: 0

Views: 1901

Answers (1)

Himanshu Kumar Sinha
Himanshu Kumar Sinha

Reputation: 1806

Looking at the tags which you have added , to me it looks like you are calling a ADB notebook from ADF . Since you can use Python on ADB , you should be useing the pytest module . Please read more about the same here .

https://learn.microsoft.com/en-us/azure/databricks/dev-tools/ci-cd/ci-cd-azure-devops#unit-tests-in-azure-databricks-notebooks .

Unit tests in Azure Databricks notebooks

For library code developed outside an Azure Databricks notebook, the process is like traditional software development practices. You write a unit test using a testing framework, like the Python pytest module, and use JUnit-formatted XML files to store the test results.

Azure Databricks code is Apache Spark code intended to be executed on Azure Databricks clusters. To unit test this code, you can use the Databricks Connect SDK configured in Set up the pipeline.

Upvotes: 0

Related Questions