CODEWITHSUNDEEP
apache-sparkpysparkapache-spark-sqldatabricksazure-databricks
Tiago
Tiago

Reputation: 65

How to get a query from a different notebook in Databricks

My first post here ever :) So I am having a problem :/ Working with Databricks (db) I know how to execute a different notebook in db using the %run But I have two questions:

  1. Notebook 1 has a single query and notebook 2 has a function that needs that query, how do I input that query over there? I tried only the %run and it runs the query I tried defining it to a variable, tried to pass it as a parameter to the function and nothing worked :( (but putting the query in the function it does work!!)

  2. Is it possible to have a notebook full of queries, in different cells, and call a specific cell? I tried to look it up but I couldn’t find it anywhere :/

Thank you so much!! Have a nice year :)

I tried only the %run and it runs the query I tried defining it to a variable, tried to pass it as a parameter to the function and nothing worked :( (but putting the query in the function it does work!!)

Upvotes: 1

Views: 1983

Answers (2)

Rakesh Govindula
Rakesh Govindula

Reputation: 11529

  1. Notebook 1 has a single query and notebook 2 has a function that needs that query, how to I input that query over there? I tried only the %run and it runs the query I tried defining it to a variable, tried to pass it as a parameter to the function and nothing worked :( (but putting the query in the function it does work!!)

Calling the Notebook from another Notebook using dbuitls.notebook.run() is the solution for it.

Pass your query to another Notebook as parameter like below. Then use myquery=dbutils.widgets.get("query") to get that parameter value and you can use that in Called notebook function like a sample demonstration below.

Nb1- Calling Notebook:

enter image description here

Nb2- Called Notebook with function executed

enter image description here

Here use eval() to execute your query in the function.

  1. Is it possible to have a notebook full of queries, in different cells, and call a specific cell? I tried to look it up but I couldn’t find it anywhere :/

AFAIK, we cannot call particular cell from a Notebook to another Notebook.

Using functions for each cell and call it using %run Notebook_name as suggested by @GregGalloway in comments can be a workaround for it.

Example:

enter image description here

enter image description here

Upvotes: 2

smfjaw
smfjaw

Reputation: 146

Q1) I would follow the documentation in here using the command, where notebook-name is the notebook you're calling

results = dbutils.notebook.run("notebook-name", 60, {"argument": "data", "argument2": "data2", ...})

https://docs.databricks.com/notebooks/notebook-workflows.html

Q2) I don't think this is possible, I would suggest splitting out the code into another notebook with only the code you want called

Upvotes: 2

Related Questions