CODEWITHSUNDEEP
apache-sparkapache-spark-sqldatabrickscatalyst-optimizer
mauna
mauna

Reputation: 1118

What happened to the ability to visualize query plans in a Databricks notebook?

There is an old (year 2014) talk on Youtube where the speaker visualized a query plan right inside a Databricks notebook. Here is the screenshot:

enter image description here

I am using databricks runtime 5.5 LTS ML and whenever I try to call viz on a query plan, I get this kind of error:

error: value viz is not a member of org.apache.spark.sql.catalyst.plans.logical.LogicalPlan

Has this feature been deprecated in Spark 2.0+ or do I need to install/import other libraries to get this feature?

Upvotes: 0

Views: 579

Answers (1)

Goldengenova
Goldengenova

Reputation: 54

I don't think there is a visualization of it anymore, but they do explain SQL queries, which I think is what you are looking for.

spark.sql("your SQL query").explain(true)

or 

yourDataframe.explain(true)

Upvotes: 0

Related Questions