CODEWITHSUNDEEP
pandaspysparkdatabricksazure-databricks
sy-huss
sy-huss

Reputation: 193

How do I create a databricks table from a pandas dataframe?

I have a pandas dataframe that I've created. This prints out fine, however I need to manipulate this in SQL.

I've run the following:

spark_df = spark.createDataFrame(df)

spark_df.write.mode("overwrite").saveAsTable("temp.testa")

pd_df = spark.sql('select * from temp.testa').toPandas()

But get an error:

AnalysisException: Database 'temp' not found;

Obviously I have not created a database, but not sure how to do this.

Can anyone advise how I might go about achieving what I need?

Upvotes: 1

Views: 2995

Answers (1)

CHEEKATLAPRADEEP
CHEEKATLAPRADEEP

Reputation: 12788

The error message clearly says "AnalysisException: Database 'temp' not found;" database temp is not found. Once the database is created you can run the query without any issue.

To create a database, you can use the below command:

To create a database in SQL:

CREATE DATABASE <database-name>

enter image description here

Reference: Azure Databricks - SQL

Upvotes: 2

Related Questions