oikonomiyaki
oikonomiyaki

Reputation: 7951

How to convert a table into a Spark Dataframe

In Spark SQL, a dataframe can be queried as a table using this:

sqlContext.registerDataFrameAsTable(df, "mytable")

Assuming what I have is mytable, how can I get or access this as a DataFrame?

Upvotes: 20

Views: 51195

Answers (2)

Igor Tavares
Igor Tavares

Reputation: 969

The cleanest way:

df = sqlContext.table("mytable")

Documentation

Upvotes: 37

Alberto Bonsanto
Alberto Bonsanto

Reputation: 18022

Well you can query it and save the result into a variable. Check that SQLContext's method sql returns a DataFrame.

df = sqlContext.sql("SELECT * FROM mytable")

Upvotes: 19

Related Questions