CODEWITHSUNDEEP
scalaapache-spark
lockwobr
lockwobr

Reputation: 1511

Apache Spark 1.3 dataframe SaveAsTable database other then default

I am trying to save a dataframe as table using saveAsTable and well it works but I want to save the table to not the default database, Does anyone know if there is a way to set the database to use? I tried with hiveContext.sql("use db_name") and this did not seem to do it. There is an saveAsTable that takes in some options. Is there a way that i can do it with the options?

Upvotes: 4

Views: 6038

Answers (2)

lockwobr
lockwobr

Reputation: 1511

It does not look like you can set the database name yet... if you read the HiveContext.scala code you see a lot comments like...

    // TODO: Database support...

So I am guessing that its not supported yet.

Update:

In spark 1.5.1 this works, which did not work in early versions. In early version you had to use a using statement like in deformitysnot answer.

 df.write.format("parquet").mode(SaveMode.Append).saveAsTable("databaseName.tablename")

Upvotes: 5

amarouni
amarouni

Reputation: 805

This was fixed in Spark 1.5 and you can do it using :

hiveContext.sql("USE sparkTables");
dataFrame.saveAsTable("tab3", "orc", SaveMode.Overwrite);

By the way in Spark 1.5 you can read Spark saved dataframes from Hive command line (beeline, ...), something that was impossible in earlier versions.

Upvotes: 3

Related Questions