CODEWITHSUNDEEP
apache-sparkpysparkapache-spark-sql
User12345
User12345

Reputation: 5480

create table like temptable in spark 1.6

I have a data frame in pyspark called df.

I have register this df as temptable

df.registerTempTable('mytempTable')

Now I want to use create table like statement to create a table in hive.

sqlContext.sql("create table {}.{} like mytempTable".format(hivedb,table))

But I am getting error like below

pyspark.sql.utils.AnalysisException: u"cannot recognize input near 'like' 'mytempTable' '<EOF>' in select clause; line 1 pos 59"

I have found a JIRA for this issue and looks like this issue has been fixed in spark 2.0.

The problem is my spark version is 1.6.

Is there a work around for this issue in spark 1.6.

The JIRA link is here https://issues.apache.org/jira/browse/SPARK-5720

Upvotes: 0

Views: 1087

Answers (1)

Ankit Kumar Namdeo
Ankit Kumar Namdeo

Reputation: 1464

try this:

sqlContext.sql("create table {}.{} as select * from mytempTable ".format(hivedb,table))

or:

df.write.mode("overwrite").saveAsTable("{}.{}".format(hivedb,table))

Upvotes: 1

Related Questions