CODEWITHSUNDEEP
jdbcapache-sparkapache-spark-sql
Laodao
Laodao

Reputation: 1689

Spark JDBC cannot find temporary tables

We created a temporary table (in memory) through spark.

When we sftp to the server and use beeline, we can query on the this temporary table like "select * from Table1" without issue.

However, when we use GUI tool with corresponding driver on local machine (the connection string is "jdbc:spark://servername:port/default" ), we have trouble. We can see the temporary table Table1 by using "show tables;" in the GUI tool. However, when we try to use "select * from Table1" in the tool, It shows an error "[Simba]JSQLEngine The table "Table1" could not be found., SQL state: HY000, Query: select * from Table1. [SQL State=HY000, DB Errorcode=500051]". Note that we are using the trial version of the Simba JDBC driver for testing.

Also, I tried hive-jdbc driver from cloudra using connection string "jdbc:hive2://servername:port/default". It is the same issue. Please help. Thanks a lot.

Upvotes: 0

Views: 872

Answers (1)

Laodao
Laodao

Reputation: 1689

It turns out that some of the drivers requires a "limit" clause after the select. Once I add that, it retrieved data.

Upvotes: 0

Related Questions