CODEWITHSUNDEEP
apache-sparkapache-spark-sqlimpalaapache-kudu
Joseratts
Joseratts

Reputation: 97

How to access to apache kudu table created from impala using apache spark

I downloaded the quickstart VM of apache kudu and I have followed the examples just like they appears in this page https://kudu.apache.org/docs/quickstart.html, in fact I created the table named "sfmta" but when I tried to to access to the kudu table using spark-shell with the following sentence:

val df = spark.sqlContext.read.options(Map("kudu.master" -> "quickstart.cloudera:7051","kudu.table" -> "sfmta")).kudu

I get this error

org.apache.kudu.client.NonRecoverableException: The table does not exist: table_name: "sfmta"

I have tried also specifying different tables names like "default:sfmta", "default::sfmta" with the same result. Could you tell me why is this happening? why I can not access to kudu table? is this for being created with impala?

Thanks in advance.

Upvotes: 2

Views: 3450

Answers (3)

paul_di
paul_di

Reputation: 103

May be Will's answer contains a mistype. Correct one (2 colons):

impala::database_name.table_name

For the default db:

impala::default.table_name

It was tested with CDH 5.1X and Spark 1.6.X. I Hope that will save your time.

Upvotes: 0

Will
Will

Reputation: 21

if you create a database (instead of "default"), the table should be,

impala:database_name.table_name

Upvotes: 2

Joseratts
Joseratts

Reputation: 97

Finally after asked to kudu mail user group I have found the answer to this question.

After check the tables names at http://quickstart.cloudera:8051/tables I could see the full name of the table I wanted to query and instead of use "sfmta" I should use "impala::default.sfmta" like the following sentence:

val df = spark.sqlContext.read.options(Map("kudu.master" -> "quickstart.cloudera:7051","kudu.table" -> "impala::default.sfmta")).kudu

So after replace the table name it worked.

Upvotes: 2

Related Questions