CODEWITHSUNDEEP
hadoophivehortonworks-data-platformdruid
BastienB
BastienB

Reputation: 320

Error when interacting from Hive to Druid

I'm trying to create a Druid datasource from Hive and I'm using table Hive for it.

First, I created a database Hive : database_hive then, I created a table in this database.

CREATE TABLE database_hive.hive_table (
    timemachine int,
    userId String,
    lang String,
    location String,
    name String,
    network String,
    posted String,
    sentiment String,
    text String,
);

In a second time, i'm trying to use hive_table for create a new datasource on Druid.

SET hive.druid.broker.address.default = 10.1.123.30:8082; --fake ip for example
SET hive.druid.metadata.username = druid;
SET hive.druid.metadata.password = druidpassword;
SET hive.druid.metadata.db.type = derby;
SET hive.druid.metadata.uri = jdbc:mysql://10.1.123.30:3306/druid?createDatabaseIfNoExist=true;

CREATE TABLE druid_table
STORED BY 'org.apache.hadoop.hive.druid.DruidStorageHandler'
TBLPROPERTIES (
    "druid.segment.granularity" = "MONTH",
    "druid.query.granularity" = "DAY")
    AS
    SELECT
    cast(timemachine as timestamp) as `__time`,   
    cast(userId as string) userId,
    cast(lang as string) lang,
    cast(location as string) location,
    cast(name as string) name,
    cast(network as string) network,
    cast(posted as string) posted,
    cast(sentiment as string) sentiment,
    cast(text as string) text
    FROM hive_table
;

This query return me an error :

Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org.skife.jdbi.v2.exceptions.UnableToObtainConnectionException: java.sql.SQLException: Cannot create JDBC driver of class 'org.apache.derby.jdbc.ClientDriver' for connect URL 'jdbc:mysql://10.1.123.30:3306/druid?createDatabaseIfNoExist=true' (state=08S01,code=1)

In my Hive repository, i've mysql-connector-java.jar so I don't understand what's the problem. I've tried some proposals that I read on other topics but I didn't find solution. Someone has suggestions?

Thanks for help !

Upvotes: 0

Views: 971

Answers (1)

Artem Vovsia
Artem Vovsia

Reputation: 1570

The issue is in your Druid metadata DB configuration. The Hive needs access to the relational database that Druid uses to store all the metadata. It can be set to derby on Druid side, but Derby is like SQLite, so it can't be accessed by Hive. Thus, Hive doesn't allow derby as a valid argument to hive.druid.metadata.db.type property. The only allowed are mysql and postgresql.

So to fix this issue you need:

  • Make sure that your Druid cluster uses either MySQL or PSQL for metadata storage
  • Set hive.druid.metadata.db.type to the correct db type
  • Set hive.druid.metadata.uri to correct DB url

Upvotes: 1

Related Questions