CODEWITHSUNDEEP
pluginshiveapache-drillquery-engine
Nightwing
Nightwing

Reputation: 37

error in accessing tables from hive using apache drill

I am trying to read the data from my table abc which is in hive using Drill. For that i have created hive storage plugin with the configuration mentioned below

{
"type": "hive",
"enabled": true,
"configProps": {
"hive.metastore.uris": "thrift://<ip>:<port>",
"fs.default.name": "hdfs://<ip>:<port>/",
"hive.metastore.sasl.enabled": "false",
"hive.server2.enable.doAs": "true",
"hive.metastore.execute.setugi": "true"
 }
 }

with this i am able to see the databases in hive, but when i try to access any table in the particular database 

select * from hive.db.abc;

it throws the following error

org.apache.drill.common.exceptions.UserRemoteException: VALIDATION ERROR: From line 1, column 15 to line 1, column 18: Object 'abc' not found within 'hive.db' SQL Query null [Error Id: b6c56276-6255-4b5b-a600-746dbc2f3d67 on centos2.example.com:31010] (org.apache.calcite.runtime.CalciteContextException) From line 1, column 15 to line 1, column 18: Object 'abc' not found within 'hive.db' sun.reflect.NativeConstructorAccessorImpl.newInstance0():-2 sun.reflect.NativeConstructorAccessorImpl.newInstance():62 sun.reflect.DelegatingConstructorAccessorImpl.newInstance():45 java.lang.reflect.Constructor.newInstance():423 org.apache.calcite.runtime.Resources$ExInstWithCause.ex():463 org.apache.calcite.sql.SqlUtil.newContextException():800 org.apache.calcite.sql.SqlUtil.newContextException():788 org.apache.calcite.sql.validate.SqlValidatorImpl.newValidationError():4703 org.apache.calcite.sql.validate.IdentifierNamespace.resolveImpl():127 org.apache.calcite.sql.validate.IdentifierNamespace.validateImpl():177 org.apache.calcite.sql.validate.AbstractNamespace.validate():84 org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace():947 org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery():928 org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom():2972 org.apache.drill.exec.planner.sql.SqlConverter$DrillValidator.validateFrom():267 org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom():2957 org.apache.drill.exec.planner.sql.SqlConverter$DrillValidator.validateFrom():267 org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect():3216 org.apache.calcite.sql.validate.SelectNamespace.validateImpl():60 org.apache.calcite.sql.validate.AbstractNamespace.validate():84 org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace():947 org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery():928 org.apache.calcite.sql.SqlSelect.validate():226 org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression():903 org.apache.calcite.sql.validate.SqlValidatorImpl.validate():613 org.apache.drill.exec.planner.sql.SqlConverter.validate():190 org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateNode():630 org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateAndConvert():202 org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():174 org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan():146 org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():84 org.apache.drill.exec.work.foreman.Foreman.runSQL():567 org.apache.drill.exec.work.foreman.Foreman.run():264 java.util.concurrent.ThreadPoolExecutor.runWorker():1149 java.util.concurrent.ThreadPoolExecutor$Worker.run():624 java.lang.Thread.run():748 Caused By (org.apache.calcite.sql.validate.SqlValidatorException) Object 'abc' not found within 'hive.db' sun.reflect.NativeConstructorAccessorImpl.newInstance0():-2 sun.reflect.NativeConstructorAccessorImpl.newInstance():62 sun.reflect.DelegatingConstructorAccessorImpl.newInstance():45 java.lang.reflect.Constructor.newInstance():423 org.apache.calcite.runtime.Resources$ExInstWithCause.ex():463 org.apache.calcite.runtime.Resources$ExInst.ex():572 org.apache.calcite.sql.SqlUtil.newContextException():800 org.apache.calcite.sql.SqlUtil.newContextException():788 org.apache.calcite.sql.validate.SqlValidatorImpl.newValidationError():4703 org.apache.calcite.sql.validate.IdentifierNamespace.resolveImpl():127 org.apache.calcite.sql.validate.IdentifierNamespace.validateImpl():177 org.apache.calcite.sql.validate.AbstractNamespace.validate():84 org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace():947 org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery():928 org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom():2972 org.apache.drill.exec.planner.sql.SqlConverter$DrillValidator.validateFrom():267 org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom():2957 org.apache.drill.exec.planner.sql.SqlConverter$DrillValidator.validateFrom():267 org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect():3216 org.apache.calcite.sql.validate.SelectNamespace.validateImpl():60 org.apache.calcite.sql.validate.AbstractNamespace.validate():84 org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace():947 org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery():928 org.apache.calcite.sql.SqlSelect.validate():226 org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression():903 org.apache.calcite.sql.validate.SqlValidatorImpl.validate():613 org.apache.drill.exec.planner.sql.SqlConverter.validate():190 org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateNode():630 org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateAndConvert():202 org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():174 org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan():146 org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():84 org.apache.drill.exec.work.foreman.Foreman.runSQL():567 org.apache.drill.exec.work.foreman.Foreman.run():264 java.util.concurrent.ThreadPoolExecutor.runWorker():1149 java.util.concurrent.ThreadPoolExecutor$Worker.run():624 java.lang.Thread.run():748

Upvotes: 0

Views: 799

Answers (1)

Vitalii Diravka
Vitalii Diravka

Reputation: 855

You should upgrade to the newer Hive version. For Drill 1.13 it is Hive 2.3.2 version? Starting from Drill-1.13 Drill leverages 2.3.2 version of Hive client [1].

Supporting of Hive 3.0 version is upcoming [2].

Also please follow the following guide with necessary Hive plugin configurations for your environment [3]. You could omit "hive.metastore.sasl.enabled", "hive.server2.enable.doAs" and "hive.metastore.execute.setugi" properties, since you have specified the default values [4]. Regarding "hive.metastore.uris" and "fs.default.name" you should specify the same values for them as in your hive-site.xml.

[1] https://drill.apache.org/docs/hive-storage-plugin
[2] https://issues.apache.org/jira/browse/DRILL-6604
[3] https://drill.apache.org/docs/hive-storage-plugin/#hive-remote-metastore-configuration
[4] https://github.com/apache/hive/blob/rel/release-2.3.2/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java#L824

Upvotes: 0

Related Questions