CODEWITHSUNDEEP
jdbchivehortonworks-data-platform
jintocvg
jintocvg

Reputation: 158

Hive jdbc connection is giving error if MR is involved

I am working on Hive-jdbc connection in HDP 2.1

Code is working fine for queries where mapreduce is not involved like "select * from tabblename". The same code is showing error when the query is modified with a 'where' clause or if we specify columnnames(which will run mapreduce in the the background).

I have verified the correctness of the query by executing it in HiveCLI. Also I have verified the read/write permissions for the table for the user through which I am running the java-jdbc code.

The error is as follows

    java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
    at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:275)
    at org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:355)
    at com.testing.poc.hivejava.HiveJDBCTest.main(HiveJDBCTest.java:25)

Upvotes: 4

Views: 5063

Answers (2)

朱书利
朱书利

Reputation: 11

Today I also got this exception when I submit a hive task from java. The following error:

org.apache.hive.jdbc.HiveDriverorg.apache.hive.jdbc.HiveDriverhive_driver:
org.apache.hive.jdbc.HiveDriverhive_url:jdbc:hive2://10.174.242.28:10000/defaultget 
connection sessucess获取hive连接成功！
java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask

I tried to use the sql execute in hive and it works well. Then I saw the log in /var/log/hive/hadoop-cmf-hive-HIVESERVER2-cloud000.log.out then I found the reason of this error. The following error:

Job Submission failed with exception 'org.apache.hadoop.security.AccessControlException(Permission denied: user=anonymous, access=WRITE, inode="/user":hdfs:supergroup:drwxr-xr-x

Solution

I used the following command :

sudo -u hdfs hadoop fs -chmod -R 777  /

This solved the error!

hive_driver:org.apache.hive.jdbc.HiveDriver
hive_url:jdbc:hive2://cloud000:10000/default
get connection sessucess
获取hive连接成功！
Heart beat
执行insert成功！

Upvotes: 1

Holman
Holman

Reputation: 26

If you use beeline to execute the same queries, do you see the same behaviour as you get while running your test program?

The beeline client also uses the open source JDBC driver and connects to Hive server, which is similar to what you do in your program. HiveCLI on the other hand has Hive embedded in it and does not connect to a remote Hive server by default. You can use HiveCLI to connect to a remote Hive Server 1 but I don't believe you can use it to connect to Hive Server2 (use beeline for Hive Server 2).

For this error, you can take a look at the hive.log and hiveserver2.log on the server side to get more insight into what might have caused the MapReduce error.

Hope this helps.

Cheers, Holman

Upvotes: 0

Related Questions