Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask

Question

I am facing problem in executing CONDITIONAL QUERIES IN HiveQL.The basic select * from tablename statement works fine. The error is :

Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
java.net.ConnectException: Call to /0.0.0.0:8021 failed on connection exception: java.net.ConnectException: Connection refused
        at org.apache.hadoop.ipc.Client.wrapException(Client.java:1134)
        at org.apache.hadoop.ipc.Client.call(Client.java:1110)
        at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
        at org.apache.hadoop.mapred.$Proxy5.getProtocolVersion(Unknown Source)
        at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:398)
        at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:384)
        at org.apache.hadoop.mapred.JobClient.createRPCProxy(JobClient.java:501)
        at org.apache.hadoop.mapred.JobClient.init(JobClient.java:486)
        at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:469)
        at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:655)
        at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:123)
        at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:130)
        at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
        at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1063)
        at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:900)
        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:748)
        at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:209)
        at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:286)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:516)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
Caused by: java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
        at org.apache.hadoop.net.SocketIOWithTimeout.connect(
....
Job Submission failed with exception 'java.net.ConnectException(Call to /0.0.0.0:8021 failed on connection exception: java.net.ConnectException: Connection refused)'
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask

I have tried from both Eclipse IDE and Hive shell .The output is same. Please Help!

Answer 1

This worked for me:

SET hive.auto.convert.join=false;

As default Hive converts simple joins to map joins which try to load input tables to memory. This option will disable loading input tables to memory.

Answer 2

For me, it was due to the queue not being set.

set mapred.job.queue.name=xxxx;
set mapreduce.job.reduces=1;

Setting both parameters it worked for me.

Answer 3

I have a cloudera quickstart image mounted using VirtualBox. I had to do the following to fix the issue

sudo vi /etc/hive/conf/hive-site.xml and set the hive.auto.convert.join property to false (you need to root to edit and hence the sudo).

I did not have the hive.auto.convert.join.noconditionaltask property in the hive-site.xml.xml.

Answer 4

This should solve your issue as this property will set to true by default.

set hive.auto.convert.join.noconditionaltask=false

hive.auto.convert.join.noconditionaltask - Whether Hive enable the optimization about converting common join into mapjoin based on the input file size.

If this paramater is on, and the sum of size for n-1 of the tables/partitions for a n-way join is smaller than the specified size, the join is directly converted to a mapjoin (there is no conditional task).

Answer 5

I have noticed that with Hive2 you need to change the way you specify the number of reducers while using HiveQL. On Hive1 I have used:

SET mapred.reduce.tasks=1

However on Hive2 I have noticed I need to use:

SET mapreduce.job.reduces=1

I had the same error message and changing this resolved the issue for me.

Answer 6

In most cases, this error is caused by a permissions issue, where the Hive user running the command does not have access rights to /user/hive

You'll want to check the configuration of the Hadoop Hive cluster