CODEWITHSUNDEEP
hadoopjoinhivehiveql
user3829376
user3829376

Reputation: 75

FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask during join operation in hive

I am trying to run the join query in hive on the following two tables-

select b.location from user_activity_rule a inner join user_info_rule b where a.uid=b.uid and a.cancellation=true;

Query ID = username_20180530154141_0a187506-7aca-442a-8310-582d335ad78d
Total jobs = 1
OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support was removed in 8.0
Execution log at: /tmp/username/username_20180530154141_0a187506-7aca-442a-8310-582d335ad78d.log
2018-05-30 03:41:51     Starting to launch local task to process map join;      maximum memory = 2058354688
Execution failed with exit status: 2
Obtaining error information

Task failed!
Task ID:
  Stage-4

Logs:

/tmp/username/hive.log
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask

What does this error mean and how to resolve this?

Upvotes: 0

Views: 8868

Answers (3)

Rearchy
Rearchy

Reputation: 76

First of all, make sure the HADOOP_USER that you used to run SQL can run MapReduce.

Then, use SQL like following:

set hive.auto.convert.join = false;
select b.location 
from user_activity_rule a 
inner join user_info_rule b 
where a.uid=b.uid and a.cancellation=true;

Upvotes: 0

Batman
Batman

Reputation: 50

This happens when the job you are trying to run runs out of memory. one way of overcoming this is to use this command:

set hive.auto.convert.join = false;

This will help in join optimization.

Sometimes when the number of concurrent users using it is high(at some peak time), this happens. Alternatively, you can fire this query when not many users are using it. Apparently, there would be much free memory so that your job can consume required. This alternative can be adopted when nodes are less in Dev environment and you are sure that there will be no memory issues in production.

Upvotes: 2

Rohit Nimmala
Rohit Nimmala

Reputation: 1539

Instead of where can you use the below code and try

SELECT b.location FROM user_activity_rule a JOIN user_info_rule b ON(a.uid=b.uid) WHERE a.cancellation="true";

Upvotes: 0

Related Questions