CODEWITHSUNDEEP
hadoophiveoozie
Shinta Smith
Shinta Smith

Reputation: 500

Hive action failed with Main class [org.apache.oozie.action.hadoop.HiveMain], exit code [40000]

I am new in this so it's completely possible I miss something basic.

I am trying to run an Oozie workflow that gets kicked off from a Coordinator. The Coordinator waits until files show up in a directory. The workflow contains a Hive action, that runs this script:

CREATE external TABLE IF NOT EXISTS daily_dump (
        id bigint,
        creationdate timestamp,
        datelastupdated timestamp,
        data1 string,
        data2 string) LOCATION '/data/daily_dump';

FROM daily_dump d 
INSERT OVERWRITE TABLE mydata_orc 
PARTITION(id, datelastupdated) 
SELECT d.id, d.creationdate, d.datelastupdated, d.data1, d.data2;

DROP TABLE daily_dump;

If I run the script manually from hive CLI, it works fine.

The workflow got kicked off correctly when _SUCCESS file shows up. It appears the script is halfway executed as I can see from hive CLI that the table "daily_dump" got created. I can see data in it. I checked the hivemetastore.log and did not see any errors.

But the statement after that seemed to die in Oozie with this error:

2015-01-30 18:04:40,086  WARN HiveActionExecutor:542 - USER[me] GROUP[-]
TOKEN[] APP[guzzler] JOB[0000162-150114210350250-oozie-oozi-W]
ACTION[0000162-150114210350250-oozie-oozi-W@copy_to_mydata] Launcher
ERROR, reason: Main class [org.apache.oozie.action.hadoop.HiveMain], exit
code [40000]

What does error 40000 mean?

My hive.log shows the last command in the script (the DROP TABLE) and no ERROR logging after that:

2015-01-30 15:25:05,001 INFO  ql.Driver (Driver.java:execute(1197)) - Starting command:

DROP TABLE daily_dump
2015-01-30 15:25:05,001 INFO  hooks.ATSHook (ATSHook.java:<init>(85)) - Created ATS Hook
2015-01-30 15:25:05,001 INFO  log.PerfLogger (PerfLogger.java:PerfLogBegin(108)) - <PERFLOG method=PreHook.org.apache.hadoop.hive.ql.hooks.ATSHook from=org.apache.hadoop.hive.ql.Driver>
2015-01-30 15:25:05,001 INFO  log.PerfLogger (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=PreHook.org.apache.hadoop.hive.ql.hooks.ATSHook start=1422631505001 end=1422631505001 duration=0 from=org.apache.hadoop.hive.ql.Driver>
2015-01-30 15:25:05,001 INFO  log.PerfLogger (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=TimeToSubmit start=1422631504958 end=1422631505001 duration=43 from=org.apache.hadoop.hive.ql.Driver>
2015-01-30 15:25:05,001 INFO  log.PerfLogger (PerfLogger.java:PerfLogBegin(108)) - <PERFLOG method=runTasks from=org.apache.hadoop.hive.ql.Driver>
2015-01-30 15:25:05,001 INFO  log.PerfLogger (PerfLogger.java:PerfLogBegin(108)) - <PERFLOG method=task.DDL.Stage-0 from=org.apache.hadoop.hive.ql.Driver>
2015-01-30 15:25:05,095 INFO  log.PerfLogger (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=runTasks start=1422631505001 end=1422631505095 duration=94 from=org.apache.hadoop.hive.ql.Driver>

I am running oozie-4.0.0, hive-0.13. Anyone has any ideas?

Upvotes: 4

Views: 8504

Answers (2)

Yogesh Varal
Yogesh Varal

Reputation: 131

This error will be solved if you use database name along with table name while creating or deleting table. For example if database name is hivetempdb then use create statement as- create table hivetempdb.daily_dump(id int) ; and for drop table use statement as DROP Table hivetempdb.daily_dump;

Upvotes: 0

Shinta Smith
Shinta Smith

Reputation: 500

We resolved the issue. It turned out that Hive ran my hive script fine, but it was having troubles creating temporary files in /tmp/hive-yarn. The directory was owned by a person who first ran his script in this cluster. I was running it as my user and did not have permissions to write in that directory.

How we found it out was by going to the actual Hadoop job log associated with the Hive action. The actual error was not propagated properly to Hive logs and Oozie logs. :-(

Upvotes: 2

Related Questions