ozzie shell script not executing...error Main class [org.apache.oozie.action.hadoop.ShellMain], exit 1

Question

When I am trying to run oozie Job it's throwing following error Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.ShellMain], exit code [1]

My shell script runs fine when run independently.

job.properties

nameNode=hdfs://ns613.mycyberhosting.com:54310
jobTracker=ns613.mycyberhosting.com:8032
queueName=default
user.name=root
oozie.libpath=${nameNode}/user/root/share/lib/lib_20160905172157
oozie.use.system.libpath=true
seeds_shRoot=seeds_sh
oozie.wf.application.path=${nameNode}/user/${user.name}/${seeds_shRoot}/apps/shell

workflow.xml

<?xml version="1.0" encoding="UTF-8"?>
<workflow-app xmlns="uri:oozie:workflow:0.4" name="seeds-shell-wf">
<start to="seed-shell"/>
<action name="seed-shell">
<shell xmlns="uri:oozie:shell-action:0.1">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<exec>job.sh</exec>
<env-var>HADOOP_USER_NAME=root</env-var>
<file>/user/root/seeds_sh/job.sh#job.sh</file>
</shell>
<ok to="end"/>
<error to="kill"/>
</action>
<kill name="kill">
<message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<end name="end"/>
</workflow-app>

script- job.sh

/home/c1/apache-nutch-2.3.1/runtime/deploy/bin/crawl /user/root/seeds_sh/input-data/seeds_test 2 http://ns613.mycyberhosting.com:8983/solr/ddcds 1

plz help to execute this with oozie??

Answer 1

Welcome to oozie world. I am also relatively new :) I faced the same error when executing shell script from oozie.

This error appears when oozie is not able to execute your shell script due to some issue within the script.

I received this error when ozzie was not able to find ini file for my shell script.

Something like below,

cat: /topurl/framework/scripts/shell/cleanup.ini: No such file or directory
  File "<string>", line 1
    import datetime;print int(datetime.datetime.now().strftime('%s'))-
                                                                     ^
SyntaxError: invalid syntax
  File "<string>", line 1
    import datetime;print int(datetime.datetime.strptime('
                                                         ^
SyntaxError: EOL while scanning string literal
rm: `.Trash': Is a directory
Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.ShellMain], exit code [1]
log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.impl.MetricsSystemImpl).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

What you should do is check the log. Follow below steps to find the correct log,

• go to oozie webconsole - usually like - http://hostname:11000/oozie/
• click on the workflow which falied.
• Open the action which failed.

• When the tab opens up, on the right side of the console URL option you will find a search button. Click on it.

  You will be now directed to Mapreduce log.

• Click on Task type Map
• Click on the name some thing like - task_1473079044012_4249_m_000000
• Then click on the logs You will find all the logs in the page opened.

You should see what is stopping your shell to run within oozie.

Let me know if you need more information or pictures.