CODEWITHSUNDEEP
javashellhadoop
user5697689
user5697689

Reputation: 11

Hadoop commands written in shell not getting called from java

Have defined hadoop commands from shell and just trying to do normal processing :

The shell file is like :

#!/bin/bash
success = $?

echo " The hadoop execution started at " $(date) ;

echo " Processing the files ";

echo "Starting the process for mapping file to HDFS and then to mysql"

hadoop fs -rm -r -f /FileMappingData && 
hadoop fs -put /home/nishant/data/mappingfile/ /FileMappingData &&

mysql -u root -padmin -e 'use MiningProcessSchema;drop table if exists MiningMasterData;create table MiningMasterData (PointID VARCHAR(20), PointX int, PointY int, FileName VARCHAR(80));' &&

echo "Table created in mysql" &&


if [ "$?" = "0" ] ; then
    echo "Process success"
else 
    echo "------------!!!!!Something wrong !!Process failed!!!!------------";
fi

This goes fine when executed from terminal. When , now I am trying to call the shell from java code . Which is not getting called though . 

StringBuffer output = new StringBuffer();

Process p,p1;
try {
    String[] envp= {"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/home/nishant/bin:/usr/local/java/jdk1.8.0_66/bin:/home/nishant/bin:/usr/local/java/jre1.8.0_66/bin:/usr/lib/jvm/java-7-openjdk-amd64/bin:/usr/local/hadoop/bin:/usr/local/hadoop/sbin:/usr/local/hive/bin:/usr/lib/sqoop/bin"};

    p1 = Runtime.getRuntime().exec("bash exec bash");
    p1.waitFor();

    p = Runtime.getRuntime().exec( command, envp );
    // " > " + prop.getProperty("LOGDIR_PROCESS_DATA")+ sdfRec.format(new Date())
    //    +collectNextVal+".log  2>&1 &"

    //Runtime.getRuntime().
    p.waitFor();
    BufferedReader reader = 
        new BufferedReader(new InputStreamReader(p.getInputStream()));

    String line = "";           
    while ((line = reader.readLine())!= null) {
        output.append(line + "\n");
    }

} catch (Exception e) {
    e.printStackTrace();
}

Output :

The hadoop execution started at Sat Dec 19 14:19:23 IST 2015 Processing the files Starting the process for mapping file to HDFS and then to mysql 111 ------------!!!!!Something wrong !!Process failed!!!!------------

Seems mostly the issue of PATH variable not getting accessed in java executing , but not sure where i can add that .

Please help if in that case or any other.

Thanks in advance.

Upvotes: 0

Views: 292

Answers (1)

user5697689
user5697689

Reputation: 11

Got the answer and successfully running the code now.

The Trick was to include Path variable properly :

String[] envp= {"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/home/nishant/bin:/usr/local/java/jdk1.8.0_66/bin:/home/nishant/bin:/usr/local/java/jre1.8.0_66/bin:/usr/lib/jvm/java-7-openjdk-amd64/bin:/usr/local/hadoop/bin:/usr/local/hadoop/sbin:/usr/local/hive/bin:/usr/lib/sqoop/bin"};

And it worked.

Upvotes: 1

Related Questions