CODEWITHSUNDEEP
javashellhadoop
user3648335
user3648335

Reputation: 137

Running Hadoop commands in a shell script

I have a simple shell script that I'd like to use to move log files into my Hadoop cluster.

When I invoke the the shell script in my Java program, the script can not be found "Hadoop Commands", and I do not take any action.

For instance, I would like to run the following command:

#!/bin/bash
hadoop fs -put /home/can/workspace2/myWebApp/tweets.txt /user/training/tweetFolder

How can I solve this in a simple manner?

Upvotes: 2

Views: 10009

Answers (2)

Tariq
Tariq

Reputation: 34184

If you have installed Hadoop using some executables like CDH, HDP etc, it should work. But if it's a setup through Apache's tarball you need to use the complete path along with bin/, as bin directory contains all the executable files.

HADOOP_HOME/bin/hadoop fs -put /home/can/workspace2/myWebApp/tweets.txt /user/training/tweetFolder

Also, it's always a good practice to set the HADOOP_HOME variable.

Upvotes: 2

Tiago Lopo
Tiago Lopo

Reputation: 7959

I believe it's a PATH problem. check the path for hadoop with which:

which hadoop

Then put the full path on your script.

Upvotes: 1

Related Questions