CODEWITHSUNDEEP
hadoopmahout
Sreejithc321
Sreejithc321

Reputation: 307

Mahout : output file giving error in Hadoop

I am getting the following error while running Mahout on Hadoop.

Command:

 mahout recommenditembased -s SIMILARITY_LOGLIKELIHOOD -i /user/inp -o /user/op --numRecommendations 5

Error :

Running on hadoop, using /usr/local/hadoop/bin/hadoop and HADOOP_CONF_DIR=
MAHOUT-JOB: /opt/apache-mahout-distribution- 0.11.0/examples/target/mahout-examples-0.11.0-job.jar
15/11/05 17:52:29 INFO AbstractJob: Command line arguments: {--booleanData=[false], --endPhase=[2147483647], --input=[/user/inp], --maxPrefsInItemSimilarity=[500], --maxPrefsPerUser=[10], --maxSimilaritiesPerItem=[100], --minPrefsPerUser=[1], --numRecommendations=[5], --output=[/user/], --similarityClassname=[SIMILARITY_LOGLIKELIHOOD], --startPhase=[0], --tempDir=[temp]}
15/11/05 17:52:29 INFO AbstractJob: Command line arguments: {--booleanData=[false], --endPhase=[2147483647], --input=[/user/inp], --minPrefsPerUser=[1], --output=[temp/preparePreferenceMatrix], --ratingShift=[0.0], --startPhase=[0], --tempDir=[temp]}
15/11/05 17:52:30 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
15/11/05 17:52:30 INFO deprecation: mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
15/11/05 17:52:30 INFO deprecation: mapred.compress.map.output is deprecated. Instead, use mapreduce.map.output.compress
15/11/05 17:52:30 INFO deprecation: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
15/11/05 17:52:30 INFO deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
15/11/05 17:52:30 INFO JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
Exception in thread "main" org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory temp/preparePreferenceMatrix/itemIDIndex already exists

But I couldn't find any output folder created :

hadoop fs -ls /
15/11/05 17:55:40 WARN util.NativeCodeLoader: Unable to load native- hadoop library for your platform... using builtin-java classes where applicable
Found 1 items
drwxr-xr-x   - hduser supergroup          0 2015-11-05 17:47 /user

How to solve this ?

Mahout Version :0.11.0

Upvotes: 0

Views: 781

Answers (1)

Somnath Muluk
Somnath Muluk

Reputation: 57656

I was facing same issue. Found solution to this.

Mahout gives error if output file directory or tempDir already exists. So first delete and then execute command. Also use custom --tempDir option and set temporary directory for output generation. Otherwise it will generate temp folder at location where you are executing command. If you don't want to have custom tempDir then delete temp/ directory before you are executing command.

rm -rf /user/op /tmp/mahout/ && mahout recommenditembased -s SIMILARITY_LOGLIKELIHOOD -i /user/inp -o /user/op --tempDir /tmp/mahout/ --numRecommendations 5

Upvotes: 2

Related Questions