CODEWITHSUNDEEP
hadoopmapreduceapache-pig
nomier
nomier

Reputation: 412

Pig 0.13 error only in mapreduce mode

I'm using cloudera CDH5 with Pig 0.13 ( tried Pig 0.12.1 as well ) and I'm building my own custom loadFunc and I have a simple script that just load using the new loadFunc.

When running in local mode everything works fine and the output is correct.

But when running in mapreduce mode ( still locally ), I get the error:

Unable to open iterator for alias output_hash
    at org.apache.pig.PigServer.openIterator(PigServer.java:880)
    at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:774)
    at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:372)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)
    at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
    at org.apache.pig.Main.run(Main.java:478)
    at org.apache.pig.Main.main(Main.java:156)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: java.io.IOException: Job terminated with anomalous status FAILED
    at org.apache.pig.PigServer.openIterator(PigServer.java:872)
    ... 12 more

I tried setting the properties verbose to true or traces to debug, but didn't didn't get useful info.

Any insights on how to resolve or debug this ?

Upvotes: 1

Views: 257

Answers (1)

nomier
nomier

Reputation: 412

Found the solution. The problem was that I was using maven to build the project and I was building the jar with dependencies. This caused the dependencies that has class names with the same class path to override each other (like FileSystem.java for hadoop-hdfs and hadoop-common and the solution was just to build the jar without including dependencies.

Upvotes: 2

Related Questions