Strange error on running pigunit sample test

Question

I'm trying to setup the pig unit tests and I was looking into the documentation which they've provided. It seems a bit outdated so I switched to the svn trunk. The first strange thing is that actually it needs some more libs, not only pigunit, pig and hadoop-commons in order to work(add hadoop-hdfs, hadoop-mapreduce-client-core, hadoop-mapreduce-client-jobclient). I'm not sure that having these in my dependency manager is something good but this is not the main problem. So here's the test I'm trying to execute:

 @Test
public void testNtoN() throws ParseException, IOException {
    String[] args = {
                    "n=3",
                    "reducers=1",
                    "input=top_queries_input_data.txt",
                    "output=top_3_queries",
    };
    test = new PigTest("script dir", args);

    String[] output = {
                    "(yahoo,25)",
                    "(facebook,15)",
                    "(twitter,7)",
    };

    test.assertOutput("queries_limit", output);
}

And here's the actual script:

 data =
     LOAD '$input'
     AS (query:CHARARRAY, count:INT);

 queries_group = 
     GROUP data 
     BY query
     PARALLEL $reducers;

 queries_sum = 
     FOREACH queries_group 
     GENERATE 
         group AS query, 
         SUM(data.count) AS count;

 queries_ordered = 
     ORDER queries_sum 
     BY count DESC
     PARALLEL $reducers;

 queries_limit = LIMIT queries_ordered $n;

 STORE queries_limit INTO '$output';

Here's the stacktrace:

 STORE queries_limit INTO 'top_3_queries';
 --> none

org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for alias queries_limit

 at org.apache.pig.PigServer.openIterator(PigServer.java:1019)
 at org.apache.pig.pigunit.PigTest.getAliasFromCache(PigTest.java:224)
 at org.apache.pig.pigunit.PigTest.getActualResults(PigTest.java:319)
 at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:409)
 at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:400)
 at BlaUnitTest.testBla(BlaUnitTest.java:24)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
 at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
 at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
 at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
 at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
 at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
 at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
 at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
 at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
 at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
 at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
 at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
 at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
 at org.mockito.internal.runners.JUnit45AndHigherRunnerImpl.run(JUnit45AndHigherRunnerImpl.java:37)
 at org.mockito.runners.MockitoJUnitRunner.run(MockitoJUnitRunner.java:62)
 at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
 at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:117)
 at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:42)
 at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:262)
 at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:84)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
Caused by: java.io.IOException: Couldn't retrieve job.
 at org.apache.pig.PigServer.store(PigServer.java:1083)
 at org.apache.pig.PigServer.openIterator(PigServer.java:994)
 ... 34 more

I tried to debug it to see what actually happens and this occurs when it tries to build the query plan and acquire the ExecJob but I couldn't figure it out. I even tried to simplify the script and remove everything but the code for loading and storing the data. The result was the same.

Answer 1

I succeeded to resolve the issue. The problem was that I had included some dependencies in the classpath which seems to mess up the correct execution. The only needed dependencies are hadoop-core(I'm using hadoop-aws because I'm using it with aws), hadoop-client, pig and pigunit. So now everything is running correctly.