Pig: Failed to parse: mismatched input 'id' expecting set null

Question

I am using Pig 0.12.1 and have the following Pig code:

C = LOAD '$file' USING myCustomLoader();
D = FOREACH C GENERATE key#id;

I am loading a file using a custom loader. I then want to generate all of the IDs stored in key, a map.

Why am I getting the following error message:

14/06/27 16:56:21 ERROR pig.PigServer: exception during parsing: Error during parsing.     <line 3, column 28>  mismatched input 'id' expecting set null
Failed to parse: <line 3, column 28>  mismatched input 'id' expecting set null

Here is the full stacktrace:

14/06/27 16:56:21 ERROR pig.PigServer: exception during parsing: Error during parsing.     <line 3, column 28>  mismatched input 'id' expecting set null
Failed to parse: <line 3, column 28>  mismatched input 'id' expecting set null
        at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:241)
        at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:179)
        at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1676)
        at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1623)
        at org.apache.pig.PigServer.registerQuery(PigServer.java:575)
        at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:1093)
        at org.apache.pig.pigunit.pig.GruntParser.processPig(GruntParser.java:61)
        at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:501)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198)
        at org.apache.pig.pigunit.pig.PigServer.registerScript(PigServer.java:56)
        at org.apache.pig.pigunit.PigTest.registerScript(PigTest.java:170)
        at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:249)
        at com.testpig.PigTest.testPig(PigTest.java:159)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:601)
        at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
        at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
        at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
        at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
        at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
        at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
        at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
        at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
        at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
        at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
        at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
        at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
        at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:53)
        at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:123)
        at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:104)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:601)
        at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:164)
        at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:110)
        at org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:172)
        at org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcessWhenForked(SurefireStarter.java:104)
        at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:70)
14/06/27 16:56:21 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
14/06/27 16:56:21 INFO compress.CodecPool: Got brand-new compressor

Answer 1

The erroneous code in the question is this:

C = LOAD '$file' USING myCustomLoader();
D = FOREACH C GENERATE key#id;

The correct code is:

C = LOAD '$file' USING myCustomLoader();
D = FOREACH C GENERATE key#'id';

In Pig, to access a map's key, you have to use single quotation marks around the key.

For more information, see Philip (flip) Kromer's post at https://www.mail-archive.com/dev@pig.apache.org/msg24691.html:

Omitting the quotes on the key dereference gives a very unhelpful error message.

{code}
users = FOREACH user_hashes GENERATE info#userid AS userid:chararray;

-- 400   ERROR: ERROR 1200: <file ./foo.pig, line 8, column 42>  [...] 
mismatched input 'userid' expecting set null
{code}

It may be that the user forgot the quotes, or may instead be assuming that Pig 
allows dereferencing a map by the value of an alias or expression:

{code}
users = FOREACH user_hashes GENERATE
  info#'username',               -- works
  info#username,                 -- need quotes around literal
  info#fullref,                  -- no, can't use an alias' value to deref
  info#(CONCAT('user',shortref)) -- and can't use an expression to deref
  ;
{code}