Pig ERROR 1066: Unable to open iterator for alias test

Question

I get:

org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for test

when I have the code:

my_file = LOAD '$my_records_file' USING PigStorage('\t') AS (field0:chararray, field1:int);
test = FILTER my_file BY field0 != null;

Why am I getting this error?

Answer 1

One of the other reasons to get this error is that running Pig through local Mode or HDFS mode. Apart from this error there is a message on our console, failed to read data from the path. If this is the case then check first you are working with Pig in Local mode or HDFS mode. In local mode PIG will take input data from LFS (Local file system). However, in HDFS it will take input data from the HDFS directory.

Backend error message during job submission org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Input path does not exist:

Pig Stack Trace: ERROR 1066: Unable to open iterator for alias

So, be sure while loading the file to avoid this error.

Answer 2

I think you want to filter rows that have null values ? the Syntax is field0 IS NOT NULL.

my_file = LOAD '$my_records_file' USING PigStorage('\t') AS (field0:chararray, field1:int);
test = FILTER my_file BY field0 IS NOT NULL;

but if you want to filter rows that have a value equals to null, the code is :

my_file = LOAD '$my_records_file' USING PigStorage('\t') AS (field0:chararray, field1:int);
test = FILTER my_file BY field0 !='null';

Answer 3

The code:

my_file = LOAD '$my_records_file' USING PigStorage('\t') AS (field0:chararray, field1:int);
test = FILTER my_file BY field0 != null;

has an error.

field0 != null should be field0 != 'null'.

This code:

my_file = LOAD '$my_records_file' USING PigStorage('\t') AS (field0:chararray, field1:int);
test = FILTER my_file BY field0 != 'null';

corrects the error.