CODEWITHSUNDEEP
hadoopforeachfilterapache-pig
Gurpreet Singh
Gurpreet Singh

Reputation: 197

how to Use Filter operator in foreach in pig latin

I am trying to use FILTER operator within foreach to filter some of my data, but it is throwing an error. I am using the below query:

ctm_pm_v1_stg = LOAD '/datalake/uhc/ei/pi_ara/hive/warehouse/ctm_pm.db/ctm_pm_t1' USING PigStorage ('\u0001') AS (lob:chararray,day_phnno:chararray,eve_phnno:chararray,mbr_name:chararray,hic_no:chararray,contract_no:chararray,ctm_risk_category:chararray);
ctm_pm_v1 = FOREACH ctm_pm_v1_stg {
fdata = FILTER ctm_pm_v1_stg by ctm_risk_category=='High';
GENERATE ctm_risk_category;
};

if anyone have some idea on this please help me. Thanks in advance.

Upvotes: 0

Views: 414

Answers (1)

Sivasakthi Jayaraman
Sivasakthi Jayaraman

Reputation: 4724

Nested FOREACH will work only with bags but your relation(ctm_pm_v1_stg) doesn't have any bags. To fix this issue just remove the nested part and use only FILTER stmt.

ctm_pm_v1_stg = LOAD '/datalake/uhc/ei/pi_ara/hive/warehouse/ctm_pm.db/ctm_pm_t1' USING PigStorage ('\u0001') AS (lob:chararray,day_phnno:chararray,eve_phnno:chararray,mbr_name:chararray,hic_no:chararray,contract_no:chararray,ctm_risk_category:chararray);
fdata = FILTER ctm_pm_v1_stg by (ctm_risk_category=='High');
DUMP fdata;

Upvotes: 3

Related Questions